arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Azure AKS (with Terraform)

hashtag
Overview

This guide installs Feast on Azure using our reference Terraform configurationarrow-up-right.

circle-info

The Terraform configuration used here is a greenfield installation that neither assumes anything about, nor integrates with, existing resources in your Azure account. The Terraform configuration presents an easy way to get started, but you may want to customize this set up before using Feast in production.

This Terraform configuration creates the following resources:

  • Kubernetes cluster on Azure AKS

  • Kafka managed by HDInsight

  • Postgres database for Feast metadata, running as a pod on AKS

hashtag
1. Requirements

  • Create an Azure account and

  • Install (tested with 0.13.5)

  • Install (tested with v3.4.2)

hashtag
2. Configure Terraform

Create a .tfvars file underfeast/infra/terraform/azure. Name the file. In our example, we use my_feast.tfvars. You can see the full list of configuration variables in variables.tf. At a minimum, you need to set name_prefix and resource_group:

hashtag
3. Apply

After completing the configuration, initialize Terraform and apply:

hashtag
4. Connect to Feast using Jupyter

After all pods are running, connect to the Jupyter Notebook Server running in the cluster.

To connect to the remote Feast server you just created, forward a port from the remote k8s cluster to your local machine.

You can now connect to the bundled Jupyter Notebook Server at localhost:8888 and follow the example Jupyter notebook.

Redis cluster, using Azure Cache for Redis
  • to run Spark

  • Staging Azure blob storage container to store temporary data

  • configure credentials locallyarrow-up-right
    Terraformarrow-up-right
    Helmarrow-up-right
    my_feast.tfvars
    name_prefix = "feast"
    resource_group = "Feast" # pre-existing resource group
    $ cd feast/infra/terraform/azure
    $ terraform init
    $ terraform apply -var-file=my_feast.tfvars
    kubectl port-forward $(kubectl get pod -o custom-columns=:metadata.name | grep jupyter) 8888:8888
    Forwarding from 127.0.0.1:8888 -> 8888
    Forwarding from [::1]:8888 -> 8888
    spark-on-k8s-operatorarrow-up-right