Amazon EKS (with Terraform)
Last updated
Was this helpful?
Last updated
Was this helpful?
This guide installs Feast on AWS using our .
This Terraform configuration creates the following resources:
Kubernetes cluster on Amazon EKS (3x r3.large nodes)
Kafka managed by Amazon MSK (2x kafka.t3.small nodes)
Postgres database for Feast metadata, using serverless Aurora (min capacity: 2)
Redis cluster, using Amazon Elasticache (1x cache.t2.micro)
Amazon EMR cluster to run Spark (3x spot m4.xlarge)
Staging S3 bucket to store temporary data
Create a .tfvars
file underfeast/infra/terraform/aws
. Name the file. In our example, we use my_feast.tfvars
. You can see the full list of configuration variables in variables.tf
. At a minimum, you need to set name_prefix
and an AWS region:
After completing the configuration, initialize Terraform and apply:
Starting may take a minute. A kubectl configuration file is also created in this directory, and the file's name will start with kubeconfig_
and end with a random suffix.
After all pods are running, connect to the Jupyter Notebook Server running in the cluster.
To connect to the remote Feast server you just created, forward a port from the remote k8s cluster to your local machine. Replace kubeconfig_XXXXXXX
below with the kubeconfig file name Terraform generates for you.
You can now connect to the bundled Jupyter Notebook Server at localhost:8888
and follow the example Jupyter notebook.
Create an AWS account and
Install > = 0.12 (tested with 0.13.3)
Install (tested with v3.3.4)