Kubernetes (GKE)

This guide will install Feast into a Kubernetes cluster on Google Cloud Platform. It assumes that all of your services will run within a single Kubernetes cluster.

Kubernetes deployment is recommended when deploying Feast as a shared service for production workloads.

This guide requires Google Cloud Platform for installation.

0. Requirements

  1. Google Cloud SDK installed, authenticated, and configured to the project you will use.

  2. Kubectl installed and configured.

  3. Helm 3 installed.

1. Set up Google Cloud Platform

Historical Serving currently requires Google Cloud Platform (GCP) to function, specifically a Service Account with access to Google Cloud Storage (GCS) and BigQuery.

Create a service account for Feast to use:

gcloud iam service-accounts create feast-service-account
gcloud projects add-iam-policy-binding my-gcp-project \
--member serviceAccount:feast-service-account@my-gcp-project.iam.gserviceaccount.com \
--role roles/editor
# Please use "credentials.json" as the file name
gcloud iam service-accounts keys create credentials.json --iam-account \
feast-service-account@my-gcp-project.iam.gserviceaccount.com

Create a Google Cloud Storage bucket for Feast to stage batch data exports:

gsutil mb gs://my-feast-staging-bucket

Create a BigQuery dataset for Feast to store historical data:

bq --location=US mk --dataset my_project:feast

2. Set up a Kubernetes (GKE) cluster

Create a Kubernetes cluster:

gcloud container clusters create feast-cluster \
--machine-type n1-standard-4 \
--zone us-central1-a \
--scopes=bigquery,storage-rw,compute-ro

Create a secret in the GKE cluster from the service account credentials.json:

kubectl create secret generic feast-gcp-service-account \
--from-file=credentials.json

3. Install Feast with Helm

Add the Feast Helm repository and download the latest charts:

helm repo add feast-charts https://feast-charts.storage.googleapis.com
helm repo update

Create secret to store the password for PostgreSQL:

kubectl create secret generic feast-postgresql \
--from-literal=postgresql-password=password

Create a values.yml file to configure your Feast deployment

curl https://raw.githubusercontent.com/feast-dev/feast/master/infra/charts/feast/values-batch-serving.yaml \
> values.yaml

Update application-values.ymlin values.yaml to configure Feast based on your GCP and GKE environment. Minimally the following properties must be set under feast.stores[].config:

Property

Description

project_id

This is your GCP Project Id.

dataset_id

This is the dataset name of the BigQuery dataset to use.

staging_location

This is the GCS bucket used for staging data being loaded into BigQuery.

Install the Feast Helm chart to deploy Feast:

helm install feast-release -f values.yaml feast-charts/feast

Wait for the Feast pods to start running and become become ready:

watch kubectl get pods

5. Connect to Feast using Jupyter

Once the pods are all running we can connect to the Jupyter notebook server running in the cluster

kubectl port-forward \
$(kubectl get pod -o custom-columns=:metadata.name | grep jupyter) 8888:8888
Forwarding from 127.0.0.1:8888 -> 8888
Forwarding from [::1]:8888 -> 8888

You should be able to connect at localhost:8888 to the bundled Jupyter Notebook Server with example notebooks.

6. Further Reading