LogoLogo
v0.30-branch
v0.30-branch
  • Introduction
  • Community & getting help
  • Roadmap
  • Changelog
  • Getting started
    • Quickstart
    • Concepts
      • Overview
      • Data ingestion
      • Entity
      • Feature view
      • Feature retrieval
      • Point-in-time joins
      • Registry
      • [Alpha] Saved dataset
    • Architecture
      • Overview
      • Registry
      • Offline store
      • Online store
      • Batch Materialization Engine
      • Provider
    • Third party integrations
    • FAQ
  • Tutorials
    • Sample use-case tutorials
      • Driver ranking
      • Fraud detection on GCP
      • Real-time credit scoring on AWS
      • Driver stats on Snowflake
    • Validating historical features with Great Expectations
    • Using Scalable Registry
    • Building streaming features
  • How-to Guides
    • Running Feast with Snowflake/GCP/AWS
      • Install Feast
      • Create a feature repository
      • Deploy a feature store
      • Build a training dataset
      • Load data into the online store
      • Read features from the online store
      • Scaling Feast
      • Structuring Feature Repos
    • Running Feast in production (e.g. on Kubernetes)
    • Upgrading for Feast 0.20+
    • Customizing Feast
      • Adding a custom batch materialization engine
      • Adding a new offline store
      • Adding a new online store
      • Adding a custom provider
    • Adding or reusing tests
  • Reference
    • Codebase Structure
    • Type System
    • Data sources
      • Overview
      • File
      • Snowflake
      • BigQuery
      • Redshift
      • Push
      • Kafka
      • Kinesis
      • Spark (contrib)
      • PostgreSQL (contrib)
      • Trino (contrib)
      • Azure Synapse + Azure SQL (contrib)
    • Offline stores
      • Overview
      • File
      • Snowflake
      • BigQuery
      • Redshift
      • Spark (contrib)
      • PostgreSQL (contrib)
      • Trino (contrib)
      • Azure Synapse + Azure SQL (contrib)
    • Online stores
      • Overview
      • SQLite
      • Snowflake
      • Redis
      • Datastore
      • DynamoDB
      • Bigtable
      • PostgreSQL (contrib)
      • Cassandra + Astra DB (contrib)
      • MySQL (contrib)
      • Rockset (contrib)
    • Providers
      • Local
      • Google Cloud Platform
      • Amazon Web Services
      • Azure
    • Batch Materialization Engines
      • Bytewax
      • Snowflake
      • AWS Lambda (alpha)
      • Spark (contrib)
    • Feature repository
      • feature_store.yaml
      • .feastignore
    • Feature servers
      • Python feature server
      • [Alpha] Go feature server
      • [Alpha] AWS Lambda feature server
    • [Beta] Web UI
    • [Alpha] On demand feature view
    • [Alpha] Data quality monitoring
    • Feast CLI reference
    • Python API reference
    • Usage
  • Project
    • Contribution process
    • Development guide
    • Backwards Compatibility Policy
      • Maintainer Docs
    • Versioning policy
    • Release process
    • Feast 0.9 vs Feast 0.10+
Powered by GitBook
On this page
  • Overview
  • CLI
  • Installation
  • Alpha features
  • Feature logging
  • Python SDK retrieval

Was this helpful?

Edit on GitHub
Export as PDF
  1. Reference
  2. Feature servers

[Alpha] Go feature server

PreviousPython feature serverNext[Alpha] AWS Lambda feature server

Last updated 2 years ago

Was this helpful?

Overview

The Go feature server is an HTTP/gRPC endpoint that serves features. It is written in Go, and is therefore significantly faster than the Python feature server. See this for more details on the comparison between Python and Go. In general, we recommend the Go feature server for all production use cases that require extremely low-latency feature serving. Currently only the Redis and SQLite online stores are supported.

CLI

By default, the Go feature server is turned off. To turn it on you can add go_feature_serving: True to your feature_store.yaml:

feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: redis
  connection_string: "localhost:6379"
go_feature_serving: True

Then the feast serve CLI command will start the Go feature server. As with Python, the Go feature server uses port 6566 by default; the port be overridden with a --port flag. Moreover, the server uses HTTP by default, but can be set to use gRPC with --type=grpc.

Alternatively, if you wish to experiment with the Go feature server instead of permanently turning it on, you can just run feast serve --go.

Installation

The Go component comes pre-compiled when you install Feast with Python versions 3.8-3.10 on macOS or Linux (on x86). In order to install the additional Python dependencies, you should install Feast with

pip install feast[go]

For macOS, run brew install apache-arrow. For linux users, you have to install libarrow-dev.

sudo apt update
sudo apt install -y -V ca-certificates lsb-release wget
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
sudo apt update
sudo apt install -y -V libarrow-dev # For C++

For developers, if you want to build from source, run make compile-go-lib to build and compile the go server. In order to build the go binaries, you will need to install the apache-arrow c++ libraries.

Alpha features

Feature logging

The Go feature server can log all requested entities and served features to a configured destination inside an offline store. This allows users to create new datasets from features served online. Those datasets could be used for future trainings or for feature validations. To enable feature logging we need to edit feature_store.yaml:

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: redis
  connection_string: "localhost:6379"
go_feature_serving: True
feature_server:
  feature_logging:
    enabled: True

Feature logging configuration in feature_store.yaml also allows to tweak some low-level parameters to achieve the best performance:

feature_server:
  feature_logging:
    enabled: True
    flush_interval_secs: 300
    write_to_disk_interval_secs: 30
    emit_timeout_micro_secs: 10000
    queue_capacity: 10000

All these parameters are optional.

Python SDK retrieval

The logic for the Go feature server can also be used to retrieve features during a Python get_online_features call. To enable this behavior, you must add go_feature_retrieval: True to your feature_store.yaml. You must also have all the dependencies installed as detailed above.

You must also install the Apache Arrow C++ libraries. This is because the Go feature server uses the cgo memory allocator from the Apache Arrow C++ library for interoperability between Go and Python, to prevent memory from being accidentally garbage collected when executing on-demand feature views. You can read more about the usage of the cgo memory allocator in these .

blog post
docs