Feast's architecture is designed to be flexible and scalable. It is composed of several components that work together to provide a feature store that can be used to serve features for training and inference.
Feast uses a Push Model to ingest data from different sources and store feature values in the online store. This allows Feast to serve features in real-time with low latency.
Feast supports feature transformation for On Demand and Streaming data sources and will support Batch transformations in the future. For Streaming and Batch data sources, Feast requires a separate Feature Transformation Engine (in the batch case, this is typically your Offline Store). We are exploring adding a default streaming engine to Feast.
We recommend using Python for your Feature Store microservice. As mentioned in the document, precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable. Because of this we believe the pros of Python outweigh the costs, as reimplementing feature logic is undesirable. Java and Go Clients are also available for online feature retrieval.
Role-Based Access Control (RBAC) is a security mechanism that restricts access to resources based on the roles/groups/namespaces of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups can access or modify specific resources, thereby maintaining data security and operational integrity.
Push vs Pull Model
Push vs Pull Model
Feast uses a Push Model, i.e., Data Producers push data to the feature store and Feast stores the feature values in the online store, to serve features in real-time.
In a Pull Model, Feast would pull data from the data producers at request time and store the feature values in the online store before serving them (storing them would actually be unnecessary). This approach would incur additional network latency as Feast would need to orchestrate a request to each data producer, which would mean the latency would be at least as long as your slowest call. So, in order to serve features as fast as possible, we push data to Feast and store the feature values in the online store.
The trade-off with the Push Model is that strong consistency is not guaranteed out of the box. Instead, strong consistency has to be explicitly designed for in orchestrating the updates to Feast and the client usage.
The significant advantage with this approach is that Feast is read-optimized for low-latency feature retrieval.
How to Push
Implicit in the Push model are decisions about how and when to push feature values to the online store.
From a developer's perspective, there are three ways to push feature values to the online store with different tradeoffs.
They are discussed further in the Write Patterns section.
Feature Serving and Model Inference
Note: this ML Infrastructure diagram highlights an orchestration pattern that is driven by a client application. This is not the only approach that can be taken and different patterns will result in different trade-offs.
Production machine learning systems can choose from four approaches to serving machine learning predictions (the output of model inference):
Online model inference with online features
Offline mode inference without online features
Online model inference with online features and cached predictions
Online model inference without features
Note: online features can be sourced from batch, streaming, or request data sources.
These three approaches have different tradeoffs but, in general, have significant implementation differences.
1. Online Model Inference with Online Features
Online model inference with online features is a powerful approach to serving data-driven machine learning applications. This requires a feature store to serve online features and a model server to serve model predictions (e.g., KServe). This approach is particularly useful for applications where request-time data is required to run inference.
2. Offline Model Inference without Online Features
Typically, Machine Learning teams find serving precomputed model predictions to be the most straightforward to implement. This approach simply treats the model predictions as a feature and serves them from the feature store using the standard Feast sdk. These model predictions are typically generated through some batch process where the model scores are precomputed. As a concrete example, the batch process can be as simple as a script that runs model inference locally for a set of users that can output a CSV. This output file could be used for materialization so that the model could be served online as shown in the code below.
Notice that the model server is not involved in this approach. Instead, the model predictions are precomputed and materialized to the online store.
While this approach can lead to quick impact for different business use cases, it suffers from stale data as well as only serving users/entities that were available at the time of the batch computation. In some cases, this tradeoff may be tolerable.
3. Online Model Inference with Online Features and Cached Predictions
This approach is the most sophisticated where inference is optimized for low-latency by caching predictions and running model inference when data producers write features to the online store. This approach is particularly useful for applications where features are coming from multiple data sources, the model is computationally expensive to run, or latency is a significant constraint.
Note that in this case a seperate call to write_to_online_store is required when the underlying data changes and predictions change along with it.
While this requires additional writes for every data producer, this approach will result in the lowest latency for model inference.
4. Online Model Inference without Features
This approach does not require Feast. The model server can directly serve predictions without any features. This approach is common in Large Language Models (LLMs) and other models that do not require features to make predictions.
Note that generative models using Retrieval Augmented Generation (RAG) do require features where the document embeddings are treated as features, which Feast supports (this would fall under "Online Model Inference with Online Features").
Client Orchestration
Implicit in the code examples above is a design choice about how clients orchestrate calls to get features and run model inference. The examples had a Feast-centric pattern because they are inputs to the model, so the sequencing is fairly obvious. An alternative approach can be Inference-centric where a client would call an inference endpoint and the inference service would be responsible for orchestration.
[Alpha] Saved dataset
Feast datasets allow for conveniently saving dataframes that include both features and entities to be subsequently used for data analysis and model training. Data Quality Monitoring was the primary motivation for creating dataset concept.
Dataset's metadata is stored in the Feast registry and raw data (features, entities, additional input keys and timestamp) is stored in the offline store.
Dataset can be created from:
Results of historical retrieval
[planned] Logging request (including input for on demand transformation) and response during feature serving
[planned] Logging features during writing to online store (from batch source or stream)
Creating a saved dataset from historical retrieval
To create a saved dataset from historical features for later retrieval or analysis, a user needs to call get_historical_features method first and then pass the returned retrieval job to create_saved_dataset method. create_saved_dataset will trigger the provided retrieval job (by calling .persist() on it) to store the data using the specified storage behind the scenes. Storage type must be the same as the globally configured offline store (e.g it's impossible to persist data to a different offline source). create_saved_dataset will also create a SavedDataset object with all of the related metadata and will write this object to the registry.
Saved dataset can be retrieved later using the get_saved_dataset method in the feature store:
Tags in Feast allow for efficient filtering of Feast objects when listing them in the UI, CLI, or querying the registry directly.
The way to define tags on the feast objects is through the definition file or directly in the object that will be applied to the feature store.
Examples
In this example we define a Feature View in a definition file that has a tag:
driver_stats_fv = FeatureView(
name="driver_hourly_stats",
entities=[driver],
ttl=timedelta(days=1),
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
Field(name="avg_daily_trips", dtype=Int64, description="Average daily trips"),
],
online=True,
source=driver_stats_source,
# Tags are user defined key/value pairs that are attached to each
# feature view
tags={"team": "driver_performance"},
)
In this example we define a Stream Feature View that has a tag, in the code:
An example of filtering feature-views with the tag team:driver_performance:
$ feast feature-views list --tags team:driver_performance
NAME ENTITIES TYPE
driver_hourly_stats {'driver'} FeatureView
driver_hourly_stats_fresh {'driver'} FeatureView
The same example of listing feature-views without tag filtering:
Use Cases
This page covers common use cases for Feast and how a feature store can benefit your AI/ML workflows.
Recommendation Engines
Recommendation engines require personalized feature data related to users, items, and their interactions. Feast can help by:
Managing feature data: Store and serve user preferences, item characteristics, and interaction history
Low-latency serving: Provide real-time features for dynamic recommendations
Point-in-time correctness: Ensure training and serving data are consistent to avoid data leakage
Feature reuse: Allow different recommendation models to share the same feature definitions
Example: User-Item Recommendations
A typical recommendation engine might need features such as:
User features: demographics, preferences, historical behavior
Interaction features: past user-item interactions, ratings
Feast allows you to define these features once and reuse them across different recommendation models, ensuring consistency between training and serving environments.
Risk scorecards (such as credit risk, fraud risk, and marketing propensity models) require a comprehensive view of entity data with historical contexts. Feast helps by:
Feature consistency: Ensure all models use the same feature definitions
Historical feature retrieval: Generate training datasets with correct point-in-time feature values
Feature monitoring: Track feature distributions to detect data drift
Governance: Maintain an audit trail of features used in regulated environments
Example: Credit Risk Scoring
Credit risk models might use features like:
Transaction history patterns
Account age and status
Payment history features
External credit bureau data
Employment and income verification
Feast enables you to combine these features from disparate sources while maintaining data consistency and freshness.
Natural Language Processing (NLP) and Retrieval Augmented Generation (RAG) applications require efficient storage and retrieval of text embeddings. Feast supports these use cases by:
Vector storage: Store and index embedding vectors for efficient similarity search
Document metadata: Associate embeddings with metadata for contextualized retrieval
Scaling retrieval: Serve vectors with low latency for real-time applications
Versioning: Track changes to embedding models and document collections
Example: Retrieval Augmented Generation
RAG systems can leverage Feast to:
Store document embeddings and chunks in a vector database
Retrieve contextually relevant documents for user queries
Combine document retrieval with entity-specific features
Scale to large document collections
Feast makes it remarkably easy to make data available for retrieval by providing a simple API for both storing and querying vector embeddings.
Time series forecasting for demand planning, inventory management, and anomaly detection benefits from Feast through:
Temporal feature management: Store and retrieve time-bound features
Feature engineering: Create time-based aggregations and transformations
Consistent feature retrieval: Ensure training and inference use the same feature definitions
Backfilling capabilities: Generate historical features for model training
Example: Demand Forecasting
Demand forecasting applications typically use features such as:
Historical sales data with temporal patterns
Seasonal indicators and holiday flags
Weather data
Price changes and promotions
External economic indicators
Feast allows you to combine these diverse data sources and make them available for both batch training and online inference.
Image and Multi-Modal Processing
While Feast was initially built for structured data, it can also support multi-modal applications by:
Storing feature metadata: Keep track of image paths, embeddings, and metadata
Vector embeddings: Store image embeddings for similarity search
Feature fusion: Combine image features with structured data features
Why Feast Is Impactful
Across all these use cases, Feast provides several core benefits:
Consistency between training and serving: Eliminate training-serving skew by using the same feature definitions
Feature reuse: Define features once and use them across multiple models
Scalable feature serving: Serve features at low latency for production applications
Feature governance: Maintain a central registry of feature definitions with metadata
Data freshness: Keep online features up-to-date with batch and streaming ingestion
Reduced operational complexity: Standardize feature access patterns across models
By implementing a feature store with Feast, teams can focus on model development rather than data engineering challenges, accelerating the delivery of ML applications to production.
The Feast feature registry is a central catalog of all feature definitions and their related metadata. Feast uses the registry to store all applied Feast objects (e.g. Feature views, entities, etc). It allows data scientists to search, discover, and collaborate on new features. The registry exposes methods to apply, list, retrieve and delete these objects, and is an abstraction with multiple implementations.
Feast comes with built-in file-based and sql-based registry implementations. By default, Feast uses a file-based registry, which stores the protobuf representation of the registry as a serialized file in the local file system. For more details on which registries are supported, please see Registries.
Updating the registry
We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD automatically stays synced with the registry. Users will often also want multiple registries to correspond to different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write access since they can impact real user traffic. See Running Feast in Production for details on how to set this up.
Deleting objects from the registry
Simply removing a feature definition from your code and running feast apply or FeatureStore.apply()does not delete the object from the registry. You must explicitly delete objects using the dedicated delete methods or CLI commands.
Using the CLI
The simplest way to delete objects is using the feast delete command:
To delete objects programmatically, use the explicit delete methods provided by the FeatureStore class:
Deleting feature views
Deleting feature services
Deleting entities, data sources, and other objects
For entities, data sources, and other registry objects, you can use the registry methods directly:
When using feast apply via the CLI, you can also use the objects_to_delete parameter with partial=False to delete objects as part of the apply operation. However, this is less common and typically used in automated deployment scenarios.
Accessing the registry from clients
Users can specify the registry through a feature_store.yaml config file, or programmatically. We often see teams preferring the programmatic approach because it makes notebook driven development very easy:
Option 1: programmatically specifying the registry
Option 2: specifying the registry in the project's feature_store.yaml file
Instantiating a FeatureStore object can then point to this:
The file-based feature registry is a Protobuf representation of Feast metadata. This Protobuf file can be read programmatically from other programming languages, but no compatibility guarantees are made on the internal structure of the registry.
Offline store
An offline store is an interface for working with historical time-series feature values that are stored in data sources. The OfflineStore interface has several different implementations, such as the BigQueryOfflineStore, each of which is backed by a different storage and compute engine. For more details on which offline stores are supported, please see Offline Stores.
Offline stores are primarily used for two reasons:
Building training datasets from time-series features.
Materializing (loading) features into an online store to serve those features at low-latency in a production setting.
Offline stores are configured through the feature_store.yaml. When building training datasets or materializing features into an online store, Feast will use the configured offline store with your configured data sources to execute the necessary data operations.
Only a single offline store can be used at a time. Moreover, offline stores are not compatible with all data sources; for example, the BigQuery offline store cannot be used to query a file-based data source.
Please see Push Source for more details on how to push features directly to the offline store in your feature store.
Compute Engine
Note: The materialization is now constructed via unified compute engine interface.
A Compute Engine in Feast is a component that handles materialization and historical retrieval tasks. It is responsible for executing the logic defined in feature views, such as aggregations, transformations, and custom user-defined functions (UDFs).
A materialization task abstracts over specific technologies or frameworks that are used to materialize data. It allows users to use a pure local serialized approach (which is the default LocalComputeEngine), or delegates the materialization to seperate components (e.g. AWS Lambda, as implemented by the the LambdaComputeEngine).
If the built-in engines are not sufficient, you can create your own custom materialization engine. Please see this guide for more details.
Batch Engine Config can be configured in the feature_store.yaml file, and it serves as the default configuration for all materialization and historical retrieval tasks. The batch_engine config in BatchFeatureView. E.g
in BatchFeatureView.
Then, when you materialize the feature view, it will use the batch_engine configuration specified in the feature view, which has shuffle partitions set to 200 and executor memory set to 8g.
Stream Engine
Stream Engine Config can be configured in the feature_store.yaml file, and it serves as the default configuration for all stream materialization and historical retrieval tasks. The stream_engine config in FeatureView. E.g
Then, when you materialize the feature view, it will use the stream_engine configuration specified in the feature view, which has shuffle partitions set to 200 and executor memory set to 8g.
API
The compute engine builds the execution plan in a DAG format named FeatureBuilder. It derives feature generation from Feature View definitions including:
Components
The compute engine is responsible for executing the materialization and retrieval tasks defined in the feature views. It builds a directed acyclic graph (DAG) of operations that need to be performed to generate the features. The Core components of the compute engine are:
Feature Builder
The Feature builder is responsible for resolving the features from the feature views and executing the operations defined in the DAG. It handles the execution of transformations, aggregations, joins, and filters.
Feature Resolver
The Feature resolver is the core component of the compute engine that constructs the execution plan for feature generation. It takes the definitions from feature views and builds a directed acyclic graph (DAG) of operations that need to be performed to generate the features.
DAG
The DAG represents the directed acyclic graph of operations that need to be performed to generate the features. It contains nodes for each operation, such as transformations, aggregations, joins, and filters. The DAG is built by the Feature Resolver and executed by the Feature Builder.
DAG nodes are defined as follows:
Retrieval Augmented Generation (RAG) with Feast
This tutorial demonstrates how to use Feast with Docling and Milvus to build a Retrieval Augmented Generation (RAG) application. You'll learn how to store document embeddings in Feast and retrieve the most relevant documents for a given query.
RAG is a technique that combines generative models (e.g., LLMs) with retrieval systems to generate contextually relevant output for a particular goal (e.g., question and answering). Feast makes it easy to store and retrieve document embeddings for RAG applications by providing integrations with vector databases like Milvus.
The typical RAG process involves:
Sourcing text data relevant for your application
Transforming each text document into smaller chunks of text
Transforming those chunks of text into embeddings
Inserting those chunks of text along with some identifier for the chunk and document in a database
Retrieving those chunks of text along with the identifiers at run-time to inject that text into the LLM's context
Calling some API to run inference with your LLM to generate contextually relevant output
Returning the output to some end user
Prerequisites
Python 3.10 or later
Feast installed with Milvus support: pip install feast[milvus, nlp]
A basic understanding of feature stores and vector embeddings
Step 0: Download, Compute, and Export the Docling Sample Dataset
Step 1: Configure Milvus in Feast
Create a feature_store.yaml file with the following configuration:
Step 2: Define your Data Sources and Views
Create a feature_repo.py file to define your entities, data sources, and feature views:
Step 3: Update your Registry
Apply the feature view definitions to the registry:
Step 4: Ingest your Data
Process your documents, generate embeddings, and ingest them into the Feast online store:
Step 5: Retrieve Relevant Documents
Now you can retrieve the most relevant documents for a given query:
Step 6: Use Retrieved Documents for Generation
Finally, you can use the retrieved documents as context for an LLM:
Alternative: Using DocEmbedder for Simplified Ingestion
Instead of manually chunking, embedding, and writing documents as shown above, you can use Feast's DocEmbedder class to handle the entire pipeline in a single step. DocEmbedder automates chunking, embedding generation, FeatureView creation, and writing to the online store.
Install Dependencies
Set Up and Ingest with DocEmbedder
Retrieve and Query
Once documents are ingested, you can retrieve them the same way as shown in Step 5 above:
Customizing the Pipeline
DocEmbedder is extensible at every stage. Below are examples of how to create custom components and wire them together.
Custom Chunker
Subclass BaseChunker to implement your own chunking strategy. The load_parse_and_chunk method receives each document and must return a list of chunk dictionaries.
Or simply configure the built-in TextChunker:
Custom Embedder
Subclass BaseEmbedder to use a different embedding model. Register modality handlers in _register_default_modalities and implement the embed method.
Custom Logical Layer Function
The schema transform function transforms the chunked + embedded DataFrame into the exact schema your FeatureView expects. It must accept a pd.DataFrame and return a pd.DataFrame.
Putting It All Together
Pass your custom components to DocEmbedder:
Note: When using a custom schema_transform_fn, ensure the returned DataFrame columns match your FeatureView schema. When using a custom embedder with a different output dimension, set vector_length accordingly (or let it auto-detect via get_embedding_dim).
Install Feast with Snowflake dependencies (required when using Snowflake):
pip install 'feast[snowflake]'
Install Feast with GCP dependencies (required when using BigQuery or Firestore):
pip install 'feast[gcp]'
Install Feast with AWS dependencies (required when using Redshift or DynamoDB):
pip install 'feast[aws]'
Install Feast with Redis dependencies (required when using Redis, either through AWS Elasticache or independently):
Deploy a feature store
The Feast CLI can be used to deploy a feature store to your infrastructure, spinning up any necessary persistent resources like buckets or tables in data stores. The deployment target and effects depend on the provider that has been configured in your feature_store.yaml file, as well as the feature definitions found in your feature repository.
Here we'll be using the example repository we created in the previous guide, Create a feature store. You can re-create it by running feast init in a new directory.
Deploying
To have Feast deploy your infrastructure, run feast apply from your command line while inside a feature repository:
Depending on whether the feature repository is configured to use a local provider or one of the cloud providers like GCP or AWS, it may take from a couple of seconds to a minute to run to completion.
At this point, no data has been materialized to your online store. Feast apply simply registers the feature definitions with Feast and spins up any necessary infrastructure such as tables. To load data into the online store, run feast materialize. See Load data into the online store for more details.
Cleaning up
If you need to clean up the infrastructure created by feast apply, use the teardown command.
Warning: teardown is an irreversible command and will remove all feature store infrastructure. Proceed with caution!
****
Build a training dataset
Feast allows users to build a training dataset from time-series feature data that already exists in an offline store. Users are expected to provide a list of features to retrieve (which may span multiple feature views), and a dataframe to join the resulting features onto. Feast will then execute a point-in-time join of multiple feature views onto the provided dataframe, and return the full resulting dataframe.
Retrieving historical features
1. Register your feature views
Please ensure that you have created a feature repository and that you have registered (applied) your feature views with Feast.
Start by defining the feature references (e.g., driver_trips:average_daily_rides) for the features that you would like to retrieve from the offline store. These features can come from multiple feature tables. The only requirement is that the feature tables that make up the feature references have the same entity (or composite entity), and that they aren't located in the same offline store.
3. Create an entity dataframe
An entity dataframe is the target dataframe on which you would like to join feature values. The entity dataframe must contain a timestamp column called event_timestamp and all entities (primary keys) necessary to join feature tables onto. All entities found in feature views that are being joined onto the entity dataframe must be found as column on the entity dataframe.
It is possible to provide entity dataframes as either a Pandas dataframe or a SQL query.
Pandas:
In the example below we create a Pandas based entity dataframe that has a single row with an event_timestamp column and a driver_id entity column. Pandas based entity dataframes may need to be uploaded into an offline store, which may result in longer wait times compared to a SQL based entity dataframe.
SQL (Alternative):
Below is an example of an entity dataframe built from a BigQuery SQL query. It is only possible to use this query when all feature views being queried are available in the same offline store (BigQuery).
4. Launch historical retrieval
Once the feature references and an entity dataframe are defined, it is possible to call get_historical_features(). This method launches a job that executes a point-in-time join of features from the offline store onto the entity dataframe. Once completed, a job reference will be returned. This job reference can then be converted to a Pandas dataframe by calling to_df().
Load data into the online store
Feast allows users to load their feature data into an online store in order to serve the latest features to models for online prediction.
Materializing features
1. Register feature views
Before proceeding, please ensure that you have applied (registered) the feature views that should be materialized.
The materialize command allows users to materialize features over a specific historical time range into the online store.
The above command will query the batch sources for all feature views over the provided time range, and load the latest feature values into the configured online store.
It is also possible to materialize for specific feature views by using the -v / --views argument.
The materialize command is completely stateless. It requires the user to provide the time ranges that will be loaded into the online store. This command is best used from a scheduler that tracks state, like Airflow.
2.b Materialize Incremental (Alternative)
For simplicity, Feast also provides a materialize command that will only ingest new data that has arrived in the offline store. Unlike materialize, materialize-incremental will track the state of previous ingestion runs inside of the feature registry.
The example command below will load only new data that has arrived for each feature view up to the end date and time (2021-04-08T00:00:00).
The materialize-incremental command functions similarly to materialize in that it loads data over a specific time range for all feature views (or the selected feature views) into the online store.
Unlike materialize, materialize-incremental automatically determines the start time from which to load features from batch sources of each feature view. The first time materialize-incremental is executed it will set the start time to the oldest timestamp of each data source, and the end time as the one provided by the user. For each run of materialize-incremental, the end timestamp will be tracked.
Subsequent runs of materialize-incremental will then set the start time to the end time of the previous run, thus only loading new data that has arrived into the online store. Note that the end time that is tracked for each run is at the feature view level, not globally for all feature views, i.e, different feature views may have different periods that have been materialized into the online store.
Adding a new online store
Overview
Feast makes adding support for a new online store (database) easy. Developers can simply implement the OnlineStore interface to add support for a new store (other than the existing stores like Redis, DynamoDB, SQLite, and Datastore).
In this guide, we will show you how to integrate with MySQL as an online store. While we will be implementing a specific store, this guide should be representative for adding support for any new online store.
The process of using a custom online store consists of 6 steps:
Defining the OnlineStore class.
Defining the OnlineStoreConfig class.
Referencing the OnlineStore in a feature repo's feature_store.yaml file.
Testing the OnlineStore class.
Update dependencies.
Add documentation.
1. Defining an OnlineStore class
OnlineStore class names must end with the OnlineStore suffix!
Contrib online stores
New online stores go in sdk/python/feast/infra/online_stores/.
What is a contrib plugin?
Not guaranteed to implement all interface methods
Not guaranteed to be stable.
Should have warnings for users to indicate this is a contrib plugin that is not maintained by the maintainers.
How do I make a contrib plugin an "official" plugin?
To move an online store plugin out of contrib, you need:
GitHub actions (i.e make test-python-integration) is setup to run all tests against the online store and pass.
At least two contributors own the plugin (ideally tracked in our OWNERS / CODEOWNERS file).
The OnlineStore class broadly contains two sets of methods
One set deals with managing infrastructure that the online store needed for operations
One set deals with writing data into the store, and reading data from the store.
1.1 Infrastructure Methods
There are two methods that deal with managing infrastructure for online stores, update and teardown
update is invoked when users run feast apply as a CLI command, or the FeatureStore.apply() sdk method.
The update method should be used to perform any operations necessary before data can be written to or read from the store. The update method can be used to create MySQL tables in preparation for reads and writes to new feature views.
teardown is invoked when users run feast teardown or FeatureStore.teardown().
The teardown method should be used to perform any clean-up operations. teardown can be used to drop MySQL indices and tables corresponding to the feature views being deleted.
1.2 Read/Write Methods
There are two methods that deal with writing data to and from the online stores.online_write_batch and online_read.
online_write_batch is invoked when running materialization (using the feast materialize or feast materialize-incremental commands, or the corresponding FeatureStore.materialize() method.
online_read is invoked when reading values from the online store using the FeatureStore.get_online_features() method.
2. Defining an OnlineStoreConfig class
Additional configuration may be needed to allow the OnlineStore to talk to the backing store. For example, MySQL may need configuration information like the host at which the MySQL instance is running, credentials for connecting to the database, etc.
To facilitate configuration, all OnlineStore implementations are required to also define a corresponding OnlineStoreConfig class in the same file. This OnlineStoreConfig class should inherit from the FeastConfigBaseModel class, which is defined here.
The FeastConfigBaseModel is a pydantic class, which parses yaml configuration into python objects. Pydantic also allows the model classes to define validators for the config classes, to make sure that the config classes are correctly defined.
This config class must container a type field, which contains the fully qualified class name of its corresponding OnlineStore class.
Additionally, the name of the config class must be the same as the OnlineStore class, with the Config suffix.
An example of the config class for MySQL :
This configuration can be specified in the feature_store.yaml as follows:
This configuration information is available to the methods of the OnlineStore, via theconfig: RepoConfig parameter which is passed into all the methods of the OnlineStore interface, specifically at the config.online_store field of the config parameter.
3. Using the custom online store
After implementing both these classes, the custom online store can be used by referencing it in a feature repo's feature_store.yaml file, specifically in the online_store field. The value specified should be the fully qualified class name of the OnlineStore.
As long as your OnlineStore class is available in your Python environment, it will be imported by Feast dynamically at runtime.
To use our MySQL online store, we can use the following feature_store.yaml:
If additional configuration for the online store is **not **required, then we can omit the other fields and only specify the type of the online store class as the value for the online_store.
4. Testing the OnlineStore class
4.1 Integrating with the integration test suite and unit test suite.
Even if you have created the OnlineStore class in a separate repo, you can still test your implementation against the Feast test suite, as long as you have Feast as a submodule in your repo.
In the Feast submodule, we can run all the unit tests and make sure they pass:
The universal tests, which are integration tests specifically intended to test offline and online stores, should be run against Feast to ensure that the Feast APIs works with your online store.
Feast parametrizes integration tests using the FULL_REPO_CONFIGS variable defined in sdk/python/tests/universal/feature_repos/repo_configuration.py which stores different online store classes for testing.
To overwrite these configurations, you can simply create your own file that contains a FULL_REPO_CONFIGS variable, and point Feast to that file by setting the environment variable FULL_REPO_CONFIGS_MODULE to point to that file.
A sample FULL_REPO_CONFIGS_MODULE looks something like this:
If you are planning to start the online store up locally(e.g spin up a local Redis Instance) for testing, then the dictionary entry should be something like:
If you are planning instead to use a Dockerized container to run your tests against your online store, you can define a OnlineStoreCreator and replace the None object above with your OnlineStoreCreator class. You should make this class available to pytest through the PYTEST_PLUGINS environment variable.
If you create a containerized docker image for testing, developers who are trying to test with your online store will not have to spin up their own instance of the online store for testing. An example of an OnlineStoreCreator is shown below:
3. Add a Makefile target to the Makefile to run your datastore specific tests by setting the FULL_REPO_CONFIGS_MODULE environment variable. Add PYTEST_PLUGINS if pytest is having trouble loading your DataSourceCreator. You can remove certain tests that are not relevant or still do not work for your datastore using the -k option.
If there are some tests that fail, this indicates that there is a mistake in the implementation of this online store!
5. Add Dependencies
Add any dependencies for your online store to pyproject.toml under [project.optional-dependencies] as a new extra (e.g. <online_store> = ["package1>=1.0", "package2"]). These packages should be defined as extras so that they are not installed by users by default.
You will need to regenerate our requirements lock files:
6. Add Documentation
Remember to add the documentation for your online store.
Add a new markdown file to docs/reference/online-stores/.
You should also add a reference in docs/reference/online-stores/README.md and docs/SUMMARY.md. Add a new markdown document to document your online store functionality similar to how the other online stores are documented.
NOTE:Be sure to document the following things about your online store:
Be sure to cover how to create the datasource and what configuration is needed in the feature_store.yaml file in order to create the datasource.
Make sure to flag that the online store is in alpha development.
Add some documentation on what the data model is for the specific online store for more clarity.
Finally, generate the python code docs by running:
Codebase Structure
Let's examine the Feast codebase. This analysis is accurate as of Feast 0.23.
Python SDK
The Python SDK lives in sdk/python/feast. The majority of Feast logic lives in these Python files:
The core Feast objects (entities, feature views, data sources, etc.) are defined in their respective Python files, such as entity.py, feature_view.py, and data_source.py.
The FeatureStore class is defined in feature_store.py and the associated configuration object (the Python representation of the feature_store.yaml file) are defined in repo_config.py.
The CLI and other core feature store logic are defined in cli.py and repo_operations.py.
The type system that is used to manage conversion between Feast types and external typing systems is managed in type_map.py.
The Python feature server (the server that is started through the feast serve command) is defined in feature_server.py.
There are also several important submodules:
infra/ contains all the infrastructure components, such as the provider, offline store, online store, batch materialization engine, and registry.
dqm/ covers data quality monitoring, such as the dataset profiler.
diff/ covers the logic for determining how to apply infrastructure changes upon feature repo changes (e.g. the output of feast plan and feast apply).
embedded_go/ covers the Go feature server.
ui/ contains the embedded Web UI, to be launched on the feast ui command.
The tests for the Python SDK are contained in sdk/python/tests. For more details, see this overview of the test suite.
Example flow: feast apply
Let's walk through how feast apply works by tracking its execution across the codebase.
All CLI commands are in cli.py. Most of these commands are backed by methods in repo_operations.py. The feast apply command triggers apply_total_command, which then calls apply_total in repo_operations.py.
With a FeatureStore object (from feature_store.py) that is initialized based on the feature_store.yaml in the current working directory, apply_total first parses the feature repo with parse_repo and then calls either FeatureStore.apply or FeatureStore._apply_diffs to apply those changes to the feature store.
Let's examine FeatureStore.apply. It splits the objects based on class (e.g. Entity, FeatureView, etc.) and then calls the appropriate registry method to apply or delete the object. For example, it might call self._registry.apply_entity to apply an entity. If the default file-based registry is used, this logic can be found in infra/registry/registry.py.
Then the feature store must update its cloud infrastructure (e.g. online store tables) to match the new feature repo, so it calls Provider.update_infra, which can be found in infra/provider.py.
Assuming the provider is a built-in provider (e.g. one of the local, GCP, or AWS providers), it will call PassthroughProvider.update_infra in infra/passthrough_provider.py.
This delegates to the online store and batch materialization engine. For example, if the feature store is configured to use the Redis online store then the update method from infra/online_stores/redis.py will be called. And if the local materialization engine is configured then the update method from infra/materialization/local_engine.py will be called.
At this point, the feast apply command is complete.
Example flow: feast materialize
Let's walk through how feast materialize works by tracking its execution across the codebase.
The feast materialize command triggers materialize_command in cli.py, which then calls FeatureStore.materialize from feature_store.py.
This then calls Provider.materialize_single_feature_view, which can be found in infra/provider.py.
As with feast apply, the provider is most likely backed by the passthrough provider, in which case PassthroughProvider.materialize_single_feature_view will be called.
This delegates to the underlying batch materialization engine. Assuming that the local engine has been configured, LocalMaterializationEngine.materialize from infra/materialization/local_engine.py will be called.
Since materialization involves reading features from the offline store and writing them to the online store, the local engine will delegate to both the offline store and online store. Specifically, it will call OfflineStore.pull_latest_from_table_or_query and OnlineStore.online_write_batch. These two calls will be routed to the offline store and online store that have been configured.
Example flow: get_historical_features
Let's walk through how get_historical_features works by tracking its execution across the codebase.
We start with FeatureStore.get_historical_features in feature_store.py. This method does some internal preparation, and then delegates the actual execution to the underlying provider by calling Provider.get_historical_features, which can be found in infra/provider.py.
As with feast apply, the provider is most likely backed by the passthrough provider, in which case PassthroughProvider.get_historical_features will be called.
That call simply delegates to OfflineStore.get_historical_features. So if the feature store is configured to use Snowflake as the offline store, SnowflakeOfflineStore.get_historical_features will be executed.
Java SDK
The java/ directory contains the Java serving component. See here for more details on how the repo is structured.
Go feature server
The go/ directory contains the Go feature server. Most of the files here have logic to help with reading features from the online store. Within go/, the internal/feast/ directory contains most of the core logic:
onlineserving/ covers the core serving logic.
model/ contains the implementations of the Feast objects (entity, feature view, etc.).
For example, entity.go is the Go equivalent of entity.py. It contains a very simple Go implementation of the entity object.
registry/ covers the registry.
Currently only the file-based registry supported (the sql-based registry is unsupported). Additionally, the file-based registry only supports a file-based registry store, not the GCS or S3 registry stores.
onlinestore/ covers the online stores (currently only Redis and SQLite are supported).
Protobufs
Feast uses protobuf to store serialized versions of the core Feast objects. The protobuf definitions are stored in protos/feast.
The registry consists of the serialized representations of the Feast objects.
Typically, changes being made to the Feast objects require changes to their corresponding protobuf representations. The usual best practices for making changes to protobufs should be followed ensure backwards and forwards compatibility.
Web UI
The ui/ directory contains the Web UI. See here for more details on the structure of the Web UI.
Kafka
Warning: This is an experimental feature. It's intended for early testing and feedback, and could change without warnings in future releases.
Description
Kafka sources allow users to register Kafka streams as data sources. Feast currently does not launch or monitor jobs to ingest data from Kafka. Users are responsible for launching and monitoring their own ingestion jobs, which should write feature values to the online store through FeatureStore.write_to_online_store. An example of how to launch such a job with Spark can be found here. Feast also provides functionality to write to the offline store using the write_to_offline_store functionality.
Kafka sources must have a batch source specified. The batch source will be used for retrieving historical features. Thus users are also responsible for writing data from their Kafka streams to a batch data source such as a data warehouse table. When using a Kafka source as a stream source in the definition of a feature view, a batch source doesn't need to be specified in the feature view definition explicitly.
Stream sources
Streaming data sources are important sources of feature values. A typical setup with streaming data looks like:
Raw events come in (stream 1)
Streaming transformations applied (e.g. generating features like last_N_purchased_categories) (stream 2)
Write stream 2 values to an offline store as a historical log for training (optional)
Write stream 2 values to an online store for low latency feature serving
Periodically materialize feature values from the offline store into the online store for decreased training-serving skew and improved model performance
Example
Defining a Kafka source
Note that the Kafka source has a batch source.
Using the Kafka source in a stream feature view
The Kafka source can be used in a stream feature view.
Ingesting data
See here for a example of how to ingest data from a Kafka source into Feast.
Kinesis
Warning: This is an experimental feature. It's intended for early testing and feedback, and could change without warnings in future releases.
Description
Kinesis sources allow users to register Kinesis streams as data sources. Feast currently does not launch or monitor jobs to ingest data from Kinesis. Users are responsible for launching and monitoring their own ingestion jobs, which should write feature values to the online store through FeatureStore.write_to_online_store. An example of how to launch such a job with Spark to ingest from Kafka can be found here; by using a different plugin, the example can be adapted to Kinesis. Feast also provides functionality to write to the offline store using the write_to_offline_store functionality.
Kinesis sources must have a batch source specified. The batch source will be used for retrieving historical features. Thus users are also responsible for writing data from their Kinesis streams to a batch data source such as a data warehouse table. When using a Kinesis source as a stream source in the definition of a feature view, a batch source doesn't need to be specified in the feature view definition explicitly.
Stream sources
Streaming data sources are important sources of feature values. A typical setup with streaming data looks like:
Raw events come in (stream 1)
Streaming transformations applied (e.g. generating features like last_N_purchased_categories) (stream 2)
Write stream 2 values to an offline store as a historical log for training (optional)
Write stream 2 values to an online store for low latency feature serving
Periodically materialize feature values from the offline store into the online store for decreased training-serving skew and improved model performance
Example
Defining a Kinesis source
Note that the Kinesis source has a batch source.
Using the Kinesis source in a stream feature view
The Kinesis source can be used in a stream feature view.
Ingesting data
See here for a example of how to ingest data from a Kafka source into Feast. The approach used in the tutorial can be easily adapted to work for Kinesis as well.
Spark (contrib)
Description
Spark data sources are tables or files that can be loaded from some Spark store (e.g. Hive or in-memory). They can also be specified by a SQL query.
New in Feast: SparkSource now supports advanced table formats including Apache Iceberg, Delta Lake, and Apache Hudi, enabling ACID transactions, time travel, and schema evolution capabilities. See the Table Formats guide for detailed documentation.
Disclaimer
The Spark data source does not achieve full test coverage. Please do not assume complete stability.
Examples
Basic Examples
Using a table reference from SparkSession (for example, either in-memory or a Hive Metastore):
Using a query:
Using a file reference:
Table Format Examples
SparkSource supports advanced table formats for modern data lakehouse architectures. For detailed documentation, configuration options, and best practices, see the Table Formats guide.
Apache Iceberg
Delta Lake
Apache Hudi
For advanced configuration including time travel, incremental queries, and performance tuning, see the Table Formats guide.
Configuration Options
The full set of configuration options is available here.
Spark data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see here.
# Client Reads
features = store.get_online_features(
feature_refs=[
"user_data:click_through_rate",
"user_data:number_of_clicks",
"user_data:average_page_duration",
"user_data:model_predictions",
],
entity_rows=[{"user_id": 1}],
)
if features.to_dict().get('user_data:model_predictions') is None:
model_predictions = model_server.predict(features)
store.write_to_online_store(feature_view_name="user_data", df=pd.DataFrame(model_predictions))
# Client Writes from the Data Producer
user_data = request.POST.get('user_data')
model_predictions = model_server.predict(user_data) # assume this includes `user_data` in the Data Frame
store.write_to_online_store(feature_view_name="user_data", df=pd.DataFrame(model_predictions))
from feast import FeatureStore
from feast.infra.offline_stores.bigquery_source import SavedDatasetBigQueryStorage
store = FeatureStore()
historical_job = store.get_historical_features(
features=["driver:avg_trip"],
entity_df=...,
)
dataset = store.create_saved_dataset(
from_=historical_job,
name='my_training_dataset',
storage=SavedDatasetBigQueryStorage(table_ref='<gcp-project>.<gcp-dataset>.my_training_dataset'),
tags={'author': 'oleksii'}
)
dataset.to_df()
# Delete an entity
store._registry.delete_entity("my_entity", project=store.project)
# Delete a data source
store._registry.delete_data_source("my_data_source", project=store.project)
# Delete a saved dataset
store._registry.delete_saved_dataset("my_saved_dataset", project=store.project)
# Delete a validation reference
store._registry.delete_validation_reference("my_validation_reference", project=store.project)
repo_config = RepoConfig(
registry=RegistryConfig(path="gs://feast-test-gcs-bucket/registry.pb"),
project="feast_demo_gcp",
provider="gcp",
offline_store="file", # Could also be the OfflineStoreConfig e.g. FileOfflineStoreConfig
online_store="null", # Could also be the OnlineStoreConfig e.g. RedisOnlineStoreConfig
)
store = FeatureStore(config=repo_config)
1. Transformation (via Transformation API)
2. Aggregation (via Aggregation API)
3. Join (join with entity datasets, customized JOIN or join with another Feature View)
4. Filter (Point in time filter, ttl filter, filter by custom expression)
...
+---------------------+
| SourceReadNode | <- Read data from offline store (e.g. Snowflake, BigQuery, etc. or custom source)
+---------------------+
|
v
+--------------------------------------+
| TransformationNode / JoinNode (*) | <- Merge data sources, custom transformations by user, or default join
+--------------------------------------+
|
v
+---------------------+
| FilterNode | <- used for point-in-time filtering
+---------------------+
|
v
+---------------------+
| AggregationNode (*) | <- only if aggregations are defined
+---------------------+
|
v
+---------------------+
| DeduplicationNode | <- used if no aggregation and for history
+---------------------+ retrieval
|
v
+---------------------+
| ValidationNode (*) | <- optional validation checks
+---------------------+
|
v
+----------+
| Output |
+----------+
/ \
v v
+----------------+ +----------------+
| OnlineStoreWrite| OfflineStoreWrite|
+----------------+ +----------------+
import os
import io
import pypdf
import logging
import hashlib
from datetime import datetime
import requests
import pandas as pd
from transformers import AutoTokenizer
from sentence_transformers import SentenceTransformer
from docling.datamodel.base_models import ConversionStatus, InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.chunking import HybridChunker
logging.basicConfig(level=logging.INFO)
_log = logging.getLogger(__name__)
# Base URL for PDFs
BASE_URL = 'https://raw.githubusercontent.com/DS4SD/docling/refs/heads/main/tests/data/pdf/'
PDF_FILES = [
'2203.01017v2.pdf', '2305.03393v1-pg9.pdf', '2305.03393v1.pdf',
'amt_handbook_sample.pdf', 'code_and_formula.pdf', 'picture_classification.pdf',
'redp5110_sampled.pdf', 'right_to_left_01.pdf', 'right_to_left_02.pdf', 'right_to_left_03.pdf'
]
INPUT_DOC_PATHS = [os.path.join(BASE_URL, pdf_file) for pdf_file in PDF_FILES]
# Configure PDF processing
pipeline_options = PdfPipelineOptions()
pipeline_options.generate_page_images = True
doc_converter = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
# Load tokenizer and embedding model
EMBED_MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"
MAX_TOKENS = 64 # Small token limit for demonstration
tokenizer = AutoTokenizer.from_pretrained(EMBED_MODEL_ID)
embedding_model = SentenceTransformer(EMBED_MODEL_ID)
chunker = HybridChunker(tokenizer=tokenizer, max_tokens=MAX_TOKENS, merge_peers=True)
def embed_text(text: str) -> list[float]:
"""Generate an embedding for a given text."""
return embedding_model.encode([text], normalize_embeddings=True).tolist()[0]
def generate_document_rows(conv_results):
"""
Generator that yields one row per chunk from each successfully converted document.
Each yielded dict contains:
- file_name: Name of the source file.
- raw_markdown: Serialized text for the chunk.
- chunk_embedding: The embedding vector for that chunk.
"""
processed_docs = 0
for conv_res in conv_results:
if conv_res.status != ConversionStatus.SUCCESS:
continue
processed_docs += 1
file_name = conv_res.input.file.stem # FIX: Use `.file.stem` instead of `.path`
# Extract the document object (which contains iterate_items)
document = conv_res.document
try:
document_markdown = document.export_to_markdown()
except:
document_markdown = ""
if document is None:
_log.warning(f"Document conversion failed for {file_name}")
continue
# Process each chunk from the document
for chunk in chunker.chunk(dl_doc=document): # Use `document` here!
raw_chunk = chunker.serialize(chunk=chunk)
embedding = embed_text(raw_chunk)
yield {
"file_name": file_name,
"full_document_markdown": document_markdown,
"raw_chunk_markdown": raw_chunk,
"chunk_embedding": embedding,
}
_log.info(f"Processed {processed_docs} documents successfully.")
def generate_chunk_id(file_name: str, raw_chunk_markdown: str) -> str:
"""Generate a unique chunk ID based on file_name and raw_chunk_markdown."""
unique_string = f"{file_name}-{raw_chunk_markdown}"
return hashlib.sha256(unique_string.encode()).hexdigest()
conv_results = doc_converter.convert_all(INPUT_DOC_PATHS, raises_on_error=False)
# Build a DataFrame where each row is a unique chunk record
rows = list(generate_document_rows(conv_results))
df = pd.DataFrame.from_records(rows)
output_dict = {}
for file_name in PDF_FILES:
try:
r = requests.get(BASE_URL + file_name)
pdf_bytes = io.BytesIO(r.content)
output_dict[file_name] = pdf_bytes.getvalue()
except Exception as e:
print(f"error with {file_name} \n{e}")
odf = pd.DataFrame.from_dict(output_dict, orient='index', columns=['bytes']).reset_index()
odf.rename({"index": "file_name"}, axis=1, inplace=True)
odf['file_name'] = odf['file_name'].str.replace('.pdf', '')
finaldf = df.merge(odf, on='file_name', how='left')
finaldf["chunk_id"] = finaldf.apply(lambda row: generate_chunk_id(row["file_name"], row["raw_chunk_markdown"]), axis=1)
finaldf['created'] = datetime.now()
pdf_example = pypdf.PdfReader(io.BytesIO(finaldf['bytes'].values[0]))
finaldf.drop(['full_document_markdown', 'bytes'], axis=1).to_parquet('feature_repo/data/docling_samples.parquet', index=False)
odf.to_parquet('feature_repo/data/metadata_samples.parquet', index=False)
from datetime import timedelta
import pandas as pd
from feast import (
FeatureView,
Field,
FileSource,
Entity,
RequestSource,
)
from feast.data_format import ParquetFormat
from feast.types import Float64, Array, String, ValueType, PdfBytes
from feast.on_demand_feature_view import on_demand_feature_view
from sentence_transformers import SentenceTransformer
from typing import Dict, Any, List
import hashlib
from docling.datamodel.base_models import DocumentStream
import io
from docling.document_converter import DocumentConverter
from transformers import AutoTokenizer
from sentence_transformers import SentenceTransformer
from docling.chunking import HybridChunker
# Load tokenizer and embedding model
EMBED_MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"
MAX_TOKENS = 64 # Small token limit for demonstration
tokenizer = AutoTokenizer.from_pretrained(EMBED_MODEL_ID)
embedding_model = SentenceTransformer(EMBED_MODEL_ID)
chunker = HybridChunker(tokenizer=tokenizer, max_tokens=MAX_TOKENS, merge_peers=True)
def embed_text(text: str) -> list[float]:
"""Generate an embedding for a given text."""
return embedding_model.encode([text], normalize_embeddings=True).tolist()[0]
def generate_chunk_id(file_name: str, raw_chunk_markdown: str="") -> str:
"""Generate a unique chunk ID based on file_name and raw_chunk_markdown."""
unique_string = f"{file_name}-{raw_chunk_markdown}" if raw_chunk_markdown != "" else f"{file_name}"
return hashlib.sha256(unique_string.encode()).hexdigest()
# Define entities
chunk = Entity(
name="chunk_id",
description="Chunk ID",
value_type=ValueType.STRING,
join_keys=["chunk_id"],
)
document = Entity(
name="document_id",
description="Document ID",
value_type=ValueType.STRING,
join_keys=["document_id"],
)
source = FileSource(
file_format=ParquetFormat(),
path="./data/docling_samples.parquet",
timestamp_field="created",
)
input_request_pdf = RequestSource(
name="pdf_request_source",
schema=[
Field(name="document_id", dtype=String),
Field(name="pdf_bytes", dtype=PdfBytes),
Field(name="file_name", dtype=String),
],
)
# Define the view for retrieval
docling_example_feature_view = FeatureView(
name="docling_feature_view",
entities=[chunk],
schema=[
Field(name="file_name", dtype=String),
Field(name="raw_chunk_markdown", dtype=String),
Field(
name="vector",
dtype=Array(Float64),
vector_index=True,
vector_search_metric="COSINE",
),
Field(name="chunk_id", dtype=String),
],
source=source,
ttl=timedelta(hours=2),
)
@on_demand_feature_view(
entities=[chunk, document],
sources=[input_request_pdf],
schema=[
Field(name="document_id", dtype=String),
Field(name="chunk_id", dtype=String),
Field(name="chunk_text", dtype=String),
Field(
name="vector",
dtype=Array(Float64),
vector_index=True,
vector_search_metric="L2",
),
],
mode="python",
write_to_online_store=True,
singleton=True,
)
def docling_transform_docs(inputs: dict[str, Any]):
document_ids, chunks, embeddings, chunk_ids = [], [], [], []
buf = io.BytesIO(
inputs["pdf_bytes"],
)
doc_source = DocumentStream(name=inputs["file_name"], stream=buf)
converter = DocumentConverter()
result = converter.convert(doc_source)
for i, chunk in enumerate(chunker.chunk(dl_doc=result.document)):
raw_chunk = chunker.serialize(chunk=chunk)
embedding = embed_text(raw_chunk)
chunk_id = f"chunk-{i}"
document_ids.append(inputs["document_id"])
chunks.append(raw_chunk)
chunk_ids.append(chunk_id)
embeddings.append(embedding)
return {
"document_id": document_ids,
"chunk_id": chunk_ids,
"vector": embeddings,
"chunk_text": chunks,
}
feast apply
import pandas as pd
from feast import FeatureStore
store = FeatureStore(repo_path=".")
df = pd.read_parquet("./data/docling_samples.parquet")
mdf = pd.read_parquet("./data/metadata_samples.parquet")
df['chunk_embedding'] = df['vector'].apply(lambda x: x.tolist())
embedding_length = len(df['vector'][0])
print(f'embedding length = {embedding_length}')
df['created'] = pd.Timestamp.now()
mdf['created'] = pd.Timestamp.now()
# Ingesting transformed data to the feature view that has no associated transformation
store.write_to_online_store(feature_view_name='docling_feature_view', df=df)
# Turning off transformation on writes is as simple as changing the default behavior
store.write_to_online_store(
feature_view_name='docling_transform_docs',
df=df[df['document_id']!='doc-1'],
transform_on_write=False,
)
# Now we can transform a raw PDF on the fly
store.write_to_online_store(
feature_view_name='docling_transform_docs',
df=mdf[mdf['document_id']=='doc-1'],
transform_on_write=True, # this is the default
)
from feast import FeatureStore
# Initialize FeatureStore
store = FeatureStore(".")
# Generate query embedding
question = 'Who are the authors of the paper?'
query_embedding = embed_text(question)
# Retrieve similar documents
context_data = store.retrieve_online_documents_v2(
features=[
"docling_feature_view:vector",
"docling_feature_view:file_name",
"docling_feature_view:raw_chunk_markdown",
"docling_feature_view:chunk_id",
],
query=query_embedding,
top_k=3,
distance_metric='COSINE',
).to_df()
print(context_data)
from openai import OpenAI
import os
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"),
)
# Format documents for context
def format_documents(context_data, base_prompt):
documents = "\n".join([f"Document {i+1}: {row['embedded_documents__sentence_chunks']}"
for i, row in context_data.iterrows()])
return f"{base_prompt}\n\nContext documents:\n{documents}"
BASE_PROMPT = """You are a helpful assistant that answers questions based on the provided context."""
FULL_PROMPT = format_documents(context_data, BASE_PROMPT)
# Generate response
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": FULL_PROMPT},
{"role": "user", "content": query_embedding}
],
)
print('\n'.join([c.message.content for c in response.choices]))
pip install feast[milvus,rag]
from feast import DocEmbedder
import pandas as pd
# Prepare your documents as a DataFrame
df = pd.DataFrame({
"id": ["doc1", "doc2", "doc3"],
"text": [
"Aaron is a prophet, high priest, and the brother of Moses...",
"God at Sinai granted Aaron the priesthood for himself...",
"His rod turned into a snake. Then he stretched out...",
],
})
# DocEmbedder handles everything: generates FeatureView, applies repo,
# chunks text, generates embeddings, and writes to the online store
embedder = DocEmbedder(
repo_path="feature_repo/",
feature_view_name="text_feature_view",
)
result = embedder.embed_documents(
documents=df,
id_column="id",
source_column="text",
column_mapping=("text", "text_embedding"),
)
from feast import FeatureStore
store = FeatureStore("feature_repo/")
query_embedding = embed_text("Who are the authors of the paper?")
context_data = store.retrieve_online_documents_v2(
features=[
"text_feature_view:embedding",
"text_feature_view:text",
"text_feature_view:source_id",
],
query=query_embedding,
top_k=3,
distance_metric="COSINE",
).to_df()
from feast.chunker import BaseChunker, ChunkingConfig
from typing import Any, Optional
class SentenceChunker(BaseChunker):
"""Chunks text by sentences instead of word count."""
def load_parse_and_chunk(
self,
source: Any,
source_id: str,
source_column: str,
source_type: Optional[str] = None,
) -> list[dict]:
import re
text = str(source)
# Split on sentence boundaries
sentences = re.split(r'(?<=[.!?])\s+', text)
chunks = []
current_chunk = []
chunk_index = 0
for sentence in sentences:
current_chunk.append(sentence)
combined = " ".join(current_chunk)
if len(combined.split()) >= self.config.chunk_size:
chunks.append({
"chunk_id": f"{source_id}_{chunk_index}",
"original_id": source_id,
source_column: combined,
"chunk_index": chunk_index,
})
# Keep overlap by retaining the last sentence
current_chunk = [sentence]
chunk_index += 1
# Don't forget the last chunk
if current_chunk and len(" ".join(current_chunk).split()) >= self.config.min_chunk_size:
chunks.append({
"chunk_id": f"{source_id}_{chunk_index}",
"original_id": source_id,
source_column: " ".join(current_chunk),
"chunk_index": chunk_index,
})
return chunks
# Only prints out runtime warnings once.
warnings.simplefilter("once", RuntimeWarning)
def update(
self,
config: RepoConfig,
tables_to_delete: Sequence[Union[FeatureTable, FeatureView]],
tables_to_keep: Sequence[Union[FeatureTable, FeatureView]],
entities_to_delete: Sequence[Entity],
entities_to_keep: Sequence[Entity],
partial: bool,
):
"""
An example of creating managing the tables needed for a mysql-backed online store.
"""
warnings.warn(
"This online store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
conn = self._get_conn(config)
cur = conn.cursor(buffered=True)
project = config.project
for table in tables_to_keep:
cur.execute(
f"CREATE TABLE IF NOT EXISTS {_table_id(project, table)} (entity_key VARCHAR(512), feature_name VARCHAR(256), value BLOB, event_ts timestamp, created_ts timestamp, PRIMARY KEY(entity_key, feature_name))"
)
cur.execute(
f"CREATE INDEX {_table_id(project, table)}_ek ON {_table_id(project, table)} (entity_key);"
)
for table in tables_to_delete:
cur.execute(
f"DROP INDEX {_table_id(project, table)}_ek ON {_table_id(project, table)};"
)
cur.execute(f"DROP TABLE IF EXISTS {_table_id(project, table)}")
def teardown(
self,
config: RepoConfig,
tables: Sequence[Union[FeatureTable, FeatureView]],
entities: Sequence[Entity],
):
warnings.warn(
"This online store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
conn = self._get_conn(config)
cur = conn.cursor(buffered=True)
project = config.project
for table in tables:
cur.execute(
f"DROP INDEX {_table_id(project, table)}_ek ON {_table_id(project, table)};"
)
cur.execute(f"DROP TABLE IF EXISTS {_table_id(project, table)}")
feast_custom_online_store/mysql.py
# Only prints out runtime warnings once.
warnings.simplefilter("once", RuntimeWarning)
def online_write_batch(
self,
config: RepoConfig,
table: Union[FeatureTable, FeatureView],
data: List[
Tuple[EntityKeyProto, Dict[str, ValueProto], datetime, Optional[datetime]]
],
progress: Optional[Callable[[int], Any]],
) -> None:
warnings.warn(
"This online store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
conn = self._get_conn(config)
cur = conn.cursor(buffered=True)
project = config.project
for entity_key, values, timestamp, created_ts in data:
entity_key_bin = serialize_entity_key(
entity_key,
entity_key_serialization_version=config.entity_key_serialization_version,
).hex()
timestamp = _to_naive_utc(timestamp)
if created_ts is not None:
created_ts = _to_naive_utc(created_ts)
for feature_name, val in values.items():
self.write_to_table(created_ts, cur, entity_key_bin, feature_name, project, table, timestamp, val)
self._conn.commit()
if progress:
progress(1)
def online_read(
self,
config: RepoConfig,
table: Union[FeatureTable, FeatureView],
entity_keys: List[EntityKeyProto],
requested_features: Optional[List[str]] = None,
) -> List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]]:
warnings.warn(
"This online store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
conn = self._get_conn(config)
cur = conn.cursor(buffered=True)
result: List[Tuple[Optional[datetime], Optional[Dict[str, ValueProto]]]] = []
project = config.project
for entity_key in entity_keys:
entity_key_bin = serialize_entity_key(
entity_key,
entity_key_serialization_version=config.entity_key_serialization_version,
).hex()
print(f"entity_key_bin: {entity_key_bin}")
cur.execute(
f"SELECT feature_name, value, event_ts FROM {_table_id(project, table)} WHERE entity_key = %s",
(entity_key_bin,),
)
res = {}
res_ts = None
for feature_name, val_bin, ts in cur.fetchall():
val = ValueProto()
val.ParseFromString(val_bin)
res[feature_name] = val
res_ts = ts
if not res:
result.append((None, None))
else:
result.append((res_ts, res))
return result
project: test_custom
registry: data/registry.db
provider: local
online_store:
# Make sure to specify the type as the fully qualified path that Feast can import.
type: feast_custom_online_store.mysql.MySQLOnlineStore
user: foo
password: bar
feature_repo/feature_store.yaml
project: test_custom
registry: data/registry.db
provider: local
online_store: feast_custom_online_store.mysql.MySQLOnlineStore
{
"sqlite": ({"type": "sqlite"}, None),
# Specifies sqlite as the online store. The `None` object specifies to not use a containerized docker container.
}
Role-Based Access Control (RBAC) is a security mechanism that restricts access to resources based on the roles/groups/namespaces of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups/namespaces can access or modify specific resources, thereby maintaining data security and operational integrity.
Functional Requirements
The RBAC implementation in Feast is designed to:
Assign Permissions: Allow administrators to assign permissions for various operations and resources to users or groups/namespaces.
Seamless Integration: Integrate smoothly with existing business code without requiring significant modifications.
Backward Compatibility: Maintain support for non-authorized models as the default to ensure backward compatibility.
Business Goals
The primary business goals of implementing RBAC in the Feast are:
Feature Sharing: Enable multiple teams to share the feature store while ensuring controlled access. This allows for collaborative work without compromising data security.
Access Control Management: Prevent unauthorized access to team-specific resources and spaces, governing the operations that each user or group can perform.
Reference Architecture
Feast operates as a collection of connected services, each enforcing authorization permissions. The architecture is designed as a distributed microservices system with the following key components:
Service Endpoints: These enforce authorization permissions, ensuring that only authorized requests are processed.
Client Integration: Clients authenticate with feature servers by attaching authorization token to each request.
Service-to-Service Communication: This is always granted.
rbac.jpg
Permission Model
The RBAC system in Feast uses a permission model that defines the following concepts:
Resource: An object within Feast that needs to be secured against unauthorized access.
Action: A logical operation performed on a resource, such as Create, Describe, Update, Delete, Read, or write operations.
Policy: A set of rules that enforce authorization decisions on resources. The polices are based on user roles or groups or namespaces or combined.
Authorization Architecture
The authorization architecture in Feast is built with the following components:
Token Extractor: Extracts the authorization token from the request header.
Token Parser: Parses the token to retrieve user details.
Policy Enforcer: Validates the secured endpoint against the retrieved user details.
Token Injector: Adds the authorization token to each secured request header.
Overview
Feast Architecture Diagram
Functionality
Create Batch Features: ELT/ETL systems like Spark and SQL are used to transform data in the batch store.
Create Stream Features: Stream features are created from streaming services such as Kafka or Kinesis, and can be pushed directly into Feast via the Push API.
Feast Apply: The user (or CI) publishes versioned controlled feature definitions using feast apply. This CLI command updates infrastructure and persists definitions in the object store registry.
Feast Materialize: The user (or scheduler) executes feast materialize (with timestamps or --disable-event-timestamp to materialize all data with current timestamps) which loads features from the offline store into the online store.
Model Training: A model training pipeline is launched. It uses the Feast Python SDK to retrieve a training dataset that can be used for training models.
Get Historical Features: Feast exports a point-in-time correct training dataset based on the list of features and entity dataframe provided by the model training pipeline.
Deploy Model: The trained model binary (and list of features) are deployed into a model serving system. This step is not executed by Feast.
Prediction: A backend system makes a request for a prediction from the model serving service.
Get Online Features: The model serving service makes a request to the Feast Online Serving service for online features using a Feast SDK.
Feature Retrieval: The online serving service retrieves the latest feature values from the online store and returns them to the model serving service.
Components
A complete Feast deployment contains the following components:
Feast Registry: An object store (GCS, S3) based registry used to persist feature definitions that are registered with the feature store. Systems can discover feature data by interacting with the registry through the Feast SDK.
Feast Python SDK/CLI: The primary user facing SDK. Used to:
Manage version controlled feature definitions.
Materialize (load) feature values into the online store.
Build and retrieve training datasets from the offline store.
Retrieve online features.
Feature Server: The Feature Server is a REST API server that serves feature values for a given entity key and feature reference. The Feature Server is designed to be horizontally scalable and can be deployed in a distributed manner.
Stream Processor: The Stream Processor can be used to ingest feature data from streams and write it into the online or offline stores. Currently, there's an experimental Spark processor that's able to consume data from Kafka.
Compute Engine: The component launches a process which loads data into the online store from the offline store. By default, Feast uses a local in-process engine implementation to materialize data. However, additional infrastructure can be used for a more scalable materialization process.
Online Store: The online store is a database that stores only the latest feature values for each entity. The online store is either populated through materialization jobs or through .
Offline Store: The offline store persists batch data that has been ingested into Feast. This data is used for producing training datasets. For feature retrieval and materialization, Feast does not manage the offline store directly, but runs queries against it. However, offline stores can be configured to support writes if Feast configures logging functionality of served features.
Authorization Manager: The authorization manager detects authentication tokens from client requests to Feast servers and uses this information to enforce permission policies on the requested services.
Online store
Feast uses online stores to serve features at low latency. Feature values are loaded from data sources into the online store through materialization, which can be triggered through the materialize command (either with specific timestamps or using --disable-event-timestamp to materialize all data with current timestamps).
The storage schema of features within the online store mirrors that of the original data source. One key difference is that for each entity key, only the latest feature values are stored. No historical values are stored.
Here is an example batch data source:
Once the above data source is materialized into Feast (using feast materialize with timestamps or feast materialize --disable-event-timestamp), the feature values will be stored as follows:
Features can also be written directly to the online store via push sources .
FAQ
Don't see your question?
We encourage you to ask questions on GitHub. Even better, once you get an answer, add the answer to this FAQ via a pull request!
Getting started
Which programming language should I use to run Feast in a microservice architecture?
How does Feast handle model or feature versioning?
Feast expects that each version of a model corresponds to a different feature service.
Feature views once they are used by a feature service are intended to be immutable and not deleted (until a feature service is removed). In the future, feast plan and feast apply will throw errors if it sees this kind of behavior.
What is the difference between data sources and the offline store?
The data source itself defines the underlying data warehouse table in which the features are stored. The offline store interface defines the APIs required to make an arbitrary compute layer work for Feast (e.g. pulling features given a set of feature views from their sources, exporting the data set results to different formats). Please see data sources and offline store for more details.
Is it possible to have offline and online stores from different providers?
Yes, this is possible. For example, you can use BigQuery as an offline store and Redis as an online store.
Functionality
How do I run get_historical_features without providing an entity dataframe?
Feast does support fetching historical features without passing an entity dataframe with the request.
Supported offline stores: Entity-less (entity dataframe–less) retrieval is supported for the Postgres, Dask, Spark, and Ray offline stores. Postgres was the first to support it; Dask, Spark, and Ray have followed. Other offline stores may be added based on priority and community demand.
Date range: Retrieval is controlled by the start_date and end_date parameters. Supported combinations:
Both params given → data in the given start-to-end time range.
Only start_date given → data from the start date to now.
Only end_date given → data from (end_date minus feature view TTL) to end_date.
Neither given → data from (TTL window) to now.
Multiple feature views: When requesting features from multiple feature views in entity-less mode, the feature views must share entity keys so that joins can be performed correctly.
We welcome contributions to add or improve entity-less retrieval. See GitHub issue #1611.
Does Feast provide security or access control?
Feast currently does not support any access control other than the access control required for the Provider's environment (for example, GCP and AWS permissions).
It is a good idea though to lock down the registry file so only the CI/CD pipeline can modify it. That way data scientists and other users cannot accidentally modify the registry and lose other team's data.
Does Feast support streaming sources?
Yes. In earlier versions of Feast, we used Feast Spark to manage ingestion from stream sources. In the current version of Feast, we support push based ingestion. Feast also defines a stream processor that allows a deeper integration with stream sources.
These transformations are Pandas transformations run on batch data when you call get_historical_features and at online serving time when you call `get_online_features.
Note that if you use push sources to ingest streaming features, these transformations will execute on the fly as well
Batch transformations (WIP, see )
These will include SQL + PySpark based transformations on batch data sources.
A feature view can be defined with multiple entities. Since each entity has a unique join_key, using multiple entities will achieve the effect of a composite key.
What are the performance/latency characteristics of Feast?
Feast is designed to work at scale and support low latency online serving. See our benchmark blog post for details.
Does Feast support embeddings and list features?
Yes. Specifically:
Simple lists / dense embeddings:
BigQuery supports list types natively
Redshift does not support list types, so you'll need to serialize these features into strings (e.g. json or protocol buffers)
Feast's implementation of online stores serializes features into Feast protocol buffers and supports list types (see )
Sparse embeddings (e.g. one hot encodings)
One way to do this efficiently is to have a protobuf or string representation of
Does Feast support X storage engine?
The list of supported offline and online stores can be found here and here, respectively. The roadmap indicates the stores for which we are planning to add support. Finally, our Provider abstraction is built to be extensible, so you can plug in your own implementations of offline and online stores. Please see more details about customizing Feast here.
Does Feast support using different clouds for offline vs online stores?
Yes. Using a GCP or AWS provider in feature_store.yaml primarily sets default offline / online stores and configures where the remote registry file can live. You can override the offline and online stores to be in different clouds if you wish.
What is the difference between a data source and an offline store?
The data source and the offline store are closely tied, but separate concepts. The offline store controls how feast talks to a data store for historical feature retrieval, and the data source points to specific table (or query) within a data store. Offline stores are infrastructure-level connectors to data stores like Snowflake.
Additional differences:
Data sources may be specific to a project (e.g. feed ranking), but offline stores are agnostic and used across projects.
A feast project may define several data sources that power different feature views, but a feast project has a single offline store.
Feast users typically need to define data sources when using feast, but only need to use/configure existing offline stores without creating new ones.
Using the s3_endpoint_override in a FileSource data source. This endpoint is more suitable for quick proof of concepts that won't necessarily scale for production use cases.
For more details on contributing to the Feast community, see here and this here.
Feast 0.9 (legacy)
What is the difference between Feast 0.9 and Feast 0.10+?
Feast 0.10+ is much lighter weight and more extensible than Feast 0.9. It is designed to be simple to install and use. Please see this document for more details.
How do I migrate from Feast 0.9 to Feast 0.10+?
Please see this document. If you have any questions or suggestions, feel free to leave a comment on the document!
What are the plans for Feast Core, Feast Serving, and Feast Spark?
Feast Core and Feast Serving were both part of Feast Java. We plan to support Feast Serving. We will not support Feast Core; instead we will support our object store based registry. We will not support Feast Spark. For more details on what we plan on supporting, please see the roadmap.
Real-time credit scoring on AWS
Credit scoring models are used to approve or reject loan applications. In this tutorial we will build a real-time credit scoring system on AWS.
When individuals apply for loans from banks and other credit providers, the decision to approve a loan application is often made through a statistical model. This model uses information about a customer to determine the likelihood that they will repay or default on a loan, in a process called credit scoring.
In this example, we will demonstrate how a real-time credit scoring system can be built using Feast and Scikit-Learn on AWS, using feature data from S3.
This real-time system accepts a loan request from a customer and responds within 100ms with a decision on whether their loan has been approved or rejected.
Deploying Redshift as the interface Feast uses to build training datasets
Registering your features with Feast and configuring DynamoDB for online serving
Building a training dataset with Feast to train your credit scoring model
Loading feature values from S3 into DynamoDB
Making online predictions with your credit scoring model using features from DynamoDB
RAG Fine Tuning with Feast and Milvus
Introduction
This example notebook provides a step-by-step demonstration of building and using a RAG system with Feast and the custom FeastRagRetriever. The notebook walks through:
Clone this repository: https://github.com/feast-dev/feast.git Navigate to the examples/rag-retriever directory. Here you will find the following files:
feature_repo/feature_store.yaml This is the core configuration file for the RAG project's feature store, configuring a Milvus online store on a local provider.
In order to configure Milvus you should:
Update feature_store.yaml with your Milvus connection details:
host
port (default: 19530)
credentials (if required)
feature_repo/ragproject_repo.py This is the Feast feature repository configuration that defines the schema and data source for Wikipedia passage embeddings.
rag_feast.ipynb This is a notebook demonstrating the implementation of a RAG system using Feast. The notebook provides:
A complete end-to-end example of building a RAG system with:
Data preparation using the Wiki DPR dataset
Open rag_feast.ipynb and follow the steps in the notebook to run the example.
Using DocEmbedder for Simplified Ingestion
As an alternative to the manual data preparation steps in the notebook above, Feast provides the DocEmbedder class that automates the entire document-to-embeddings pipeline: chunking, embedding generation, FeatureView creation, and writing to the online store.
Install Dependencies
pip install feast[milvus,rag]
Quick Start
from feast import DocEmbedder
from datasets import load_dataset
# Load your dataset
dataset = load_dataset("facebook/wiki_dpr", "psgs_w100.nq.exact", split="train[:1%]",
with_index=False, trust_remote_code=True)
df = dataset.select(range(100)).to_pandas()
# DocEmbedder handles everything in one step
embedder = DocEmbedder(
repo_path="feature_repo_docembedder/",
feature_view_name="text_feature_view",
)
result = embedder.embed_documents(
documents=df,
id_column="id",
source_column="text",
column_mapping=("text", "text_embedding"),
)
What DocEmbedder Does
Generates a FeatureView: Automatically creates a Python file with Entity and FeatureView definitions compatible with feast apply
Applies the repo: Registers the FeatureView in the Feast registry and deploys infrastructure (e.g., Milvus collection)
Chunks documents: Splits text into smaller passages using TextChunker (configurable chunk size, overlap, etc.)
Generates embeddings: Produces vector embeddings using MultiModalEmbedder (defaults to all-MiniLM-L6-v2)
Writes to online store: Stores the processed data in your configured online store (e.g., Milvus)
Customization
Custom Chunker: Subclass BaseChunker for your own chunking strategy
Custom Embedder: Subclass BaseEmbedder to use a different embedding model
Logical Layer Function: Provide a SchemaTransformFn to control how the output maps to your FeatureView schema
Example Notebook
See rag_feast_docembedder.ipynb for a complete end-to-end example that uses DocEmbedder with the Wiki DPR dataset and then queries the results using FeastRAGRetriever.
FeastRagRetriver Low Level Design
Helpful Information
Ensure your Milvus instance is properly configured and running
Vector dimensions and similarity metrics can be adjusted in the feature store configuration
The example uses Wikipedia data, but the system can be adapted for other datasets
Multi-Team Feature Store Setup
A multi-team feature store architecture (sometimes called a "federated" feature store) allows multiple teams to collaborate on a shared Feast registry while maintaining clear ownership boundaries. This pattern is particularly useful for organizations with multiple teams or projects that need to share features while preserving autonomy.
Overview
In a multi-team setup, you typically have:
Platform Repository (Central): A single repository managed by the platform team that acts as the source of truth for core infrastructure objects like entities, data sources, and batch/stream feature views.
Team Repositories (Distributed): Team-owned repositories for training pipelines and inference services. Teams define their own FeatureServices and On-Demand Feature Views (ODFVs) and apply them safely without affecting objects they don't own.
Key Concept: Understanding partial=True vs partial=False
The partial parameter in FeatureStore.apply() controls how Feast handles object deletions:
partial=True (default): Only adds or updates the specified objects. Does NOT delete any existing objects in the registry. This is safe for teams to use when they only want to manage a subset of objects.
partial=False: Performs a full synchronization - adds, updates, AND deletes objects. Objects not in the provided list (but tracked by Feast) will be removed from the registry. This should only be used by the platform team with full control.
Architecture Diagram
Setting Up the Platform Repository
The platform repository maintains full control over core Feast objects.
1. Platform feature_store.yaml
2. Platform Repository Structure
3. Platform Object Definitions
entities.py
data_sources.py
feature_views.py
4. Platform Apply Script
apply.py
Setting Up Team Repositories
Team repositories add their own FeatureServices and ODFVs without interfering with platform-managed objects.
1. Team feature_store.yaml
Teams use the same registry configuration as the platform:
2. Team Repository Structure
3. Team Object Definitions
on_demand_views.py
feature_services.py
4. Team Apply Script
apply.py
Ownership Boundaries
Object Type
Platform Repository
Team Repositories
Strategies for Avoiding Registry Drift
1. Naming Conventions
Establish clear naming conventions to prevent collisions:
2. CI/CD Integration
Use CI/CD pipelines to automate applies and catch errors early:
3. Registry Validation
Implement validation to check for ownership violations:
4. Read-Only Access for Teams
Configure infrastructure permissions so teams have:
Read access to the registry (to get platform objects)
Write access only for their specific object types
No delete permissions on registry objects
Multi-Tenant Configuration with Schema Isolation
For stronger isolation, use separate database schemas per team:
Platform Configuration
Team Configuration with Custom Schema
This approach allows teams to materialize features to their own schemas while sharing the central registry.
Complete Working Example
Here's a complete example showing both platform and team repositories in action:
Platform Repository
Team Repository
Using the Features
Best Practices
Platform Team Responsibilities:
Maintain core entities and data sources
Define and manage batch/stream feature views
Troubleshooting
Problem: Team accidentally deleted platform objects
Cause: Team used partial=False instead of partial=True.
Solution:
Restore from registry backup
Add validation in team CI/CD to prevent partial=False
Set registry permissions to prevent teams from deleting objects
Problem: Feature service references unknown feature view
Cause: Feature view not yet applied by platform team.
Solution:
Ensure platform applies are run before team applies
Use CI/CD dependencies to enforce ordering
Add validation to check that referenced feature views exist
Problem: Registry conflicts between teams
Cause: Teams using same object names.
Solution:
Enforce naming conventions with prefixes
Add validation to check for name collisions
Consider using separate projects per team (advanced)
Related Resources
- Multi-environment setup patterns
- Remote registry configuration
- Registry server setup
Summary
A multi-team feature store architecture enables scalable collaboration across multiple teams:
The platform team maintains core objects (entities, sources, views) with partial=False
Individual teams safely add their FeatureServices and ODFVs with partial=True
Clear ownership boundaries prevent accidental deletions and conflicts
This pattern (sometimes referred to as "federated" feature stores) allows organizations to scale Feast usage across many teams while maintaining a consistent, well-governed feature store.
Running Feast in production (e.g. on Kubernetes)
Overview
After learning about Feast concepts and playing with Feast locally, you're now ready to use Feast in production. This guide aims to help with the transition from a sandbox project to production-grade deployment in the cloud or on-premise (e.g. on Kubernetes).
A typical production architecture looks like:
Overview
Important note: Feast is highly customizable and modular.
Most Feast blocks are loosely connected and can be used independently. Hence, you are free to build your own production configuration.
For example, you might not have a stream source and, thus, no need to write features in real-time to an online store. Or you might not need to retrieve online features. Feast also often provides multiple options to achieve the same goal. We discuss tradeoffs below.
Additionally, please check the how-to guide for some specific recommendations on how to scale Feast.
Looking for production deployment patterns? See the Feast Production Deployment Topologies guide for three Kubernetes-ready topologies (Minimal, Standard, Enterprise), sample FeatureStore CRs, RBAC policies, infrastructure recommendations, and scaling best practices.
In this guide we will show you how to:
Deploy your feature store and keep your infrastructure in sync with your feature repository
Keep the data in your online store up to date (from batch and stream sources)
Use Feast for model training and serving
1. Automatically deploying changes to your feature definitions
1.1 Setting up a feature repository
The first step to setting up a deployment of Feast is to create a Git repository that contains your feature definitions. The recommended way to version and track your feature definitions is by committing them to a repository and tracking changes through commits. If you recall, running feast apply commits feature definitions to a registry, which users can then read elsewhere.
1.2 Setting up a database-backed registry
Out of the box, Feast serializes all of its state into a file-based registry. When running Feast in production, we recommend using the more scalable SQL-based registry that is backed by a database. Details are available here.
Note: A SQL-based registry primarily works with a Python feature server. The Java feature server does not understand this registry type yet.
1.3 Setting up CI/CD to automatically update the registry
We recommend typically setting up CI/CD to automatically run feast plan and feast apply when pull requests are opened / merged.
1.4 Setting up multiple environments
A common scenario when using Feast in production is to want to test changes to Feast object definitions. For this, we recommend setting up a staging environment for your offline and online stores, which mirrors production (with potentially a smaller data set).
Having this separate environment allows users to test changes by first applying them to staging, and then promoting the changes to production after verifying the changes on staging.
Different options are presented in the how-to guide.
2. How to load data into your online store and keep it up to date
To keep your online store up to date, you need to run a job that loads feature data from your feature view sources into your online store. In Feast, this loading operation is called materialization.
2.1 Scalable Materialization
Out of the box, Feast's materialization process uses an in-process materialization engine. This engine loads all the data being materialized into memory from the offline store, and writes it into the online store.
This approach may not scale to large amounts of data, which users of Feast may be dealing with in production. In this case, we recommend using one of the more scalable compute engines, such as Snowflake Compute Engine. Users may also need to write a custom compute engine to work on their existing infrastructure.
It is up to you to orchestrate and schedule runs of materialization.
Feast keeps the history of materialization in its registry so that the choice could be as simple as a unix cron util. Cron util should be sufficient when you have just a few materialization jobs (it's usually one materialization job per feature view) triggered infrequently.
However, the amount of work can quickly outgrow the resources of a single machine. That happens because the materialization job needs to repackage all rows before writing them to an online store. That leads to high utilization of CPU and memory. In this case, you might want to use a job orchestrator to run multiple jobs in parallel using several workers. Kubernetes Jobs or Airflow are good choices for more comprehensive job orchestration.
For large datasets, you can also reduce peak memory on the materialization worker by setting online_write_batch_size in feature_store.yaml. This breaks the proto conversion and write into chunks instead of loading the entire dataset into memory at once:
If you are using Airflow as a scheduler, Feast can be invoked through a PythonOperator after the Python SDK has been installed into a virtual environment and your feature repo has been synced:
from airflow.decorators import task
from feast import RepoConfig, FeatureStore
from feast.infra.online_stores.dynamodb import DynamoDBOnlineStoreConfig
from feast.repo_config import RegistryConfig
# Define Python callable
@task()
def materialize(data_interval_start=None, data_interval_end=None):
repo_config = RepoConfig(
registry=RegistryConfig(path="s3://[YOUR BUCKET]/registry.pb"),
project="feast_demo_aws",
provider="aws",
offline_store="file",
online_store=DynamoDBOnlineStoreConfig(region="us-west-2"),
entity_key_serialization_version=3
)
store = FeatureStore(config=repo_config)
# Option 1: materialize just one feature view
# store.materialize_incremental(datetime.datetime.now(), feature_views=["my_fv_name"])
# Option 2: materialize all feature views incrementally
# store.materialize_incremental(datetime.datetime.now())
# Option 3: Let Airflow manage materialization state
# Add 1 hr overlap to account for late data
store.materialize(data_interval_start.subtract(hours=1), data_interval_end)
Important note: Airflow worker must have read and write permissions to the registry file on GCS / S3 since it pulls configuration and updates materialization history.
2.3 Stream feature ingestion
See more details at data ingestion, which shows how to ingest streaming features or 3rd party feature data via a push API.
This supports pushing feature values into Feast to both online or offline stores.
2.4 Scheduled batch transformations with Airflow + dbt
Feast does not orchestrate batch transformation DAGs. For this, you can rely on tools like Airflow + dbt. See Feast Workshop - Module 3 for an example and some tips.
After we've defined our features and data sources in the repository, we can generate training datasets. We highly recommend you use a FeatureService to version the features that go into a specific model version.
The first thing we need to do in our training code is to create a FeatureStore object with a path to the registry.
One way to ensure your production clients have access to the feature store is to provide a copy of the feature_store.yaml to those pipelines. This feature_store.yaml file will have a reference to the feature store registry, which allows clients to retrieve features from offline or online stores.
from feast import FeatureStore
fs = FeatureStore(repo_path="production/")
Then, you need to generate an entity dataframe. You have two options
Create an entity dataframe manually and pass it in
Use a SQL query to dynamically generate lists of entities (e.g. all entities within a time range) and timestamps to pass into Feast
Then, training data can be retrieved as follows:
3.2 Versioning features that power ML models
The most common way to productionize ML models is by storing and versioning models in a "model store", and then deploying these models into production. When using Feast, it is recommended that the feature service name and the model versions have some established convention.
For example, in MLflow:
import mlflow.pyfunc
# Load model from MLflow
model_name = "my-model"
model_version = 1
model = mlflow.pyfunc.load_model(
model_uri=f"models:/{model_name}/{model_version}"
)
fs = FeatureStore(repo_path="production/")
# Read online features using the same model name and model version
feature_vector = fs.get_online_features(
features=fs.get_feature_service(f"{model_name}_v{model_version}"),
entity_rows=[{"driver_id": 1001}]
).to_dict()
# Make a prediction
prediction = model.predict(feature_vector)
It is important to note that both the training pipeline and model serving service need only read access to the feature registry and associated infrastructure. This prevents clients from accidentally making changes to the feature store.
4. Retrieving online features for prediction
Once you have successfully loaded data from batch / streaming sources into the online store, you can start consuming features for model inference.
4.1. Use the Python SDK within an existing Python service
This approach is the most convenient to keep your infrastructure as minimalistic as possible and avoid deploying extra services. The Feast Python SDK will connect directly to the online store (Redis, Datastore, etc), pull the feature data, and run transformations locally (if required). The obvious drawback is that your service must be written in Python to use the Feast Python SDK. A benefit of using a Python stack is that you can enjoy production-grade services with integrations with many existing data science tools.
To integrate online retrieval into your service use the following code:
from feast import FeatureStore
with open('feature_refs.json', 'r') as f:
feature_refs = json.loads(f)
fs = FeatureStore(repo_path="production/")
# Read online features
feature_vector = fs.get_online_features(
features=feature_refs,
entity_rows=[{"driver_id": 1001}]
).to_dict()
5. Using environment variables in your yaml configuration
You might want to dynamically set parts of your configuration from your environment. For instance to deploy Feast to production and development with the same configuration, but a different server. Or to inject secrets without exposing them in your git repo. To do this, it is possible to use the ${ENV_VAR} syntax in your feature_store.yaml file. For instance:
In summary, the overall architecture in production may look like:
Feast SDK is being triggered by CI (eg, Github Actions). It applies the latest changes from the feature repo to the Feast database-backed registry
Data ingestion
Batch data: Airflow manages batch transformation jobs + materialization jobs to ingest batch data from DWH to the online store periodically. When working with large datasets to materialize, we recommend using a batch materialization engine
If your offline and online workloads are in Snowflake, the Snowflake materialization engine is likely the best option.
If your offline and online workloads are not using Snowflake, but using Kubernetes is an option, the Bytewax materialization engine is likely the best option.
If none of these engines suite your needs, you may continue using the in-process engine, or write a custom engine (e.g with Spark or Ray).
Stream data: The Feast Push API is used within existing Spark / Beam pipelines to push feature values to offline / online stores
Online features are served via the Python feature server over HTTP, or consumed using the Feast Python SDK.
Feast Python SDK is called locally to generate a training dataset
From Repository to Production: Feast Production Architecture
Feast on Kubernetes
This page covers deploying Feast on Kubernetes, including the Feast Operator and feature servers.
Overview
Kubernetes is a common target environment for running Feast in production. You can use Kubernetes to:
Run Feast feature servers for online feature retrieval.
Run scheduled and ad-hoc jobs (e.g. materialization jobs) as Kubernetes Jobs.
Operate Feast components using Kubernetes-native primitives.
Planning a production deployment? See the Feast Production Deployment Topologies guide for architecture diagrams, sample FeatureStore CRs, RBAC policies, infrastructure recommendations, and scaling best practices across Minimal, Standard, and Enterprise topologies.
Feast Operator
To deploy Feast components on Kubernetes, use the included feast-operator.
For first-time Operator users, it may be a good exercise to try the Feast Operator Quickstart. The quickstart demonstrates some of the Operator's built-in features, e.g. git repos, feast apply jobs, etc.
More advanced FeatureStore CR examples can be found in the feast-operator samples directory.
Advanced configuration: To configure persistence, metrics, MCP, OpenLineage, security, batch jobs, and more via the operator, see the Operator Configuration Guides.
Upgrading the Operator
OLM-managed installations
If the operator was installed via OLM, upgrades are handled automatically. No manual steps are required — OLM recreates the operator Deployment during the upgrade process.
kubectl-managed installations
For most upgrades, re-running the install command is sufficient:
One-time step: upgrading from versions before 0.61.0
Version 0.61.0 updated the operator Deployment's spec.selector to include the app.kubernetes.io/name: feast-operator label, fixing a bug where the metrics service could accidentally target pods from other operators in shared namespaces.
Because Kubernetes treats spec.selector as an immutable field, upgrading directly from a pre-0.61.0 version with kubectl apply will fail with:
The Deployment "feast-operator-controller-manager" is invalid: spec.selector: Invalid value: ... field is immutable
To resolve this, delete the existing operator Deployment before applying the new manifest:
This is only required once. Existing FeatureStore CRs and their managed workloads (feature servers, registry, etc.) are not affected — the new operator pod will reconcile them automatically on startup. Future upgrades from 0.61.0 onward will not require this step.
Scaling & High Availability: The Feast Operator supports horizontal scaling via static replicas, HPA autoscaling, or external autoscalers like KEDA. Scaling requires DB-backed persistence for all enabled services.
When scaling is enabled, the operator auto-injects soft pod anti-affinity and zone topology spread constraints for resilience. You can also configure a PodDisruptionBudget to protect against voluntary disruptions.
Both spec.openlineage and spec.materialization are written into feature_store.yaml for all service pods — they apply to the online server, offline server, registry, and materialization jobs alike.
OpenLineage Data Lineage (spec.openlineage)
OpenLineage emits data lineage events during feast apply (registry changes) and materialization. Events go outbound from Feast pods to your OpenLineage-compatible backend (Marquez, any OpenLineage HTTP endpoint, Kafka, or a file). No inbound ports or additional Kubernetes Services are required.
Dependency: the Feast image must include feast[openlineage] (openlineage-python).
The operator reads the api_key value from the Secret and writes it into feature_store.yaml. The Secret must be in the same namespace as the FeatureStore.
Limits the number of rows written per batch during materialization. Without this, all rows for a feature view are written in a single batch — this can cause OOM for large feature views.
Supported engines: local, Spark, Ray
Minimum value: 1 (enforced by CRD validation)
materialization:
onlineWriteBatchSize: 10000 # 10 k rows per batch
extraConfig
Passes additional MaterializationConfig settings inline. Boolean strings ("true" / "false") are coerced to native YAML booleans.
materialization:
extraConfig:
pull_latest_features: "false" # only materialize the latest value per entity
Key
Type
Description
pull_latest_features
bool string
When "true", only the latest feature value per entity is materialized. Default is engine-dependent
This guide covers end-to-end performance tuning for the Feast Python online feature server, with a focus on Kubernetes deployments using the Feast Operator. It brings together worker configuration, registry caching, online store selection, horizontal scaling, observability, and network optimization into a single operational playbook.
Request lifecycle
Understanding where time is spent during a get_online_features() call helps you focus tuning efforts on the right layer:
Registry lookup — resolve feature references to metadata (cached locally).
Online store read — fetch feature values from the backing database (network I/O).
On-demand transformation — execute any ODFVs attached to the request (CPU).
Response serialization — encode the result as JSON or protobuf.
In most production deployments, step 2 (online store read) dominates latency. Registry refresh (step 1) can cause periodic latency spikes if not configured for background refresh. Step 3 matters only when ODFVs are present.
Feature view and feature service structuring
How you organize features into feature views directly affects online serving latency, because each distinct feature view in a request produces a separate online store read. This is a structural decision that no amount of runtime tuning can fix after the fact.
Fewer feature views = fewer store round-trips
When the server processes a get_online_features() call, it groups the requested features by their source feature view and issues one online_read() per group. For sync stores, these reads are sequential; for async stores (DynamoDB, MongoDB), they run concurrently via asyncio.gather(), but each is still a separate network round-trip.
Request shape
Store reads
Impact
10 features from 1 feature view
1 read
Minimal latency
10 features from 5 feature views
5 reads
5× the network round-trips (sequential) or 5 concurrent reads competing for connections (async)
Guideline: For features that share the same entity key and are frequently requested together, consolidate them into a single feature view. This reduces the number of store round-trips per request. Split feature views only when features have different entities, different materialization schedules, or different data source update frequencies.
Redis exception: The Redis online store overrides get_online_features() to batch all HMGET commands across every feature view into a single pipeline execution. Because all feature views for the same entity share one Redis hash key, the number of Redis round trips is always 1, regardless of how many feature views the request touches. This means the "fewer feature views" guideline is less critical for Redis than for other stores — but consolidating feature views still reduces serialization and protobuf overhead at the application layer.
Feature services are free
A Feature Service is a named collection of feature references — it's a convenience grouping, not a separate storage or execution unit. Using a feature service adds only a registry lookup (cached) compared to listing features individually. There is no performance penalty for using feature services, and they are the recommended way to define stable, versioned feature sets for production models.
ODFV overhead is additive
Regular feature views incur only store read cost. On-demand feature views add CPU-bound transformation cost on top:
Total request latency ≈ store_read_time + sum(odfv_transformation_times)
Each ODFV in the request runs its transformation function after all store reads complete. The overhead depends on:
Number of ODFVs in the request — each one adds its transformation time.
Transformation complexity — a simple arithmetic operation is microseconds; ML model inference can be milliseconds.
Transformation mode — mode="python" is 3–10× faster than mode="pandas" for online serving (see ODFV optimization).
Entity count — transformations run across all entity rows in the request. More entities = more work.
Hidden store reads from ODFV source dependencies
ODFVs declare source feature views. When you request features from an ODFV, the server must also read all source feature views that the ODFV depends on, even if you didn't explicitly request features from those views. This can add unexpected store reads:
@on_demand_feature_view(
sources=[driver_stats_fv, vehicle_stats_fv, request_source],
schema=[Field(name="combined_score", dtype=Float64)],
mode="python",
)
def combined_score(inputs: dict[str, Any]) -> dict[str, Any]:
return {"combined_score": [d + v for d, v in zip(inputs["driver_rating"], inputs["vehicle_rating"])]}
Requesting just combined_score triggers reads from bothdriver_stats_fv and vehicle_stats_fv. If those are already needed by other features in the same request, there's no extra cost (the server deduplicates). But if they're only needed by the ODFV, these are hidden reads you should account for.
Structuring recommendations
Practice
Why
Group co-accessed features into the same feature view
Reduces store round-trips per request
Use feature services to define stable production feature sets
No performance cost; improves governance and versioning
Minimize the number of ODFVs per request
Each ODFV adds CPU-bound transformation latency
Prefer write_to_online_store=True for ODFVs that don't need request-time data
Worker and connection tuning
The Python feature server uses Gunicorn with async workers. Tuning workers, connections, and timeouts directly impacts throughput and tail latency.
Each worker is a separate process. More workers = more parallel requests, but also more memory.
workerConnections
1000
Sizing rule of thumb: Start with workers = 2 × vCPU + 1 and allocate ~256–512 MB memory per worker. Monitor RSS per process and adjust. If p99 latency is high under load, add more workers or scale horizontally rather than increasing connections.
Registry cache tuning
The default registry configuration uses synchronous cache refresh. When the TTL expires, the nextget_online_features() call blocks while the full registry is re-downloaded. For large registries this can add tens of milliseconds to a single request.
The problem
With the default cache_mode: sync (or unset), the refresh cycle looks like:
Request N → cache hit (fast)
...TTL expires...
Request N+1 → cache miss → synchronous download → response delayed
Request N+2 → cache hit (fast)
The fix: background thread refresh
Configure the registry to refresh in a background thread so that no serving request ever blocks on a download:
For DB/SQL-backed registries deployed via the Operator, cache_mode and cache_ttl_seconds are set inside the secret's YAML payload rather than on the CR struct. Ensure your secret includes these keys.
The trade-off: freshness vs. performance
Registry caching is a freshness vs. performance trade-off. The registry contains metadata — feature view definitions, entity schemas, data source configs — not the feature values themselves. When someone runs feast apply to add or modify a feature view, the change is invisible to the serving pods until the cache refreshes.
Lower TTL (e.g., 10–30s):
Schema changes propagate faster to serving pods.
But each refresh re-downloads and deserializes the entire registry, consuming CPU and network bandwidth.
With cache_mode: sync, a lower TTL means more frequent latency spikes — the unlucky request that triggers the refresh is blocked for the full download duration.
With cache_mode: thread, no request is directly blocked, but the background thread competes with Gunicorn workers for CPU during deserialization. On CPU-constrained pods, frequent refreshes can indirectly increase p99 serving latency by starving workers of CPU cycles.
Higher TTL (e.g., 300–600s):
Fewer refreshes mean less CPU contention, resulting in more stable and predictable serving latency.
But schema changes take longer to propagate. A new feature view added via feast apply won't be queryable until the cache refreshes (up to cache_ttl_seconds delay).
This is usually acceptable in production where schema changes are infrequent and deployed through CI/CD pipelines, not ad-hoc.
Memory impact: The full registry is deserialized into memory on each worker process. For registries with hundreds of feature views, this can be tens of megabytes per worker. With 4 workers × 5 replicas, that's 20 copies of the registry in memory. A higher TTL doesn't reduce memory usage (the cache is always held), but it reduces the CPU cost of periodic deserialization — which in turn keeps CPU available for serving requests with lower latency.
The registryTTLSeconds field on the Operator CR (or --registry_ttl_sec CLI flag) controls how often the online server refreshes its cache. The cache_ttl_seconds in feature_store.yaml controls how often the SDK client refreshes. In an Operator deployment, the CR field is what matters for server performance.
Recommendations
Scenario
cache_mode
cache_ttl_seconds
Development / iteration
sync (default)
5–10
Production (low-latency)
thread
300
Online store selection
The online store is the single largest factor in get_online_features() latency. Choose based on your latency budget, throughput needs, and existing infrastructure.
Store
Typical p50 latency
Async read
Best for
Key trade-off
Redis / Dragonfly
< 1 ms
No (threadpool)
Ultra-low latency, high throughput; all FV reads batched into 1 pipeline
Requires in-memory capacity for your dataset
General rule: If your p99 latency budget is under 10 ms, use Redis. If you need serverless scaling on AWS, use DynamoDB. For everything else, PostgreSQL with connection pooling is a strong default.
Async vs sync online reads
The feature server can read from the online store using either an async or sync code path. The choice is made automatically based on the store's async_supported property — no user configuration is needed.
How it works:
When a get_online_features() request touches multiple feature views, the server must read from each one.
Async path (async_supported.online.read = True): All feature view reads run concurrently via asyncio.gather() in a single event loop — no thread overhead, minimal context switching.
Sync path (default fallback): The sync online_read() is wrapped in run_in_threadpool(), which dispatches to a thread pool. This works correctly but adds thread scheduling overhead, especially when reading from many feature views.
Current async support by store:
Store
Async read
Async write
Notes
DynamoDB
Yes
Yes
Uses aiobotocore for non-blocking I/O
MongoDB
Yes
When async matters most:
Requests that fan out across many feature views benefit the most — async gathers all reads concurrently without thread pool contention.
For requests hitting a single feature view, the difference is negligible.
If you are choosing between two stores with comparable latency, prefer the one with full async support for better throughput under concurrent load.
batch_size: DynamoDB's BatchGetItem accepts up to 100 items per request. For 500 entities, this means 5 batches. Keep at 100 unless hitting the 16 MB response limit.
max_read_workers: Controls parallelism for batch reads. With 10 workers, those 5 batches run concurrently (~10 ms) instead of sequentially (~50 ms).
consistent_reads: false: Eventually consistent reads are faster and cheaper. Use true only if you need read-after-write consistency.
max_pool_connections: Increase for high-throughput deployments to improve HTTP connection reuse to the DynamoDB endpoint.
keepalive_timeout: Longer keep-alive reduces TLS handshake overhead on reused connections.
connect_timeout / read_timeout: Lower values fail fast, improving p99. Set aggressively if your retry strategy covers transient failures.
total_max_retry_attempts: 2: Reduces p99 by capping retry delay. Combine with retry_mode: adaptive for backpressure-aware retries.
VPC Endpoints: Configure a VPC endpoint for DynamoDB to keep traffic on AWS's private network. This reduces latency by eliminating the public internet hop and can also reduce data transfer costs.
conn_type: pool: Enables connection pooling via psycopg pool. Without this, every request opens and closes a TCP connection.
min_conn / max_conn: Set min_conn to your expected steady-state concurrency and max_conn to your burst limit. Keep max_conn below the PostgreSQL max_connections divided by your replica count.
keepalives_idle: TCP keep-alive prevents idle connections from being dropped by firewalls or load balancers.
Redis tuning
online_store:
type: redis
connection_string: "redis-cluster.internal:6379,ssl=true"
redis_type: redis_cluster
key_ttl_seconds: 604800
skip_dedup: false # set true for initial bulk loads to halve write round trips
Use redis_cluster for horizontal partitioning across shards.
Set key_ttl_seconds to auto-expire stale feature data, keeping memory usage bounded. This is a key-level TTL — it expires the entire entity hash (all feature views for that entity) together.
Ensure the Redis instance is in the same availability zone as the feature server pods to minimize network round-trips.
Batched multi-feature-view reads
The Redis online store overrides get_online_features() to issue all HMGET commands — across every feature view in the request — in a single pipeline execution. This reduces Redis round trips from N (one per feature view) to 1 regardless of request size.
Feature views
Round trips (other stores)
Round trips (Redis)
1
1
1
5
5
1
This means the latency cost of adding feature views to a Redis-backed request is primarily serialization and protobuf overhead at the application layer, not Redis network latency.
Write throughput: skip_dedup
For initial bulk loads or append-only materialization pipelines:
With skip_dedup: true, online_write_batch() skips the existing-timestamp read pipeline and writes all values directly in a single pass, halving write round trips. Use only when you can guarantee write ordering — under concurrent writers, an older record can overwrite a newer one.
Cassandra / ScyllaDB tuning
Cassandra and ScyllaDB share the same Feast connector. The key read-path knobs are concurrency and data-center-aware routing:
read_concurrency (default: 100): Maximum concurrent async read operations. Increase for high fan-out workloads; decrease if the cluster shows read-timeout pressure.
write_concurrency (default: 100): Maximum concurrent async write operations during materialization.
request_timeout (default: driver default): Per-request timeout in seconds. Lower values fail fast and improve p99 at the cost of higher error rates under load.
load_balancing.local_dc: Set to your local data center name so the driver routes reads to the nearest replicas, avoiding cross-DC latency.
Bigtable tuning
Bigtable's client library uses an internal connection pool that caps concurrency:
The client library's connection pool size is hardcoded at 10 internally. For workloads that need higher concurrent reads, scale horizontally (more feature server replicas) rather than trying to push more concurrency through a single pod.
max_versions (default: 2): Number of cell versions retained. Lower values reduce storage and read amplification.
Co-locate the feature server in the same GCP region as your Bigtable instance.
MongoDB tuning
MongoDB supports async reads natively. Use client_kwargs to pass performance options directly to the PyMongo / Motor driver:
maxPoolSize / minPoolSize: Controls the driver connection pool. Default maxPoolSize is 100 in PyMongo, but explicitly setting it ensures predictability across versions.
connectTimeoutMS / socketTimeoutMS: Tighter timeouts improve p99 by failing fast on slow connections.
MongoDB is one of the stores with full async support (read and write), so it benefits from concurrent feature view reads via asyncio.gather().
Remote online store tuning
The Remote online store connects to a Feast feature server over HTTP. Connection pooling is critical:
connection_pool_size (default: 50): Number of persistent HTTP connections. Increase for high-throughput deployments to reduce connection setup overhead.
connection_idle_timeout (default: 300s): Seconds before idle connections are closed. Set to 0 to disable idle cleanup (useful when traffic is bursty).
connection_retries (default: 3): Number of retries on transient HTTP failures. Lower for tighter p99; higher for better reliability.
Batch size tuning
Different online stores have different optimal batch sizes for get_online_features() calls. The batch size controls how many entity keys are fetched per round-trip to the store.
Store
Default batch size
Max batch size
Recommendation
DynamoDB
100
100 (API limit)
Keep at 100; tune max_read_workers for parallelism
Redis
N/A (pipelined)
Profile your workload by measuring feast_feature_server_online_store_read_duration_seconds (see Metrics setup) across different entity counts to find the sweet spot.
On-demand feature view (ODFV) optimization
If your feature pipeline uses ODFVs, the transformation mode significantly affects serving latency.
Prefer native Python mode
Native Python transformations avoid pandas overhead and are substantially faster for online serving:
Offline batch retrieval where pandas ergonomics matter
Use singleton mode for single-row transforms
When your Python ODFV processes one entity at a time (common in online serving), use singleton=True to avoid the overhead of wrapping and unwrapping single values in lists:
With singleton=True, input values are plain scalars (not lists), and the function returns scalars. This avoids list comprehension and zip overhead for single-entity requests. Singleton mode requires mode="python".
Write-time transformations
For ODFVs that don't depend on request-time data, set write_to_online_store=True to compute features during materialization instead of at serving time:
@on_demand_feature_view(
sources=[driver_stats_fv],
schema=[Field(name="is_high_mileage", dtype=Bool)],
mode="python",
write_to_online_store=True,
)
def high_mileage(inputs: dict[str, Any]) -> dict[str, Any]:
return {"is_high_mileage": [m > 100000 for m in inputs["total_miles"]]}
This moves CPU cost from the serving path to the materialization path, reducing online latency.
Consistency vs. latency: choosing between read-time and write-time transforms
Read-time ODFVs (the default, write_to_online_store=False) are request-blocking — they execute synchronously during get_online_features(). This provides a strong consistency guarantee: the output is always computed from the latest source features in the online store and can incorporate request-time data (e.g., the user's current location or the current timestamp).
Write-time ODFVs (write_to_online_store=True) are pre-computed during materialization and stored. They add zero latency to the serving path, but the result can be stale — it reflects the source feature values at the time of the last materialization run, not the current request.
Use this decision framework:
Condition
Recommendation
ODFV uses request-time data (e.g., current timestamp, user input)
Must use read-time (default) — request data is not available at materialization
ODFV depends on features that change frequently and freshness matters
Keep read-time for strong consistency
ODFV is a pure derivation of slowly changing features (e.g., age buckets, tier labels)
Safe to move to write-time for lower latency
Moving an ODFV to write_to_online_store=True is a performance-consistency trade-off, not a free optimization. Verify that your use case tolerates stale results bounded by the materialization interval before switching.
Pre-load heavy resources with static artifacts
If your ODFV uses ML models, lookup tables, or other expensive resources, load them once at server startup using static artifacts loading instead of loading them per request:
# static_artifacts.py — loaded once at feature server startup
from fastapi import FastAPI
def load_artifacts(app: FastAPI):
from transformers import pipeline
app.state.model = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
import example_repo
example_repo._model = app.state.model
# example_repo.py — use pre-loaded artifact in ODFV
_model = None
@on_demand_feature_view(
sources=[text_fv],
schema=[Field(name="sentiment_score", dtype=Float64)],
mode="python",
)
def sentiment(inputs: dict[str, Any]) -> dict[str, Any]:
global _model
scores = [_model(t)[0]["score"] for t in inputs["text"]]
return {"sentiment_score": scores}
Without static artifacts, the model would be loaded on every request (or every worker restart), adding seconds of latency. With pre-loading, the ODFV pays only the inference cost.
Using static artifacts with the Feast Operator:
The feature server looks for static_artifacts.py in the feature repository directory at startup. Whether this works in an operator deployment depends on how the repo is created:
feastProjectDir.git — The operator clones your git repository via an init container. Include static_artifacts.py in the repo alongside your feature definitions and it will be available at startup. This is the recommended approach for production.
feastProjectDir.init (default) — The operator runs feast init to create a template repo. The generated template does not include static_artifacts.py. To use static artifacts in this mode, you would need to build a custom feature server image that bundles the file, or mount it via a ConfigMap/volume.
For production deployments that use ODFVs with heavy resources (ML models, large lookup tables), use feastProjectDir.git so your static_artifacts.py is version-controlled and deployed automatically alongside your feature definitions.
Transformation code best practices
Small choices inside transformation functions add up at scale:
Avoid I/O in the transform function. Network calls, file reads, or database queries inside an ODFV block the serving thread. Pre-load data via static artifacts or use write_to_online_store to compute offline.
Avoid creating DataFrames. Even in Python mode, constructing a pandas DataFrame inside the function defeats the purpose of avoiding pandas overhead.
Minimize allocations. Reuse structures where possible. List comprehensions are faster than building lists with .append() in a loop.
Profile with track_metrics=True. Enable per-ODFV timing to identify slow transforms, but disable it in production once you've tuned — the timer itself adds minor overhead.
@on_demand_feature_view(
sources=[driver_stats_fv],
schema=[Field(name="score", dtype=Float64)],
mode="python",
track_metrics=True, # enable during profiling, disable once tuned
)
def compute_score(inputs: dict[str, Any]) -> dict[str, Any]:
return {"score": [x * 0.5 + y * 0.3 for x, y in zip(inputs["a"], inputs["b"])]}
The feast_feature_server_transformation_duration_seconds metric (labeled by odfv_name and mode) shows exactly how much time each ODFV adds to the serving path.
Horizontal scaling
When a single pod cannot meet your throughput or latency requirements, scale horizontally using the Feast Operator.
Horizontal scaling requires DB-backed persistence for all services. File-based stores (SQLite, DuckDB, registry.db) do not support concurrent access from multiple pods. The Feast Operator supports both static replicas (spec.replicas) and HPA autoscaling (services.scaling.autoscaling). See Scaling Feast for full configuration examples.
Expected scaling efficiency
Replicas
Expected throughput
Notes
1
Baseline
Single-pod throughput depends on worker count and online store latency
2–3
~1.8–2.8x
Near-linear scaling; overhead from load balancer routing
Scaling the feature server is only effective if the online store can handle the aggregate connection and throughput load. Monitor the store's CPU, memory, and connection counts alongside feature server metrics.
Coordinating database connections with scaling
When you scale horizontally, every replica and every Gunicorn worker within a replica opens its own database connection pool to the online store. The total connection count multiplies quickly:
Total connections = replicas × workers_per_pod × max_conn_per_worker
Example: 5 replicas × 4 workers × 20 max_conn = 400 connections to the database. If your PostgreSQL instance has the default max_connections: 100, this will fail immediately.
This applies to every connection-oriented online store:
Store
Connection setting
Default
What to check on the store side
PostgreSQL
max_conn (per worker pool)
10
max_connections on the server (typically 100–500)
DynamoDB
How to size it:
Start from the store's limit. Determine the maximum connections your database can handle (e.g., PostgreSQL max_connections, or RDS instance class limit).
Reserve headroom for other clients (materialization jobs, feast apply, monitoring). A common rule is to reserve 20–30% of the store's capacity.
Divide by total workers. Set max_conn (or equivalent) per worker so the total stays within the budget:
# feature_store.yaml (in the Operator secret)
online_store:
type: postgres
conn_type: pool
min_conn: 2
max_conn: 7
...
If you use HPA autoscaling, size max_conn based on maxReplicas (not minReplicas) to avoid connection exhaustion during scale-out. Under-provisioning here causes cascading failures: the store rejects connections, requests fail, load increases, HPA scales up further, and even more connections are rejected.
For DynamoDB and other HTTP-based stores, the concern is less about hard connection limits and more about SDK-level pool exhaustion and throttling. Monitor ThrottledRequests (DynamoDB) or HTTP 429 responses and increase pool sizes or store capacity accordingly.
High availability
When scaling is enabled, the operator automatically injects:
Soft pod anti-affinity — prefers spreading pods across nodes.
Zone topology spread — distributes pods across availability zones (best-effort).
You can also configure a PodDisruptionBudget to limit voluntary disruptions:
This exposes a /metrics endpoint on port 8000. If the Prometheus Operator's ServiceMonitor CRD is installed, the Feast operator automatically creates a ServiceMonitor for scraping.
Key metrics for performance tuning
Metric
What to watch
feast_feature_server_request_latency_seconds
p50, p95, p99 latency by endpoint. The primary SLI.
See the OpenTelemetry Integration guide for the complete setup including Instrumentation CRDs and ServiceMonitors.
Network latency optimization
Network round-trip time to the online store is the dominant factor in get_online_features() latency. No amount of server-side tuning can compensate for a slow network path.
Co-locate feature server and online store
Deploy the feature server pods in the same AWS region and availability zone as your online store (DynamoDB, ElastiCache, RDS, etc.).
On Kubernetes or similar platforms, ensure the cluster and the managed database service share the same VPC and subnet.
Use private network endpoints
Public internet round-trips add 5–50 ms of latency compared to private network paths. Use VPC endpoints or private link:
Cloud
Service
Private endpoint
AWS
DynamoDB
AWS
ElastiCache (Redis)
Deploy in same VPC; no public endpoint needed
DNS resolution caching
Frequent DNS lookups to the store endpoint add latency. In Kubernetes, ensure CoreDNS caching is enabled (default) and consider using IP-based endpoints for critical-path connections.
Client access patterns: REST API vs. SDK with remote store
When consuming features from a Feast online server deployed via the Operator, there are three client access patterns with different performance profiles:
Pattern comparison
Pattern
How it works
Network hops
Client-side overhead
Best for
Direct REST API
HTTP POST to /get-online-features from any language
Client → feature server → online store
Minimal (JSON only)
Non-Python clients, lowest client-side latency, microservices in any language
Performance characteristics
Direct REST API has the lowest client-side overhead. The client sends a JSON payload and receives a JSON response — no SDK, no registry loading, no protobuf conversion. The feature server handles all processing (registry lookup, store read, ODFVs). This is the best option when:
Your inference service is written in Java, Go, C++, or another non-Python language.
You want to minimize client-side CPU and memory usage.
You're already behind a load balancer and want the simplest possible integration.
Feast SDK with remote store (online_store.type: remote) adds client-side overhead: the SDK loads the registry locally to resolve feature references, converts entities to protobuf, serializes to JSON for HTTP transport, and deserializes the response back to protobuf. This means:
The client needs access to the registry (or a remote registry server).
Each request pays for proto → JSON → proto round-trip conversion.
But you get the full SDK API: feature services, type checking, Python-native OnlineResponse objects, and built-in connection pooling.
store = FeatureStore(repo_path=".")
# Internally: registry lookup → proto conversion → HTTP POST → JSON parse → proto conversion
features = store.get_online_features(
features=["driver_stats:conv_rate", "driver_stats:acc_rate"],
entity_rows=[{"driver_id": 1001}, {"driver_id": 1002}],
)
Feast SDK direct (online_store.type: postgres/redis/dynamodb/...) skips the feature server entirely and reads from the online store directly. This eliminates the HTTP hop and JSON serialization, giving the lowest end-to-end latency for Python clients. However, it requires:
Direct network access from the client to the online store (often not available in production Kubernetes setups).
Online store credentials on the client side.
Each client manages its own connection pool to the store, which complicates connection budgeting at scale.
Recommendation for Operator deployments
In a typical Feast Operator deployment on Kubernetes:
Use the direct REST API for production inference services, especially non-Python ones. It has the least overhead and works with any language.
Use the SDK with remote store for Python services that benefit from the SDK ergonomics (feature services, type safety, OnlineResponse helpers).
Avoid SDK direct in most Operator deployments — the feature server already manages store connections, worker pools, and registry caching. Bypassing it means every client pod opens its own connections to the database, duplicating the connection budget problem.
Regardless of which pattern you use, the server-side performance is identical — the feature server processes /get-online-features the same way. The performance difference is entirely on the client side: how much work the client does before and after the HTTP call.
Putting it all together
Here is a complete Feast Operator CR for a production deployment with all the tuning recommendations applied:
The max_conn: 7 above is sized for the HPA configuration in this example (maxReplicas=10, 4 workers per pod). See Coordinating database connections with scaling for the calculation. Adjust based on your store's connection limit and replica count.
Materialization (feast materialize / feast materialize-incremental) reads features from the offline store and writes them to the online store. For large feature views, two common bottlenecks arise: memory exhaustion during proto conversion and write throughput to the online store.
Memory: online_write_batch_size
By default, Feast converts the entire Arrow table returned by the offline store into Python protobuf objects in a single pass before writing to the online store. For datasets with millions of rows this can consume tens of gigabytes of memory on the materialization worker.
Set online_write_batch_size in feature_store.yaml to break the write into manageable chunks:
materialization:
online_write_batch_size: 10000 # rows per write batch
Each chunk is independently converted and written, keeping peak memory proportional to the batch size rather than the full dataset. This is supported by the local, Spark, and Ray compute engines.
Dataset size
Without batching
With online_write_batch_size: 10000
1 M rows (100 bytes/row)
~100 MB peak
~1 MB peak
10 M rows
~1 GB peak
~1 MB peak
Choosing a value:
Larger batches (50 000+): fewer write calls to the online store, lower overhead per row — good when worker memory allows.
Smaller batches (1 000–5 000): lower peak memory — necessary for memory-constrained workers or very wide feature views (many features per row).
For Redis: pipeline overhead per batch is negligible; a batch size of 10 000–50 000 is a good starting point.
For DynamoDB: each batch maps to one or more BatchWriteItem calls (max 25 items per call); a larger online_write_batch_size amortizes the per-call overhead but doesn't change the 25-item DynamoDB limit.
Each feature view in a feast materialize call is materialized sequentially by the local engine. To materialize multiple feature views in parallel, use a job orchestrator (Airflow, Kubernetes Jobs) and materialize one feature view per job:
# Airflow / cron: one task per feature view
feast materialize-incremental $(date -u +"%Y-%m-%dT%H:%M:%S") \
--views driver_stats
Alternatively, use the Spark or Ray compute engines which distribute the work across a cluster.
Redis: combine with skip_dedup for bulk reloads
When performing a full historical reload into Redis (not an incremental update), combine online_write_batch_size with skip_dedup for maximum throughput:
skip_dedup: true eliminates the timestamp-read pipeline before each write (see Redis tuning), while online_write_batch_size prevents the write worker from converting the entire dataset into memory at once.
Reset skip_dedup to false (or remove it) after the bulk reload. Under normal incremental materialization, deduplication prevents older feature values from overwriting newer ones.
Further reading
Scaling Feast — Horizontal scaling, HPA, KEDA, and HA in detail
— Store-specific configuration and performance tuning
— Connection pooling and SSL configuration
— Cluster mode, Sentinel, TTL configuration, and batched reads
— Transformation modes and write-time transforms
— Full configuration reference including materialization options
Overview
Functionality
In Feast, each batch data source is associated with corresponding offline stores. For example, a SnowflakeSource can only be processed by the Snowflake offline store, while a FileSource can be processed by both File and DuckDB offline stores. Otherwise, the primary difference between batch data sources is the set of supported types. Feast has an internal type system that supports primitive types (bytes, string, int32, int64, float32, float64, bool, timestamp), array types, set types, map/JSON types, and struct types. However, not every batch data source supports all of these types.
For more details on the Feast type system, see here.
Functionality Matrix
There are currently four core batch data source implementations: FileSource, BigQuerySource, SnowflakeSource, and RedshiftSource. There are several additional implementations contributed by the Feast community (PostgreSQLSource, SparkSource, and TrinoSource), which are not guaranteed to be stable or to match the functionality of the core implementations. Details for each specific data source can be found here.
Below is a matrix indicating which data sources support which types.
File
BigQuery
Snowflake
Redshift
Postgres
Spark
Trino
Couchbase
bytes
yes
* Set types are defined in Feast's proto and Python type system but are not inferred by any backend. They must be explicitly declared in the feature view schema and are best suited for online serving use cases. See Type System for details.
Azure Synapse + Azure SQL (contrib)
Description
MsSQL data sources are Microsoft sql table sources. These can be specified either by a table reference or a SQL query.
Disclaimer
The MsSQL data source does not achieve full test coverage. Please do not assume complete stability.
Pay-per-request cost; batch API limits (100 items)
PostgreSQL
3–10 ms
No (threadpool)
Teams with existing Postgres infra
Connection pooling needed at scale
MongoDB
2–5 ms
Yes
Flexible schema, async-native
Requires index tuning for large datasets
Bigtable
3–8 ms
No (threadpool)
Large-scale GCP workloads
Row-key design affects read performance
Cassandra / ScyllaDB
2–5 ms
No (threadpool)
Multi-region, write-heavy
Tunable consistency; requires DC-aware routing
Remote
Varies
No (threadpool)
Centralized feature server architecture
Adds an HTTP hop; tune connection pool
SQLite
N/A
No
Local development only
No concurrent access; not suitable for production
Yes
Uses motor (async MongoDB driver)
PostgreSQL
Implemented
No
Has online_read_async but does not yet advertise via async_supported; uses sync/threadpool path
Redis
Implemented
Yes
online_read_async and online_write_batch_async both implemented; uses sync/threadpool path for get_online_features (overridden with batched single pipeline)
All others
No
No
Fall back to sync with run_in_threadpool()
10
10
1
20
20
1
N/A
All entity keys and all feature views are batched into a single pipeline; no batch tuning needed
PostgreSQL
N/A (single query)
N/A
Single SELECT ... WHERE key IN (...) query; tune connection pool instead
ODFV is computationally expensive (ML inference, complex aggregations)
Prefer write-time if staleness is acceptable; otherwise use read-time with static artifacts
5–10
~4.5–9x
Online store connection pressure becomes the bottleneck; tune pool sizes
10+
Diminishing returns
Online store or network bandwidth may saturate; monitor store-side metrics
max_pool_connections (HTTP pool)
10
No hard limit, but AWS SDK has per-process pool caps; monitor throttling
Redis
Connection per worker
1
maxclients on the Redis server (default: 10,000)
MongoDB
maxPoolSize (in client_kwargs)
100
Server's net.maxIncomingConnections
Cassandra
Driver manages pool per node
Auto
native_transport_max_threads on each Cassandra node
Remote
connection_pool_size (HTTP pool)
50
The target feature server's worker capacity
Replicas
5
Workers per pod
4
max_conn per worker
140 / (5 × 4) = 7
feast_feature_server_cpu_usage
Per-worker CPU. Sustained > 80% means you need more workers or replicas.
feast_feature_server_memory_usage
Per-worker memory. Growing over time may indicate a memory leak — use maxRequests to recycle.
feast_feature_server_request_total
Request throughput and error rates by endpoint and status.
feast_feature_freshness_seconds
Seconds since last materialization per feature view. Alerts you to stale data.
training_retrieval_job = fs.get_historical_features(
entity_df=entity_df_or_sql_string,
features=fs.get_feature_service("driver_activity_v1"),
)
# Option 1: In memory model training
model = ml.fit(training_retrieval_job.to_df())
# Option 2: Unloading to blob storage. Further post-processing can occur before kicking off distributed training.
training_retrieval_job.to_remote_storage()
# Team repository - safe partial apply
# Only adds/updates the specified FeatureService
# Does NOT delete any other objects
store.apply([my_feature_service], partial=True)
# Platform repository - full sync with deletion capability
# Syncs all objects and can remove objects via objects_to_delete
store.apply(all_objects, objects_to_delete=objects_to_remove, partial=False)
from feast import Entity
# Core entities managed by platform team
user = Entity(
name="user_id",
description="User entity",
join_keys=["user_id"]
)
driver = Entity(
name="driver_id",
description="Driver entity",
join_keys=["driver_id"]
)
from feast import SnowflakeSource
# Core data sources managed by platform team
user_features_source = SnowflakeSource(
database="FEAST_DB",
schema="FEATURES",
table="USER_FEATURES",
timestamp_field="event_timestamp",
)
driver_stats_source = SnowflakeSource(
database="FEAST_DB",
schema="FEATURES",
table="DRIVER_STATS",
timestamp_field="event_timestamp",
)
from datetime import timedelta
from feast import FeatureView, Field
from feast.types import Float32, Int64
from entities import user, driver
from data_sources import user_features_source, driver_stats_source
# Core feature views managed by platform team
user_features = FeatureView(
name="user_features",
entities=[user],
ttl=timedelta(days=1),
schema=[
Field(name="age", dtype=Int64),
Field(name="account_balance", dtype=Float32),
],
source=user_features_source,
)
driver_stats = FeatureView(
name="driver_stats",
entities=[driver],
ttl=timedelta(days=1),
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
],
source=driver_stats_source,
)
from feast import FeatureStore
from entities import user, driver
from data_sources import user_features_source, driver_stats_source
from feature_views import user_features, driver_stats
# Initialize the feature store
store = FeatureStore(repo_path=".")
# Collect all objects managed by platform
all_objects = [
user,
driver,
user_features_source,
driver_stats_source,
user_features,
driver_stats,
]
# Apply with partial=False to maintain full control
# This allows the platform team to manage deletions
store.apply(all_objects, partial=False)
print("Platform objects applied successfully!")
project: my_feast_project # Same project as platform
registry: s3://my-bucket/feast-registry/registry.db # Same registry
provider: aws
online_store:
type: dynamodb
region: us-west-2
offline_store:
type: snowflake
account: my_account
database: FEAST_DB
warehouse: FEAST_WH
from feast import FeatureStore, Field, OnDemandFeatureView, RequestSource
from feast.types import Float32
# Team A's custom on-demand feature view
# Uses platform-managed feature views as inputs
@OnDemandFeatureView(
sources=[
"user_features:age", # References platform feature view
RequestSource(schema=[Field(name="current_year", dtype=Float32)]),
],
schema=[Field(name="years_until_retirement", dtype=Float32)],
)
def user_age_odfv(inputs):
"""Calculate years until retirement."""
return {"years_until_retirement": 65 - inputs["age"]}
from feast import FeatureService, FeatureStore
# Initialize feature store to get references
store = FeatureStore(repo_path=".")
# Team A's feature service for their ML model
user_model_features = FeatureService(
name="user_model_v1",
features=[
"user_features", # Platform feature view
"user_age_odfv", # Team's ODFV
],
)
from feast import FeatureStore
from on_demand_views import user_age_odfv
from feature_services import user_model_features
# Initialize the feature store
store = FeatureStore(repo_path=".")
# Collect team's objects
team_objects = [
user_age_odfv,
user_model_features,
]
# Apply with partial=True (safe for teams)
# Only adds/updates team's objects, does NOT delete anything
store.apply(team_objects, partial=True)
print("Team objects applied successfully!")
# Platform objects: simple names
user_features = FeatureView(name="user_features", ...)
# Team objects: prefixed with team name
team_a_user_service = FeatureService(name="team_a_user_model", ...)
team_b_driver_service = FeatureService(name="team_b_driver_model", ...)
# .github/workflows/apply-platform.yml
name: Apply Platform Features
on:
push:
branches: [main]
paths:
- 'platform_repo/**'
jobs:
apply:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Apply platform features
run: |
cd platform_repo
pip install feast[aws,snowflake]
python apply.py
# .github/workflows/apply-team-a.yml
name: Apply Team A Features
on:
push:
branches: [main]
paths:
- 'team_a_repo/**'
jobs:
apply:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Apply team A features
run: |
cd team_a_repo
pip install feast[aws,snowflake]
python apply.py
def validate_team_objects(team_name, objects):
"""Validate that team only creates allowed object types."""
allowed_types = (FeatureService, OnDemandFeatureView)
for obj in objects:
if not isinstance(obj, allowed_types):
raise ValueError(
f"Team {team_name} cannot create {type(obj).__name__}. "
f"Only FeatureServices and ODFVs are allowed."
)
# Check naming convention
if not obj.name.startswith(f"{team_name}_"):
raise ValueError(
f"Team {team_name} objects must be prefixed with '{team_name}_'. "
f"Got: {obj.name}"
)
# In team apply script
validate_team_objects("team_a", team_objects)
store.apply(team_objects, partial=True)
# platform_repo/apply.py
from feast import Entity, FeatureStore, FeatureView, Field, SnowflakeSource
from feast.types import Float32, Int64
from datetime import timedelta
# Initialize feature store
store = FeatureStore(repo_path=".")
# Define entities
customer = Entity(name="customer_id", join_keys=["customer_id"])
# Define data source
customer_source = SnowflakeSource(
database="FEAST_DB",
schema="RAW",
table="CUSTOMER_FEATURES",
timestamp_field="event_timestamp",
)
# Define feature view
customer_features = FeatureView(
name="customer_features",
entities=[customer],
ttl=timedelta(days=7),
schema=[
Field(name="total_purchases", dtype=Int64),
Field(name="avg_order_value", dtype=Float32),
],
source=customer_source,
)
# Apply all platform objects with full control
platform_objects = [customer, customer_source, customer_features]
store.apply(platform_objects, partial=False)
print("✅ Platform objects applied")
# team_marketing_repo/apply.py
from feast import FeatureStore, FeatureService, Field, OnDemandFeatureView, RequestSource
from feast.types import Float32, Int64
# Initialize feature store (same registry as platform)
store = FeatureStore(repo_path=".")
# Define on-demand feature view
@OnDemandFeatureView(
sources=[
"customer_features:total_purchases",
"customer_features:avg_order_value",
RequestSource(schema=[Field(name="current_cart_value", dtype=Float32)]),
],
schema=[
Field(name="purchase_propensity_score", dtype=Float32),
],
)
def marketing_propensity(inputs):
"""Calculate purchase propensity based on history and current cart."""
purchases = inputs["total_purchases"]
avg_value = inputs["avg_order_value"]
cart_value = inputs["current_cart_value"]
# Simple propensity calculation
propensity = (purchases * 0.3 + (cart_value / max(avg_value, 1)) * 0.7)
return {"purchase_propensity_score": min(propensity, 100.0)}
# Define feature service for team's ML model
marketing_campaign_service = FeatureService(
name="team_marketing_campaign_model",
features=[
"customer_features", # Platform feature view
"marketing_propensity", # Team's ODFV
],
)
# Apply team objects safely with partial=True
team_objects = [marketing_propensity, marketing_campaign_service]
store.apply(team_objects, partial=True)
print("✅ Team marketing objects applied")
# In team's training or inference code
from feast import FeatureStore
store = FeatureStore(repo_path=".")
# Get online features using team's feature service
features = store.get_online_features(
features="team_marketing_campaign_model",
entity_rows=[
{"customer_id": "C123", "current_cart_value": 150.0}
],
).to_dict()
print(features)
# Output includes both platform features and team's ODFV:
# {
# "customer_id": ["C123"],
# "total_purchases": [42],
# "avg_order_value": [125.50],
# "purchase_propensity_score": [42.3]
# }
Blog
Welcome to the Feast blog! Here you'll find articles about feature store development, new features, and community updates.
Feast supports registering streaming feature views and Kafka and Kinesis streaming sources. It also provides an interface for stream processing called the Stream Processor. An example Kafka/Spark StreamProcessor is implemented in the contrib folder. For more details, please see the RFC for more details.
Please see here for a tutorial on how to build a versioned streaming pipeline that registers your transformations, features, and data sources in Feast.
One can use existing plugins (offline store, online store, batch materialization engine, providers) and configure those using the built in options. See reference documentation for details.
The other way to customize Feast is to build your own custom components, and then point Feast to delegate to them.
Below are some guides on how to add new custom components:
File data sources are files on disk or on S3. Currently only Parquet and Delta formats are supported.
Example
from feast import FileSource
from feast.data_format import ParquetFormat
parquet_file_source = FileSource(
file_format=ParquetFormat(),
path="file:///feast/customer.parquet",
)
The full set of configuration options is available .
Supported Types
File data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see .
Project
Projects provide complete isolation of feature stores at the infrastructure level. This is accomplished through resource namespacing, e.g., prefixing table names with the associated project. Each project should be considered a completely separate universe of entities and features. It is not possible to retrieve features from multiple projects in a single request. We recommend having a single feature store and a single project per environment (dev, staging, prod).
Users define one or more within a project. Each feature view contains one or more . These features typically relate to one or more . A feature view must always have a , which in turn is used during the generation of training and when materializing feature values into the online store.
The concept of a "project" provide the following benefits:
Provider
A provider is an implementation of a feature store using specific feature store components (e.g. offline store, online store) targeting a specific environment (e.g. GCP stack).
Providers orchestrate various components (offline store, online store, infrastructure, compute) inside an environment. For example, the gcp provider supports as an offline store and as an online store, ensuring that these components can work together seamlessly. Feast has three built-in providers (local, gcp, and aws) with default configurations that make it easy for users to start a feature store in a specific environment. These default configurations can be overridden easily. For instance, you can use the gcp provider but use Redis as the online store instead of Datastore.
Driver ranking
Making a prediction using a linear regression model is a common use case in ML. This model predicts if a driver will complete a trip based on features ingested into Feast.
In this example, you'll learn how to use some of the key functionality in Feast. The tutorial runs in both local mode and on the Google Cloud Platform (GCP). For GCP, you must have access to a GCP project already, including read and write permissions to BigQuery.
This tutorial guides you on how to use Feast with . You will learn how to:
Fraud detection on GCP
A common use case in machine learning, this tutorial is an end-to-end, production-ready fraud prediction system. It predicts in real-time whether a transaction made by a user is fraudulent.
Throughout this tutorial, we’ll walk through the creation of a production-ready fraud prediction system. A prediction is made in real-time as the user makes the transaction, so we need to be able to generate a prediction at low latency.
Our end-to-end example will perform the following workflows:
MCP - AI Agent Example
This example demonstrates how to enable MCP (Model Context Protocol) support in Feast, allowing AI agents and applications to interact with your features through standardized MCP interfaces.
Prerequisites
Python 3.8+
Data sources
Please see for a conceptual explanation of data sources.
Snowflake
Description
Snowflake data sources are Snowflake tables or views. These can be specified either by a table reference or a SQL query.
Examples
BigQuery
Description
BigQuery data sources are BigQuery tables or views. These can be specified either by a table reference or a SQL query. However, no performance guarantees can be provided for SQL query-based sources, so table references are recommended.
Redshift
Description
Redshift data sources are Redshift tables or views. These can be specified either by a table reference or a SQL query. However, no performance guarantees can be provided for SQL query-based sources, so table references are recommended.
PostgreSQL (contrib)
Description
PostgreSQL data sources are PostgreSQL tables or views. These can be specified either by a table reference or a SQL query.
Logical Grouping: Projects group related features together, making it easier to manage and track them.
Feature Definitions: Within a project, you can define features, including their metadata, types, and sources. This helps standardize how features are created and consumed.
Isolation: Projects provide a way to isolate different environments, such as development, testing, and production, ensuring that changes in one project do not affect others.
Collaboration: By organizing features within projects, teams can collaborate more effectively, with clear boundaries around the features they are responsible for.
Access Control: Projects can implement permissions, allowing different users or teams to access only the features relevant to their work.
Alternatively, you can install the dependencies separately:
Setup
Navigate to this example directory within your cloned Feast repository:
Initialize a Feast repository in this directory. We'll use the existing feature_store.yaml that's already configured for MCP:
This will create a data subdirectory and a feature_repo subdirectory if they don't exist, and will use the feature_store.yaml present in the current directory (examples/mcp_feature_store).
Apply the feature store configuration:
Starting the MCP-Enabled Feature Server
Start the Feast feature server with MCP support:
If MCP is properly configured, you should see a log message indicating that MCP support has been enabled:
Available MCP Tools
The fastapi_mcp integration automatically exposes your Feast feature server's FastAPI endpoints as MCP tools. This means AI assistants can:
Call /get-online-features to retrieve features from the feature store
Use /health to check server status
Configuration Details
The key configuration that enables MCP support:
Using a table reference:
Using a query:
Be careful about how Snowflake handles table and column name conventions. In particular, you can read more about quote identifiers here.
The full set of configuration options is available here.
Supported Types
Snowflake data sources support all eight primitive types. Array types are also supported but not with type inference. For a comparison against other batch data sources, please see here.
Examples
Using a table reference:
Using a query:
The full set of configuration options is available here.
Supported Types
BigQuery data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see here.
from feast import BigQuerySource
my_bigquery_source = BigQuerySource(
table_ref="gcp_project:bq_dataset.bq_table",
)
Examples
Using a table name:
Using a query:
The full set of configuration options is available here.
Supported Types
Redshift data sources support all eight primitive types, but currently do not support array types. For a comparison against other batch data sources, please see here.
from feast import RedshiftSource
my_redshift_source = RedshiftSource(
table="redshift_table",
)
The PostgreSQL data source does not achieve full test coverage. Please do not assume complete stability.
Examples
Defining a Postgres source:
The full set of configuration options is available here.
Supported Types
PostgreSQL data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see here.
pip install feast[mcp]
pip install feast
pip install fastapi_mcp
cd examples/mcp_feature_store
feast init .
cd feature_repo
feast apply
cd .. # Go back to examples/mcp_feature_store for the next steps
feast serve --host 0.0.0.0 --port 6566
INFO:feast.feature_server:MCP support has been enabled for the Feast feature server
feature_server:
type: mcp # Use MCP feature server type
enabled: true # Enable feature server
mcp_enabled: true # Enable MCP protocol support
mcp_server_name: "feast-feature-store"
mcp_server_version: "1.0.0"
Why should you use Python to Serve features for Machine Learning?
Python has emerged as the primary language for machine learning, and this extends to feature serving and there are five main reasons Feast recommends using a microservice written in Python.
1. Python is the language of Machine Learning
You should meet your users where they are. Python’s popularity in the machine learning community is undeniable. Its simplicity and readability make it an ideal language for writing and understanding complex algorithms. Python boasts a rich ecosystem of libraries such as TensorFlow, PyTorch, XGBoost, and scikit-learn, which provide robust support for developing and deploying machine learning models and we want Feast in this ecosystem.
2. Precomputation is The Way
Precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable. Precomputation ensures product experiences for downstream services are also fast. Slow user experiences are bad user experiences. Precompute and persist data as much as you can.
3. Serving features in another language can lead to skew
Ensuring that features used during model training (offline serving) and online serving are available in production to make real-time predictions is critical. When features are initially developed, they are typically written in Python. This is due to the convenience and efficiency provided by Python's data manipulation libraries. However, in a production environment, there is often interest or pressure to rewrite these features in a different language, like Java, Go, or C++, for performance reasons. This reimplementation introduces a significant risk: training and serving skew. Note that there will always be some minor exceptions (e.g., any Time Since Last Event types of features) but this should not be the rule.
Training and serving skew occurs when there are discrepancies between the features used during model training and those used during prediction. This can lead to degraded model performance, unreliable predictions, and reduced velocity in releasing new features and new models. The process of rewriting features in another language is prone to errors and inconsistencies, which exacerbate this issue.
4. Reimplementation is Excessive
Rewriting features in another language is not only risky but also resource-intensive. It requires significant time and effort from engineers to ensure that the features are correctly translated. This process can introduce bugs and inconsistencies, further increasing the risk of training and serving skew. Additionally, maintaining two versions of the same feature codebase adds unnecessary complexity and overhead. More importantly, the opportunity cost of this work is high and requires twice the amount of resourcing. Reimplementing code should only be done when the performance gains are worth the investment. Features should largely be precomputed so the latency performance gains should not be the highest impact work that your team can accomplish.
5. Use existing Python Optimizations
Rather than switching languages, it is more efficient to optimize the performance of your feature store while keeping Python as the primary language. Optimization is a two step process.
Step 1: Quantify latency bottlenecks in your feature calculations
Use tools like to understand latency bottlenecks in your code. This will help you prioritize the biggest inefficiencies first. When we initially launched Python native transformations in Python, helped us identify that Pandas resulted in a 10x overhead due to type conversion.
Step 2: Optimize your feature calculations
As mentioned, precomputation is the recommended path. In some cases, you may want fully synchronous writes from your data producer to your online feature store, in which case you will want your feature computations and writes to be very fast. In this case, we recommend optimizing the feature calculation code first.
You should optimize your code using libraries, tools, and caching. For example, identify whether your feature calculations can be optimized through vectorized calculations in NumPy; explore tools like Numba for faster execution; and cache frequently accessed data using tools like an lru_cache.
Lastly, Feast will continue to optimize serving in Python and making the overall infrastructure more performant. This will better serve the community.
So we recommend focusing on optimizing your feature-specific code, reporting latency bottlenecks to the maintainers, and contributing to help the infrastructure be more performant.
By keeping features in Python and optimizing performance, you can ensure consistency between training and serving, reduce the risk of errors, and focus on launching more product experiences for your customers.
Embrace Python for feature serving, and leverage its strengths to build robust and reliable machine learning systems.
Overview
Feast project structure
The top-level namespace within Feast is a project. Users define one or more feature views within a project. Each feature view contains one or more features. These features typically relate to one or more entities. A feature view must always have a data source, which in turn is used during the generation of training datasets and when materializing feature values into the online store. You can read more about Feast projects in the project page.
Data ingestion
For offline use cases that only rely on batch data, Feast does not need to ingest data and can query your existing data (leveraging a compute engine, whether it be a data warehouse or (experimental) Spark / Trino). Feast can help manage pushing streaming features to a batch source to make features available for training.
For online use cases, Feast supports ingesting features from batch sources to make them available online (through a process called materialization), and pushing streaming features to make them available both offline / online. We explore this more in the next concept page ()
Feature registration and retrieval
Features are registered as code in a version controlled repository, and tie to data sources + model versions via the concepts of entities, feature views, and feature services. We explore these concepts more in the upcoming concept pages. These features are then stored in a registry, which can be accessed across users and services. The features can then be retrieved via SDK API methods or via a deployed feature server which exposes endpoints to query for online features (to power real time models).
Feast supports several patterns of feature retrieval.
Use case
Example
API
Entity
An entity is a collection of semantically related features. Users define entities to map to the domain of their use case. For example, a ride-hailing service could have customers and drivers as their entities, which group related features that correspond to these customers and drivers.
The entity name is used to uniquely identify the entity (for example to show in the experimental Web UI). The join key is used to identify the physical primary key on which feature values should be joined together to be retrieved during feature retrieval.
Entities are used by Feast in many contexts, as we explore below:
Use case #1: Defining and storing features
Feast's primary object for defining features is a feature view, which is a collection of features. Feature views map to 0 or more entities, since a feature can be associated with:
zero entities (e.g. a global feature like num_daily_global_transactions)
one entity (e.g. a user feature like user_age or last_5_bought_items)
multiple entities, aka a composite key (e.g. a user + merchant category feature like num_user_purchases_in_merchant_category)
Feast refers to this collection of entities for a feature view as an entity key.
Entities should be reused across feature views. This helps with discovery of features, since it enables data scientists understand how other teams build features for the entity they are most interested in.
Feast will use the feature view concept to then define the schema of groups of features in a low-latency online store.
Use case #2: Retrieving features
At training time, users control what entities they want to look up, for example corresponding to train / test / validation splits. A user specifies a list of entity keys + timestamps they want to fetch correct features for to generate a training dataset.
At serving time, users specify entity key(s) to fetch the latest feature values which can power real-time model prediction (e.g. a fraud detection model that needs to fetch the latest transaction user's features to make a prediction).
Q: Can I retrieve features for all entities?
Kind of.
In practice, this is most relevant for batch scoring models (e.g. predict user churn for all existing users) that are offline only. For these use cases, Feast supports generating features for a SQL-backed list of entities. There is an that welcomes contribution to make this a more intuitive API.
For real-time feature retrieval, there is no out of the box support for this because it would promote expensive and slow scan operations which can affect the performance of other operations on your data sources. Users can still pass in a large list of entities for retrieval, but this does not scale well.
Point-in-time joins
Feature values in Feast are modeled as time-series records. Below is an example of a driver feature view with two feature columns (trips_today, and earnings_today):
The above table can be registered with Feast through the following feature view:
Feast is able to join features from one or more feature views onto an entity dataframe in a point-in-time correct way. This means Feast is able to reproduce the state of features at a specific point in the past.
Given the following entity dataframe, imagine a user would like to join the above driver_hourly_stats feature view onto it, while preserving the trip_success column:
The timestamps within the entity dataframe above are the events at which we want to reproduce the state of the world (i.e., what the feature values were at those specific points in time). In order to do a point-in-time join, a user would load the entity dataframe and run historical retrieval:
For each row within the entity dataframe, Feast will query and join the selected features from the appropriate feature view data source. Feast will scan backward in time from the entity dataframe timestamp up to a maximum of the TTL time specified.
Please note that the TTL time is relative to each timestamp within the entity dataframe. TTL is not relative to the current point in time (when you run the query).
Below is the resulting joined training dataframe. It contains both the original entity rows and joined feature values:
Three feature rows were successfully joined to the entity dataframe rows. The first row in the entity dataframe was older than the earliest feature rows in the feature view and could not be joined. The last row in the entity dataframe was outside of the TTL window (the event happened 11 hours after the feature row) and also couldn't be joined.
Feature server
The Feature Server is a core architectural component in Feast, designed to provide low-latency feature retrieval and updates for machine learning applications.
It is a REST API server built using FastAPI and exposes a limited set of endpoints to serve features, push data, and support materialization operations. The server is scalable, flexible, and designed to work seamlessly with various deployment environments, including local setups and cloud-based systems.
Motivation
In machine learning workflows, real-time access to feature values is critical for enabling low-latency predictions. The Feature Server simplifies this requirement by:
Serving Features: Allowing clients to retrieve feature values for specific entities in real-time, reducing the complexity of direct interactions with the online store.
Data Integration: Providing endpoints to push feature data directly into the online or offline store, ensuring data freshness and consistency.
Scalability: Supporting horizontal scaling to handle high request volumes efficiently.
Standardized API: Exposing HTTP/JSON endpoints that integrate seamlessly with various programming languages and ML pipelines.
Secure Communication: Supporting TLS (SSL) for secure data transmission in production environments.
Architecture
The Feature Server operates as a stateless service backed by two key components:
: The primary data store used for low-latency feature retrieval.
: The metadata store that defines feature sets, feature views, and their relationships to entities.
Key Features
RESTful API: Provides standardized endpoints for feature retrieval and data pushing.
CLI Integration: Easily managed through the Feast CLI with commands like feast serve.
Flexible Deployment: Can be deployed locally, via Docker, or on Kubernetes using Helm charts.
Endpoints Overview
Endpoint
Description
Create a feature repository
A feature repository is a directory that contains the configuration of the feature store and individual features. This configuration is written as code (Python/YAML) and it's highly recommended that teams track it centrally using git. See Feature Repository for a detailed explanation of feature repositories.
The easiest way to create a new feature repository to use feast init command:
feast init
Creating a new Feast repository in /<...>/tiny_pika.
feastinit-tsnowflakeSnowflakeDeployment
feast init -t gcp
Creating a new Feast repository in /<...>/tiny_pika.
feast init -t aws
AWS Region (e.g. us-west-2): ...
Redshift Cluster ID: ...
Redshift Database Name: ...
Redshift User Name: ...
Redshift S3 Staging Location (s3://*): ...
Redshift IAM Role for S3 (arn:aws:iam::*:role/*): ...
Should I upload example data to Redshift (overwriting 'feast_driver_hourly_stats' table)? (Y/n):
Creating a new Feast repository in /<...>/tiny_pika.
The init command creates a Python file with feature definitions, sample data, and a Feast configuration file for local development:
Enter the directory:
You can now use this feature repository for development. You can try the following:
Run feast apply to apply these definitions to Feast.
Edit the example feature definitions in example.py and run feast apply again to change feature definitions.
Read features from the online store
The Feast Python SDK allows users to retrieve feature values from an online store. This API is used to look up feature values at low latency during model serving in order to make online predictions.
Online stores only maintain the current state of features, i.e latest feature values. No historical data is stored or served.
Retrieving online features
1. Ensure that feature values have been loaded into the online store
Please ensure that you have materialized (loaded) your feature values into the online store before starting
2. Define feature references
Create a list of features that you would like to retrieve. This list typically comes from the model training step and should accompany the model binary.
3. Read online features
Next, we will create a feature store object and call get_online_features() which reads the relevant feature values directly from the online store.
Trino (contrib)
Description
Trino data sources are Trino tables or views. These can be specified either by a table reference or a SQL query.
Disclaimer
The Trino data source does not achieve full test coverage. Please do not assume complete stability.
Examples
Defining a Trino source:
The full set of configuration options is available .
Supported Types
Trino data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see .
Third party integrations
We integrate with a wide set of tools and technologies so you can make Feast work in your existing stack. Many of these integrations are maintained as plugins to the main Feast repo.
Don't see your offline store or online store of choice here? Check out our guides to make a custom one!
In order for a plugin integration to be highlighted, it must meet the following requirements:
The plugin must have tests. Ideally it would use the Feast universal tests (see this guide for an example), but custom tests are fine.
The plugin must have some basic documentation on how it should be used.
The author must work with a maintainer to pass a basic code review (e.g. to ensure that the implementation roughly matches the core Feast implementations).
In order for a plugin integration to be merged into the main Feast repo, it must meet the following requirements:
The PR must pass all integration tests. The universal tests (tests specifically designed for custom integrations) must be updated to test the integration.
There is documentation and a tutorial on how to use the integration.
The author (or someone else) agrees to take ownership of all the files, and maintain those files going forward.
If the plugin is being contributed by an organization, and not an individual, the organization should provide the infrastructure (or credits) for integration tests.
# Read in entity dataframe
entity_df = pd.read_csv("entity_df.csv")
training_df = store.get_historical_features(
entity_df=entity_df,
features = [
'driver_hourly_stats:trips_today',
'driver_hourly_stats:earnings_today'
],
)
$ tree
.
└── tiny_pika
├── data
│ └── driver_stats.parquet
├── example.py
└── feature_store.yaml
1 directory, 3 files
# Replace "tiny_pika" with your auto-generated dir name
cd tiny_pika
features = [
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
# Optionally, reference a specific version (requires enable_online_feature_view_versioning):
# "driver_hourly_stats@v2:conv_rate"
]
A feature transformation is a function that takes some set of input data and returns some set of output data. Feature transformations can happen on either raw data or derived data.
Feature Transformation Engines
Feature transformations can be executed by three types of "transformation engines":
The Feast Feature Server
An Offline Store (e.g., Snowflake, BigQuery, DuckDB, Spark, etc.)
The three transformation engines are coupled with the .
Importantly, this implies that different feature transformation code may be used under different transformation engines, so understanding the tradeoffs of when to use which transformation engine/communication pattern is extremely critical to the success of your implementation.
In general, we recommend transformation engines and network calls to be chosen by aligning it with what is most appropriate for the data producer, feature/model usage, and overall product.
API
feature_transformation
feature_transformation or udf are the core APIs for defining feature transformations in Feast. They allow you to specify custom logic that can be applied to the data during materialization or retrieval. Examples include:
OR
OR
Aggregation
Aggregation is builtin API for defining batch or streamable aggregations on data. It allows you to specify how to aggregate data over a time window, such as calculating the average or sum of a feature over a specified period. Examples include:
Filter
ttl: They amount of time that the features will be available for materialization or retrieval. The entity rows' timestamp higher that the current time minus the ttl will be used to filter the features. This is useful for ensuring that only recent data is used in feature calculations. Examples include:
Join
Feast can join multiple feature views together to create a composite feature view. This allows you to combine features from different sources or views into a single view. Examples include:
The underlying implementation of the join is an inner join by default, and join key is the entity id.
Push
Description
Push sources allow feature values to be pushed to the online store and offline store in real time. This allows fresh feature values to be made available to applications. Push sources supercede the FeatureStore.write_to_online_store.
Push sources can be used by multiple feature views. When data is pushed to a push source, Feast propagates the feature values to all the consuming feature views.
Push sources can optionally have a batch_source specified. If provided, it enables retrieval of historical features and supports materialization from the offline store to the online store. However, if your features are generated post-training or are only needed online (e.g., embeddings), you can omit the batch_source.
When a batch_source is used, users are responsible for ensuring that data is also pushed to a batch data source, such as a data warehouse. Note that when a push source is used as a stream source in a feature view definition, a batch_source does not need to be explicitly specified in the feature view itself.
Stream sources
Streaming data sources are important sources of feature values. A typical setup with streaming data looks like:
Raw events come in (stream 1)
Streaming transformations applied (e.g. generating features like last_N_purchased_categories) (stream 2)
Write stream 2 values to an offline store as a historical log for training (optional)
Feast allows users to push features previously registered in a feature view to the online store for fresher features. It also allows users to push batches of stream data to the offline store by specifying that the push be directed to the offline store. This will push the data to the offline store declared in the repository configuration used to initialize the feature store.
Example (basic)
Defining a push source
Note that the push schema needs to also include the entity.
Pushing data
Note that the to parameter is optional and defaults to online but we can specify these options: PushMode.ONLINE, PushMode.OFFLINE, or PushMode.ONLINE_AND_OFFLINE.
See also for instructions on how to push data to a deployed feature server.
Example (Spark Streaming)
The default option to write features from a stream is to add the Python SDK into your existing PySpark pipeline.
This can also be used under the hood by a contrib stream processor (see )
Write Patterns
Feast uses a to push features to the online store.
This has two important consequences: (1) communication patterns between the Data Producer (i.e., the client) and Feast (i.e,. the server) and (2) feature computation and feature value write patterns to Feast's online store.
Data Producers (i.e., services that generate data) send data to Feast so that Feast can write feature values to the online store. That data can be either raw data where Feast computes and stores the feature values or precomputed feature values.
Communication Patterns
Data ingestion
Data source
A data source in Feast refers to raw underlying data that users own (e.g. in a table in BigQuery). Feast does not manage any of the raw underlying data but instead, is in charge of loading this data and performing different operations on the data to retrieve or serve features.
Feast uses a time-series data model to represent data. This data model is used to interpret feature data in data sources in order to build training datasets or materialize features into an online store.
Below is an example data source with a single entity column (driver
Permission
Overview
The Feast permissions model allows to configure granular permission policies to all the resources defined in a feature store.
The configured permissions are stored in the Feast registry and accessible through the CLI and the registry APIs.
The permission authorization enforcement is performed when requests are executed through one of the Feast (Python) servers
Driver stats on Snowflake
Initial demonstration of Snowflake as an offline+online store with Feast, using the Snowflake demo template.
In the steps below, we will set up a sample Feast project that leverages Snowflake as an offline store + materialization engine + online store.
Starting with data in a Snowflake table, we will register that table to the feature store and define features associated with the columns in that table. From there, we will generate historical training data based on those feature definitions and then materialize the latest feature values into the online store. Lastly, we will retrieve the materialized feature values.
Our template will generate new data containing driver statistics. From there, we will show you code snippets that will call to the offline store for generating training datasets, and then the code for calling the online store to serve you the latest feature values to serve models in production.
Structuring Feature Repos
A common scenario when using Feast in production is to want to test changes to Feast object definitions. For this, we recommend setting up a staging environment for your offline and online stores, which mirrors production (with potentially a smaller data set). Having this separate environment allows users to test changes by first applying them to staging, and then promoting the changes to production after verifying the changes on staging.
Setting up multiple environments
There are three common ways teams approach having separate environments
Operator Configuration Guides
These guides cover the FeatureStore Custom Resource (CR) from an operator perspective — what to put in the CR, how the operator translates it into Kubernetes objects, and where to look for store-specific YAML options in the Feast SDK docs.
Write stream 2 values to an online store for low latency feature serving
Periodically materialize feature values from the offline store into the online store for decreased training-serving skew and improved model performance
The following files will automatically be created in your project folder:
feature_store.yaml -- This is your main configuration file
driver_repo.py -- This is your main feature definition file
test.py -- This is a file to test your feature store configuration
Inspect feature_store.yaml
Here you will see the information that you entered. This template will use Snowflake as the offline store, materialization engine, and the online store. The main thing to remember is by default, Snowflake objects have ALL CAPS names unless lower case was specified.
Run our test python script test.py
What we did in test.py
Initialize our Feature Store
Create a dummy training dataframe, then call our offline store to add additional columns
Materialize the latest feature values into our online store
Retrieve the latest values from our online store based on our entity key
Have separate git branches for each environment
Have separate feature_store.yaml files and separate Feast object definitions that correspond to each environment
Have separate feature_store.yaml files per environment, but share the Feast object definitions
Different version control branches
To keep a clear separation of the feature repos, teams may choose to have multiple long-lived branches in their version control system, one for each environment. In this approach, with CI/CD setup, changes would first be made to the staging branch, and then copied over manually to the production branch once verified in the staging environment.
Separate feature_store.yaml files and separate Feast object definitions
For this approach, we have created an example repository (Feast Repository Example) which contains two Feast projects, one per environment.
The contents of this repository are shown below:
The repository contains three sub-folders:
staging/: This folder contains the staging feature_store.yaml and Feast objects. Users that want to make changes to the Feast deployment in the staging environment will commit changes to this directory.
production/: This folder contains the production feature_store.yaml and Feast objects. Typically users would first test changes in staging before copying the feature definitions into the production folder, before committing the changes.
.github: This folder is an example of a CI system that applies the changes in either the staging or production repositories using feast apply. This operation saves your feature definitions to a shared registry (for example, on GCS) and configures your infrastructure for serving features.
The feature_store.yaml contains the following:
Notice how the registry has been configured to use a Google Cloud Storage bucket. All changes made to infrastructure using feast apply are tracked in the registry.db. This registry will be accessed later by the Feast SDK in your training pipelines or model serving services in order to read features.
It is important to note that the CI system above must have access to create, modify, or remove infrastructure in your production environment. This is unlike clients of the feature store, who will only have read access.
If your organization consists of many independent data science teams or a single group is working on several projects that could benefit from sharing features, entities, sources, and transformations, then we encourage you to utilize Python packages inside each environment:
Shared Feast Object definitions with separate feature_store.yaml files
This approach is very similar to the previous approach, but instead of having feast objects duplicated and having to copy over changes, it may be possible to share the same Feast object definitions and have different feature_store.yaml configuration.
An example of how such a repository would be structured is as follows:
Users can then apply the applying them to each environment in this way:
This setup has the advantage that you can share the feature definitions entirely, which may prevent issues with copy-pasting code.
Summary
In summary, once you have set up a Git based repository with CI that runs feast apply on changes, your infrastructure (offline store, online store, and cloud environment) will automatically be updated to support the loading of data into the feature store or retrieval of data.
feast init -t snowflake {feature_repo_name}
Snowflake Deployment URL (exclude .snowflakecomputing.com):
Snowflake User Name::
Snowflake Password::
Snowflake Role Name (Case Sensitive)::
Snowflake Warehouse Name (Case Sensitive)::
Snowflake Database Name (Case Sensitive)::
Should I upload example data to Snowflake (overwrite table)? [Y/n]: Y
cd {feature_repo_name}
Using an asynchronous API call for a small number of entities or a single entity (e.g., using the ) or the Feature Server's )
Using a "batch job" for a large number of entities (e.g., using a )
Note, in some contexts, developers may "batch" a group of entities together and write them to the online store in a single API call. This is a common pattern when writing data to the online store to reduce write loads but we would not qualify this as a batch job.
Feature Value Write Patterns
Writing feature values to the online store (i.e., the server) can be done in two ways: Precomputing the transformations client-side or Computing the transformations On Demand server-side.
Combining Approaches
In some scenarios, a combination of Precomputed and On Demand transformations may be optimal.
When selecting feature value write patterns, one must consider the specific requirements of your application, the acceptable correctness of the data, the latency tolerance, and the computational resources available. Making deliberate choices can help the performance and reliability of your service.
There are two ways the client can write feature values to the online store:
Precomputing transformations
Computing transformations On Demand
Hybrid (Precomputed + On Demand)
1. Precomputing Transformations
Precomputed transformations can happen outside of Feast (e.g., via some batch job or streaming application) or inside of the Feast feature server when writing to the online store via the push or write-to-online-store api.
2. Computing Transformations On Demand
On Demand transformations can only happen inside of Feast at either (1) the time of the client's request or (2) when the data producer writes to the online store. With the transform_on_write parameter, you can control whether transformations are applied during write operations, allowing you to skip transformations for pre-processed data while still enabling transformations during API calls.
3. Hybrid (Precomputed + On Demand)
The hybrid approach allows for precomputed transformations to happen inside or outside of Feast and have the On Demand transformations happen at client request time. This is particularly convenient for "Time Since Last" types of features (e.g., time since purchase).
Tradeoffs
When deciding between synchronous and asynchronous data writes, several tradeoffs should be considered:
Data Consistency: Asynchronous writes allow Data Producers to send data without waiting for the write operation to complete, which can lead to situations where the data in the online store is stale. This might be acceptable in scenarios where absolute freshness is not critical. However, for critical operations, such as calculating loan amounts in financial applications, stale data can lead to incorrect decisions, making synchronous writes essential.
Correctness: The risk of data being out-of-date must be weighed against the operational requirements. For instance, in a lending application, having up-to-date feature data can be crucial for correctness (depending upon the features and raw data), thus favoring synchronous writes. In less sensitive contexts, the eventual consistency offered by asynchronous writes might be sufficient.
Service Coupling: Synchronous writes result in tighter coupling between services. If a write operation fails, it can cause the dependent service operation to fail as well, which might be a significant drawback in systems requiring high reliability and independence between services.
Application Latency: Asynchronous writes typically reduce the perceived latency from the client's perspective because the client does not wait for the write operation to complete. This can enhance the user experience and efficiency in environments where operations are not critically dependent on immediate data freshness.
The table below can help guide the most appropriate data write and feature computation strategies based on specific application needs and data sensitivity.
Data Write Type
Feature Computation
Scenario
Recommended Approach
Asynchronous
On Demand
Data-intensive applications tolerant to staleness
Opt for asynchronous writes with on-demand computation to balance load and manage resource usage efficiently.
Feast supports primarily time-stamped tabular data as data sources. There are many kinds of possible data sources:
Batch data sources: ideally, these live in data warehouses (BigQuery, Snowflake, Redshift), but can be in data lakes (S3, GCS, etc). Feast supports ingesting and querying data across both.
Stream data sources: Feast does not have native streaming integrations. It does however facilitate making streaming features available in different environments. There are two kinds of sources:
Push sources allow users to push features into Feast, and make it available for training / batch scoring ("offline"), for realtime feature serving ("online") or both.
[Alpha] Stream sources allow users to register metadata from Kafka or Kinesis sources. The onus is on the user to ingest from these sources, though Feast provides some limited helper methods to ingest directly from Kafka / Kinesis topics.
(Experimental) Request data sources: This is data that is only available at request time (e.g. from a user action that needs an immediate model prediction response). This is primarily relevant as an input into , which allow light-weight feature engineering and combining features across sources.
Batch data ingestion
Ingesting from batch sources is only necessary to power real-time models. This is done through materialization. Under the hood, Feast manages an offline store (to scalably generate training data from batch sources) and an online store (to provide low-latency access to features for real-time models).
A key command to use in Feast is the materialize_incremental command, which fetches the latest values for all entities in the batch source and ingests these values into the online store.
When working with On Demand Feature Views with write_to_online_store=True, you can also control whether transformations are applied during ingestion by using the transform_on_write parameter. Setting transform_on_write=False allows you to materialize pre-transformed features without reapplying transformations, which is particularly useful for large batch datasets that have already been processed.
Materialization can be called programmatically or through the CLI:
This snippet creates a feature store object which points to the registry (which knows of all defined features) and the online store (DynamoDB in this case), and
If the schema parameter is not specified when defining a data source, Feast attempts to infer the schema of the data source during feast apply. The way it does this depends on the implementation of the offline store. For the offline stores that ship with Feast out of the box this inference is performed by inspecting the schema of the table in the cloud data warehouse, or if a query is provided to the source, by running the query with a LIMIT clause and inspecting the result.
Stream data ingestion
Ingesting from stream sources happens either via a Push API or via a contrib processor that leverages an existing Spark context.
To push data into the offline or online stores: see push sources for details.
Note that there is no permission enforcement when accessing the Feast API with a local provider.
Concepts
The permission model is based on the following components:
A resource is a Feast object that we want to secure against unauthorized access.
We assume that the resource has a name attribute and optional dictionary of associated key-value tags.
An action is a logical operation executed on the secured resource, like:
create: Create an instance.
describe: Access the instance state.
update: Update the instance state.
A policy identifies the rule for enforcing authorization decisions on secured resources, based on the current user.
A default implementation is provided for role-based policies, using the user roles to grant or deny access to the requested actions on the secured resources.
The Permission class identifies a single permission configured on the feature store and is identified by these attributes:
name: The permission name.
types: The list of protected resource types. Defaults to all managed types, e.g. the ALL_RESOURCE_TYPES alias. All sub-classes are included in the resource match.
name_patterns: A list of regex patterns to match resource names. If any regex matches, the Permission policy is applied. Defaults to [], meaning no name filtering is applied.
required_tags: Dictionary of key-value pairs that must match the resource tags. Defaults to None, meaning that no tags filtering is applied.
actions: The actions authorized by this permission. Defaults to ALL_VALUES, an alias defined in the action module.
policy: The policy to be applied to validate a client request.
To simplify configuration, several constants are defined to streamline the permissions setup:
In module feast.feast_object:
ALL_RESOURCE_TYPES is the list of all the FeastObject types.
ALL_FEATURE_VIEW_TYPES is the list of all the feature view types, including those not inheriting from FeatureView type like OnDemandFeatureView.
In module feast.permissions.action:
ALL_ACTIONS is the list of all managed actions.
READ includes all the read actions for online and offline store.
Given the above definitions, the feature store can be configured with granular control over each resource, enabling partitioned access by teams to meet organizational requirements for service and data sharing, and protection of sensitive information.
The feast CLI includes a new permissions command to list the registered permissions, with options to identify the matching resources for each configured permission and the existing resources that are not covered by any permission.
Note: Feast resources that do not match any of the configured permissions are not secured by any authorization policy, meaning any user can execute any action on such resources.
Definition examples
This permission definition grants access to the resource state and the ability to read all of the stores for any feature view or feature service to all users with the role super-reader:
This example grants permission to write on all the data sources with risk_level tag set to high only to users with role admin or data_team:
Note: When using multiple roles in a role-based policy, the user must be granted at least one of the specified roles.
The following permission grants authorization to read the offline store of all the feature views including risky in the name, to users with role trusted:
Permission granting order
When mixing and matching policies in permissions script, the permission granting order is as follows:
The first matching policy wins in the list of policies and the permission is granted based on the matching policy rules and rest policies are ignored.
If any policy matches from the list of policies, the permission is granted based on the matching policy rules and rest policies are ignored
If no policy matches, the permission is denied
Authorization configuration
In order to leverage the permission functionality, the auth section is needed in the feature_store.yaml configuration. Currently, Feast supports OIDC and Kubernetes RBAC authorization protocols.
The default configuration, if you don't specify the auth configuration section, is no_auth, indicating that no permission enforcement is applied.
The auth section includes a type field specifying the actual authorization protocol, and protocol-specific fields that are specified in Authorization Manager.
Store-specific feature_store.yaml options (online/offline store drivers, registry drivers, etc.)
Sample CRs
Rule of thumb: if you need to know what to put in the CR and why, read the operator guides. If you need to know which keys a particular store driver accepts, read the Feast SDK docs for that store.
Guide index
#
Guide
Topic
1
feastProjectDir: cloning a git repo vs feast init templates
2
File (path + PVC) vs DB store for offline/online/registry; Secret format
Quick-start: which guide do I need?
"How do I point the operator at my existing git feature repo?" → Guide 1
"How do I wire Postgres/Redis/DuckDB as my store?" → Guide 2
"How do I enable Prometheus scraping for the feature server?" → Guide 3
"How do I make all services share a remote registry?" →
"How do I enable Kubernetes RBAC or OIDC auth?" →
"How do I schedule nightly materialization?" →
"How do I send lineage events to Marquez?" →
"What are all valid fields on ServingConfig?" →
Scaling & HA
Horizontal scaling, HPA, PodDisruptionBudget, affinity, and topology spread constraints are covered in the main Feast docs:
Feast (Feature Store) is an open-source feature store that helps teams operate production ML systems at scale by allowing them to define, manage, validate, and serve features for production AI/ML.
Feast's feature store is composed of two foundational components: (1) an offline store for historical feature extraction used in model training and (2) an online store for serving features at low-latency in production systems and applications.
Feast is a configurable operational data system that re-uses existing infrastructure to manage and serve machine learning features to real-time models. For more details, please review our .
Concretely, Feast provides:
A Python SDK for programmatically defining features, entities, sources, and (optionally) transformations
A Python SDK for reading and writing features to configured offline and online data stores
An for reading and writing features (useful for non-python languages)
Feast allows ML platform teams to:
Make features consistently available for training and low-latency serving by managing an offline store (to process historical data for scale-out batch scoring or model training), a low-latency online store (to power real-time prediction), and a battle-tested feature server (to serve pre-computed features online).
Avoid data leakage by generating point-in-time correct feature sets so data scientists can focus on feature engineering rather than debugging error-prone dataset joining logic. This ensures that future feature values do not leak to models during training.
Note: Feast uses a push model for online serving. This means that the feature store pushes feature values to the online store, which reduces the latency of feature retrieval. This is more efficient than a pull model, where the model serving system must make a request to the feature store to retrieve feature values. See for a more detailed discussion.
Who is Feast for?
Feast helps ML platform/MLOps teams with DevOps experience productionize real-time models. Feast also helps these teams build a feature platform that improves collaboration between data engineers, software engineers, machine learning engineers, and data scientists.
For Data Scientists: Feast is a tool where you can easily define, store, and retrieve your features for both model development and model deployment. By using Feast, you can focus on what you do best: build features that power your AI/ML models and maximize the value of your data.
For MLOps Engineers: Feast is a library that allows you to connect your existing infrastructure (e.g., online database, application server, microservice, analytical database, and orchestration tooling) that enables your Data Scientists to ship features for their models to production using a friendly SDK without having to be concerned with software engineering challenges that occur from serving real-time production systems. By using Feast, you can focus on maintaining a resilient system, instead of implementing features for Data Scientists.
What Feast is not?
Feast is not
An / system. Feast is not a general purpose data pipelining system. Users often leverage tools like to manage upstream data transformations. Feast does support some .
A data orchestration tool: Feast does not manage or orchestrate complex workflow DAGs. It relies on upstream data pipelines to produce feature values and integrations with tools like to make features consistently available.
A data warehouse: Feast is not a replacement for your data warehouse or the source of truth for all transformed data in your organization. Rather, Feast is a lightweight downstream layer that can serve data from an existing data warehouse (or other data sources) to models in production.
Feast does not fully solve
reproducible model training / model backtesting / experiment management: Feast captures feature and model metadata, but does not version-control datasets / labels or manage train / test splits. Other tools like , , and are better suited for this.
batch feature engineering: Feast supports on-demand and streaming transformations. Feast is also investing in supporting batch transformations.
native streaming feature integration: Feast enables users to push streaming features, but does not pull from streaming sources or manage streaming pipelines.
Example use cases
Many companies have used Feast to power real-world ML use cases such as:
Personalizing online recommendations by leveraging pre-computed historical user or item features.
Online fraud detection, using features that compare against (pre-computed) historical transaction patterns
Churn prediction (an offline model), generating feature values for all users at a fixed cadence in batch
How can I get started?
The best way to learn Feast is to use it. Head over to our and try it out!
Explore the following resources to get started with Feast:
is the fastest way to get started with Feast
describes all important Feast API concepts
describes Feast's overall architecture.
OpenTelemetry Integration
The OpenTelemetry integration in Feast provides comprehensive monitoring and observability capabilities for your feature serving infrastructure. This component enables you to track key metrics, traces, and logs from your Feast deployment.
Motivation
Monitoring and observability are critical for production machine learning systems. The OpenTelemetry integration addresses these needs by:
Performance Monitoring: Track CPU and memory usage of feature servers
Operational Insights: Collect metrics to understand system behavior and performance
Troubleshooting: Enable effective debugging through distributed tracing
Resource Optimization: Monitor resource utilization to optimize deployments
Production Readiness: Provide enterprise-grade observability capabilities
Architecture
The OpenTelemetry integration in Feast consists of several components working together:
OpenTelemetry Collector: Receives, processes, and exports telemetry data
Prometheus Integration: Enables metrics collection and monitoring
Instrumentation: Automatic Python instrumentation for tracking metrics
Key Features
Automated Instrumentation: Python auto-instrumentation for comprehensive metric collection
To add monitoring to the Feast Feature Server, follow these steps:
1. Deploy Prometheus Operator
Follow the to install the operator.
2. Deploy OpenTelemetry Operator
Before installing the OpenTelemetry Operator:
Install cert-manager
Validate that the pods are running
Apply the OpenTelemetry operator:
For additional installation steps, refer to the .
3. Configure OpenTelemetry Collector
Add the OpenTelemetry Collector configuration under the metrics section in your values.yaml file:
4. Add Instrumentation Configuration
Add the following annotations and environment variables to your deployment.yaml:
5. Add Metric Checks
Add metric checks to all manifests and deployment files:
6. Add Required Manifests
Add the following components to your chart:
Instrumentation
OpenTelemetryCollector
ServiceMonitors
Prometheus Instance
7. Deploy Feast
Deploy Feast with metrics enabled:
Usage
To enable OpenTelemetry monitoring in your Feast deployment:
Set metrics.enabled=true in your Helm values
Configure the OpenTelemetry Collector endpoint
Deploy with proper annotations and environment variables
Example configuration:
Monitoring
Once configured, you can monitor various metrics including:
feast_feature_server_memory_usage: Memory utilization of the feature server
feast_feature_server_cpu_usage: CPU usage statistics
feast_feature_server_request_latency_seconds: Request latency with feature count dimensions
For the full list of metrics, see the .
These metrics can be visualized using Prometheus and other compatible monitoring tools.
4 — Registry Topology
The Feast registry stores metadata about feature views, entities, and feature services. The operator supports three topology options:
Topology
Use case
Local (file or DB)
Self-contained deployment; registry runs alongside other services
Local + server
Local registry exposed as a gRPC / REST server for remote clients
Local registry
File-backed (default)
DB-backed (SQL / Snowflake)
Exposing the registry as a server
Add server: {} under registry.local to expose it on port 6570:
The registry server also supports REST and gRPC independently:
Remote registry
A remote registry lets multiple FeatureStore CRs (e.g. in different namespaces or teams) share a single registry. One CR owns the registry; the others point at it.
Using a hostname
Point at any existing Feast registry server endpoint:
Using feastRef (recommended for operator-managed registries)
feastRef lets one FeatureStore CR reference another CR's registry without hard-coding hostnames. The operator resolves the Service name automatically:
Remote registry with TLS
If the remote registry server uses TLS, provide the CA certificate so clients can verify it:
All-remote topology
For teams that deploy services independently, all services can use remote endpoints:
See also
5 — Security
The operator supports two authorization models via spec.authz, plus TLS for all servers. Authorization is optional — omitting authz deploys Feast with no access control.
Kubernetes RBAC authorization
Kubernetes RBAC authorization uses ServiceAccount tokens. The operator creates ClusterRoles for each named role you declare and binds them to ServiceAccounts. Feast servers enforce these roles on every API call.
The operator creates ClusterRole resources named after each entry in roles. Bind them to subjects using standard Kubernetes ClusterRoleBinding or RoleBinding resources.
Kubernetes auth requires all services to be exposed as servers (the controller rejects partial configurations where some services are local while RBAC is enabled).
SDK docs:
OIDC authorization
OIDC authorization validates Bearer tokens against an OIDC provider (Keycloak, Dex, etc.).
Secret format
Create a Secret with the OIDC client credentials:
Reference the Secret from the CR:
Advanced OIDC options
SDK docs:
TLS for servers
Each server accepts a tls block pointing to a Kubernetes Secret that holds the TLS certificate and key.
Creating a TLS Secret
Applying TLS to servers
Each service can use different TLS Secrets.
Custom certificate key names
By default the operator looks for keys tls.crt and tls.key. Override with:
mTLS — providing a CA certificate
For mutual TLS (client certificate verification), supply a CA cert via a ConfigMap:
OpenShift non-TLS mode
On OpenShift, services are typically accessed via Routes with TLS termination at the edge. In this case it is common to run the Feast servers without internal TLS:
See also
Adding a custom provider
Overview
All Feast operations execute through a provider. Operations like materializing data from the offline to the online store, updating infrastructure like databases, launching streaming ingestion jobs, building training datasets, and reading features from the online store.
Custom providers allow Feast users to extend Feast to execute any custom logic. Examples include:
Launching custom streaming ingestion jobs (Spark, Beam)
Launching custom batch ingestion (materialization) jobs (Spark, Beam)
Adding custom validation to feature repositories during feast apply
Adding custom infrastructure setup logic which runs during feast apply
Extending Feast commands with in-house metrics, logging, or tracing
Feast comes with built-in providers, e.g, LocalProvider, GcpProvider, and AwsProvider. However, users can develop their own providers by creating a class that implements the contract in the .
This guide also comes with a fully functional . Please have a look at the repository for a representative example of what a custom provider looks like, or fork the repository when creating your own provider.
Guide
The fastest way to add custom logic to Feast is to extend an existing provider. The most generic provider is the LocalProvider which contains no cloud-specific logic. The guide that follows will extend the LocalProvider with operations that print text to the console. It is up to you as a developer to add your custom code to the provider methods, but the guide below will provide the necessary scaffolding to get you started.
Step 1: Define a Provider class
The first step is to define a custom provider class. We've created the MyCustomProvider below.
Notice how in the above provider we have only overwritten two of the methods on the LocalProvider, namely update_infra and materialize_single_feature_view. These two methods are convenient to replace if you are planning to launch custom batch or streaming jobs. update_infra can be used for launching idempotent streaming jobs, and materialize_single_feature_view can be used for launching batch ingestion jobs.
It is possible to overwrite all the methods on the provider class. In fact, it isn't even necessary to subclass an existing provider like LocalProvider. The only requirement for the provider class is that it follows the .
Step 2: Configuring Feast to use the provider
Configure your file to point to your new provider class:
Notice how the provider field above points to the module and class where your provider can be found.
Step 3: Using the provider
Now you should be able to use your provider by running a Feast command:
It may also be necessary to add the module root path to your PYTHONPATH as follows:
That's it. You should now have a fully functional custom provider!
Next steps
Have a look at the for a fully functional example of a custom provider. Feel free to fork it when creating your own custom provider!
Starting Feast servers in TLS(SSL) Mode
TLS (Transport Layer Security) and SSL (Secure Sockets Layer) are both protocols encrypts communications between a client and server to provide enhanced security.TLS or SSL words used interchangeably. This article is going to show the sample code to start all the feast servers such as online server, offline server, registry server and UI server in TLS mode. Also show examples related to feast clients to communicate with the feast servers started in TLS mode.
We assume you have basic understanding of feast terminology before going through this tutorial, if you are new to feast then we would recommend to go through existing starter tutorials of feast.
Obtaining a self-signed TLS certificate and key
In development mode we can generate a self-signed certificate for testing. In an actual production environment it is always recommended to get it from a trusted TLS certificate provider.
The above command will generate two files
key.pem : certificate private key
cert.pem: certificate public key
You can use the public or private keys generated from above command in the rest of the sections in this tutorial.
Create the feast demo repo for the rest of the sections.
Create a feast repo and initialize using feast init and feast apply command and use this repo as a demo for subsequent sections.
You need to execute the feast cli commands from feast_repo_ssl_demo/feature_repo directory created from the above feast init command.
Starting feast online server (feature server) in TLS mode
To start the feature server in TLS mode, you need to provide the private and public keys using the --key and --cert arguments with the feast serve command.
You will see the output something similar to as below. Note the server url starts in the https mode.
Feast client connecting to remote online sever started in TLS mode.
Sometimes you may need to pass the self-signed public key to connect to the remote online server started in SSL mode if you have not added the public key to the certificate store.
feast client example: The registry is pointing to registry of remote feature store. If it is not accessible then should be configured to use remote registry.
cert is an optional configuration to the public certificate path when the online server starts in TLS(SSL) mode. Typically, this file ends with *.crt, *.cer, or *.pem.
Starting feast Registry server in TLS mode
To start the feature server in TLS mode, you need to provide the private and public keys using the --key and --cert arguments with the feast serve_registry command.
You will see the output something similar to as below. Note the server url starts in the https mode.
Feast client connecting to remote registry sever started in TLS mode.
Sometimes you may need to pass the self-signed public key to connect to the remote registry server started in SSL mode if you have not added the public key to the certificate store.
feast client example:
cert is an optional configuration to the public certificate path when the registry server starts in TLS(SSL) mode. Typically, this file ends with *.crt, *.cer, or *.pem.
Starting feast offline server in TLS mode
To start the offline server in TLS mode, you need to provide the private and public keys using the --key and --cert arguments with the feast serve_offline command.
You will see the output something similar to as below. Note the server url starts in the https mode.
Feast client connecting to remote offline sever started in TLS mode.
Sometimes you may need to pass the self-signed public key to connect to the remote registry server started in SSL mode if you have not added the public key to the certificate store. You have to add scheme to https.
feast client example:
cert is an optional configuration to the public certificate path when the registry server starts in TLS(SSL) mode. Typically, this file ends with *.crt, *.cer, or *.pem. scheme should be https. By default, it will be http so you have to explicitly configure to https if you are planning to connect to remote offline server which is started in TLS mode.
Starting feast UI server (react app) in TLS mode
To start the feast UI server in TLS mode, you need to provide the private and public keys using the --key and --cert arguments with the feast ui command.
You will see the output something similar to as below. Note the server url starts in the https mode.
Adding public key to CA trust store and configuring the feast to use the trust store.
You can pass the public key for SSL verification using the cert parameter, however, it is sometimes difficult to maintain individual certificates and pass them individually. The alternative recommendation is to add the public certificate to CA trust store and set the path as an environment variable (e.g., FEAST_CA_CERT_FILE_PATH). Feast will use the trust store path in the FEAST_CA_CERT_FILE_PATH environment variable.
Couchbase (contrib)
Description
Couchbase Columnar data sources are Couchbase Capella Columnar collections that can be used as a source for feature data. Note that Couchbase Columnar is available through Couchbase Capella.
Disclaimer
The Couchbase Columnar data source does not achieve full test coverage. Please do not assume complete stability.
Examples
Defining a Couchbase Columnar source:
The full set of configuration options is available .
Supported Types
Couchbase Capella Columnar data sources support BOOLEAN, STRING, BIGINT, and DOUBLE primitive types. For a comparison against other batch data sources, please see .
Authorization Manager
An Authorization Manager is an instance of the AuthManager class that is plugged into one of the Feast servers to extract user details from the current request and inject them into the framework.
Note: Feast does not provide authentication capabilities; it is the client's responsibility to manage the authentication token and pass it to the Feast server, which then validates the token and extracts user details from the configured authentication server.
Two authorization managers are supported out-of-the-box:
1 — Project Provisioning
The operator needs a Feast feature repository (a directory containing feature_store.yaml and Python feature-view definitions) to work from. spec.feastProjectDir controls how that directory is created inside the pods. Exactly one of git or init must be set.
Option A — Clone from a Git repository (
delete: Delete an instance.
read: Read both online and offline stores.
read_online: Read the online store.
read_offline: Read the offline store.
write: Write on any store.
write_online: Write to the online store.
write_offline: Write to the offline store.
WRITE includes all the write actions for online and offline store.
CRUD includes all the state management actions to create, describe, update or delete a Feast resource.
Precomputed
High volume, non-critical data processing
Use asynchronous batch jobs with precomputed transformations for efficiency and scalability.
Synchronous
On Demand
High-stakes decision making
Use synchronous writes with on-demand feature computation to ensure data freshness and correctness.
Synchronous
Precomputed
User-facing applications requiring quick feedback
Use synchronous writes with precomputed features to reduce latency and improve user experience.
Synchronous
Hybrid (Precomputed + On Demand)
High-stakes decision making that want to optimize for latency under constraints
Use synchronous writes with precomputed features where possible and a select set of on demand computations to reduce latency and improve user experience.
A UI for viewing and exploring information about features defined in the project
A CLI tool for viewing and updating feature information
Decouple ML from data infrastructure by providing a single data access layer that abstracts feature storage from feature retrieval, ensuring models remain portable as you move from training models to serving models, from batch models to real-time models, and from one data infra system to another.
For Data Engineers
: Feast provides a centralized catalog for storing feature definitions, allowing one to maintain a single source of truth for feature data. It provides the abstraction for reading and writing to many different types of offline and online data stores. Using either the provided Python SDK or the feature server service, users can write data to the online and/or offline stores and then read that data out again in either low-latency online scenarios for model inference, or in batch scenarios for model training.
For AI Engineers: Feast provides a platform designed to scale your AI applications by enabling seamless integration of richer data and facilitating fine-tuning. With Feast, you can optimize the performance of your AI models while ensuring a scalable and efficient data pipeline.
A database: Feast is not a database, but helps manage data stored in other systems (e.g. BigQuery, Snowflake, DynamoDB, Redis) to make features consistently available at training / serving time
lineage: Feast helps tie feature values to model versions, but is not a complete solution for capturing end-to-end lineage from raw data sources to model versions. Feast also has community contributed plugins with DataHub and Amundsen.
data quality / drift detection: Feast has experimental integrations with Great Expectations, but is not purpose built to solve data drift / data quality issues. This requires more sophisticated monitoring across data pipelines, served feature values, labels, and model versions.
Credit scoring, using pre-computed historical features to compute the probability of default
Tutorials show full examples of using Feast in machine learning applications.
One using a configurable OIDC server to extract the user details.
One using the Kubernetes RBAC resources to extract the user details.
These instances are created when the Feast servers are initialized, according to the authorization configuration defined in their own feature_store.yaml.
Feast servers and clients must have consistent authorization configuration, so that the client proxies can automatically inject the authorization tokens that the server can properly identify and use to enforce permission validations.
Design notes
The server-side implementation of the authorization functionality is defined here. Few of the key models, classes to understand the authorization implementation on the client side can be found here.
Configuring Authorization
The authorization is configured using a dedicated auth section in the feature_store.yaml configuration.
Note: As a consequence, when deploying the Feast servers with the Helm charts, the feature_store_yaml_base64 value must include the auth section to specify the authorization configuration.
No Authorization
This configuration applies the default no_auth authorization:
OIDC Authorization
With OIDC authorization, the Feast client proxies retrieve the JWT token from an OIDC server (or Identity Provider) and append it in every request to a Feast server, using an Authorization Bearer Token.
The server, in turn, uses the same OIDC server to validate the token and extract user details — including username, roles, and groups — from the token itself.
Some assumptions are made in the OIDC server configuration:
The OIDC token refers to a client with roles matching the RBAC roles of the configured Permissions (*)
The roles are exposed in the access token under resource_access.<client_id>.roles
The JWT token is expected to have a verified signature and not be expired. The Feast OIDC token parser logic validates for verify_signature and verify_exp so make sure that the given OIDC provider is configured to meet these requirements.
The preferred_username should be part of the JWT token claim.
For GroupBasedPolicy support, the groups claim should be present in the access token (requires a "Group Membership" protocol mapper in Keycloak).
(*) Please note that the role match is case-sensitive, e.g. the name of the role in the OIDC server and in the Permission configuration must be exactly the same.
For example, the access token for a client app of a user with reader role and membership in the data-team group should have the following claims:
Server-Side Configuration
The server requires auth_discovery_url and client_id to validate incoming JWT tokens via JWKS:
When the OIDC provider uses a self-signed or untrusted TLS certificate (e.g. internal Keycloak on OpenShift), set verify_ssl to false to disable certificate verification:
Setting verify_ssl: false disables TLS certificate verification for all OIDC provider communication (discovery, JWKS, token endpoint). Only use this in development or internal environments where you accept the security risk.
Client-Side Configuration
The client supports multiple token source modes. The SDK resolves tokens in the following priority order:
Kubernetes service account token — read from /var/run/secrets/kubernetes.io/serviceaccount/token when running inside a pod
Token passthrough (for use with external token providers like kube-authkit):
Or with a bare type: oidc (no other fields) — the SDK falls back to the FEAST_OIDC_TOKEN environment variable or a mounted Kubernetes service account token:
When using client credentials or ROPC flows, the verify_ssl setting also applies to the discovery and token endpoint requests.
Multi-Token Support (OIDC + Kubernetes Service Account)
When the Feast server is configured with OIDC auth and deployed on Kubernetes, the OidcTokenParser can handle both Keycloak JWT tokens and Kubernetes service account tokens. Incoming tokens that contain a kubernetes.io claim are validated via the Kubernetes Token Access Review API and the namespace is extracted from the authenticated identity — no RBAC queries are performed, so the server service account only needs tokenreviews/create permission. All other tokens follow the standard OIDC/Keycloak JWKS validation path. This enables NamespaceBasedPolicy enforcement for service account tokens while using GroupBasedPolicy and RoleBasedPolicy for OIDC user tokens.
Kubernetes RBAC Authorization
With Kubernetes RBAC Authorization, the client uses the service account token as the authorizarion bearer token, and the server fetches the associated roles from the Kubernetes RBAC resources. Feast supports advanced authorization by extracting user groups and namespaces from Kubernetes tokens, enabling fine-grained access control beyond simple role matching. This is achieved by leveraging Kubernetes Token Access Review, which allows Feast to determine the groups and namespaces associated with a user or service account.
An example of Kubernetes RBAC authorization configuration is the following:
NOTE: This configuration will only work if you deploy feast on Openshift or a Kubernetes platform.
```yaml project: my-project auth: type: kubernetes user_token: #Optional, else service account token Or env var is used for getting the token ... ```
In case the client cannot run on the same cluster as the servers, the client token can be injected using the LOCAL_K8S_TOKEN environment variable on the client side. The value must refer to the token of a service account created on the servers cluster and linked to the desired RBAC roles/groups/namespaces.
registry:
local:
server:
restAPI: true # enable REST API (default: true when server is set)
grpc: true # enable gRPC (default: true when server is set)
authz:
oidc:
secretRef:
name: oidc-secret
secretKeyName: client_id # override the default Secret key name
tokenEnvVar: FEAST_TOKEN # env var from which servers read the Bearer token
verifySSL: false # disable SSL verification (dev only)
caCertConfigMap: oidc-ca-cert # ConfigMap with CA cert for SSL verification
feast init feast_repo_ssl_demo
#output will be something similar as below
Creating a new Feast repository in /Documents/Src/feast/feast_repo_ssl_demo.
cd feast_repo_ssl_demo/feature_repo
feast apply
#output will be something similar as below
Applying changes for project feast_repo_ssl_demo
Created project feast_repo_ssl_demo
Created entity driver
Created feature view driver_hourly_stats
Created feature view driver_hourly_stats_fresh
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v1
Created feature service driver_activity_v3
Created feature service driver_activity_v2
Created sqlite table feast_repo_ssl_demo_driver_hourly_stats_fresh
Created sqlite table feast_repo_ssl_demo_driver_hourly_stats
11/04/2024 03:10:27 PM feast.registry_server INFO: Starting grpc registry server in TLS(SSL) mode
11/04/2024 03:10:27 PM feast.registry_server INFO: Grpc server started at https://localhost:6570
11/07/2024 11:10:01 AM feast.offline_server INFO: Found SSL certificates in the args so going to start offline server in TLS(SSL) mode.
11/07/2024 11:10:01 AM feast.offline_server INFO: Offline store server serving at: grpc+tls://127.0.0.1:8815
11/07/2024 11:10:01 AM feast.offline_server INFO: offline server starting with pid: [11606]
INFO: Started server process [78872]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on https://0.0.0.0:8888 (Press CTRL+C to quit)
from feast.infra.offline_stores.contrib.couchbase_offline_store.couchbase_source import (
CouchbaseColumnarSource,
)
driver_stats_source = CouchbaseColumnarSource(
name="driver_hourly_stats_source",
query="SELECT * FROM Default.Default.`feast_driver_hourly_stats`",
database="Default",
scope="Default",
collection="feast_driver_hourly_stats",
timestamp_field="event_timestamp",
created_timestamp_column="created",
)
The operator runs an init container that clones the repository before the Feast processes start. Use this for production: your feature definitions live in version control and the operator tracks a specific commit or branch.
Minimal example
Pinning to a specific commit (recommended for production)
Monorepo: feature repo in a subdirectory
When the Feast feature repository lives inside a larger monorepo, use featureRepoPath to point at the subdirectory (relative path, no leading /):
Private repositories — token authentication
Create a Kubernetes Secret containing the token:
Reference the Secret from envFrom and rewrite the remote URL via configs:
Disabling TLS verification (not recommended for production)
Full git field reference
Field
Type
Description
url
string
Repository URL (HTTPS or SSH)
ref
string
Branch, tag, or commit SHA. Defaults to the remote HEAD
Option B — Scaffold a new project (feastProjectDir.init)
The operator runs feast init on first startup to create a minimal feature repository. Use this for development, demos, and CI environments where you do not yet have a feature repo to point at.
Templates
feast init supports store-specific templates. Set template to generate a scaffold that matches your chosen online/offline store:
Available templates (validated by the CRD): local · gcp · aws · snowflake · spark · postgres · hbase · cassandra · hazelcast · couchbase · clickhouse
Minimal scaffold
minimal: true skips example feature-view files and creates only the bare feature_store.yaml:
Full init field reference
Field
Type
Default
Description
template
string
local
Template name for feast init --template
minimal
bool
feast apply on startup
By default, when the init container completes (git clone or feast init), the operator runs feast apply before starting the servers. This registers all feature definitions with the registry.
To skipfeast apply on pod start (e.g. you manage registry updates separately):
Or to keep the init container but skip the apply step:
When feastProjectDir is omitted
If neither git nor init is set, the operator mounts an empty directory. In this case you must supply a feature_store.yaml through another mechanism (e.g. a ConfigMap volume mount via services.volumes + volumeMounts).
feastProjectDir:
git:
url: https://github.com/my-org/private-repo
configs:
# Replaces the HTTPS URL with one that includes the token
'url."https://api:${TOKEN}@github.com/".insteadOf': 'https://github.com/'
envFrom:
- secretRef:
name: git-token
Feast is designed to be easy to use and understand out of the box, with as few infrastructure dependencies as possible. However, there are components used by default that may not scale well. Since Feast is designed to be modular, it's possible to swap such components with more performant components, at the cost of Feast depending on additional infrastructure.
Scaling Feast Registry
The default Feast is a file-based registry. Any changes to the feature repo, or materializing data into the online store, results in a mutation to the registry.
However, there are inherent limitations with a file-based registry, since changing a single field in the registry requires re-writing the whole registry file. With multiple concurrent writers, this presents a risk of data loss, or bottlenecks writes to the registry since all changes have to be serialized (e.g. when running materialization for multiple feature views or time ranges concurrently).
The recommended solution in this case is to use the , which allows concurrent, transactional, and fine-grained updates to the registry. This registry implementation requires access to an existing database (such as MySQL, Postgres, etc).
Scaling Materialization
The default Feast materialization process is an in-memory process, which pulls data from the offline store before writing it to the online store. However, this process does not scale for large data sets, since it's executed on a single-process.
Feast supports pluggable , that allow the materialization process to be scaled up. Aside from the local process, Feast supports a , and a .
Users may also be able to build an engine to scale up materialization using existing infrastructure in their organizations.
Horizontal Scaling with the Feast Operator
When running Feast on Kubernetes with the , you can horizontally scale the FeatureStore deployment using spec.replicas or HPA autoscaling. The FeatureStore CRD implements the Kubernetes , so you can also use kubectl scale:
Prerequisites: Horizontal scaling requires DB-backed persistence for all enabled services (online store, offline store, and registry). File-based persistence (SQLite, DuckDB, registry.db) is incompatible with multiple replicas because these backends do not support concurrent access from multiple pods.
Static Replicas
Set a fixed number of replicas via spec.replicas:
Autoscaling with HPA
Configure a HorizontalPodAutoscaler to dynamically scale based on metrics. HPA autoscaling is configured under services.scaling.autoscaling and is mutually exclusive with spec.replicas > 1:
When autoscaling is configured, the operator automatically sets the deployment strategy to RollingUpdate (instead of the default Recreate) to ensure zero-downtime scaling, and auto-injects soft pod anti-affinity and zone topology spread constraints. You can override any of these by explicitly setting deploymentStrategy, affinity, or topologySpreadConstraints in the CR.
Validation Rules
The operator enforces the following rules:
spec.replicas > 1 and services.scaling.autoscaling are mutually exclusive -- you cannot set both.
Scaling with replicas > 1 or any autoscaling config is rejected if any enabled service uses file-based persistence.
High Availability
When scaling is enabled (replicas > 1 or autoscaling), the operator provides HA features to improve resilience:
Pod Anti-Affinity — The operator automatically injects a soft (preferredDuringSchedulingIgnoredDuringExecution) pod anti-affinity rule that prefers spreading pods across different nodes. This prevents multiple replicas from being co-located on the same node, improving resilience to node failures. You can override this by providing your own affinity configuration:
Topology Spread Constraints — The operator automatically injects a soft zone-spread constraint (whenUnsatisfiable: ScheduleAnyway) that distributes pods across availability zones. This is a best-effort spread — if zones are unavailable, pods will still be scheduled. You can override this with explicit constraints or disable it with an empty array:
To disable the auto-injected topology spread:
PodDisruptionBudget — You can configure a PDB to limit voluntary disruptions (e.g. during node drains or cluster upgrades). The PDB is only created when scaling is enabled. Exactly one of minAvailable or maxUnavailable must be set:
The PDB is not auto-injected — you must explicitly configure it. This is intentional because a misconfigured PDB (e.g. minAvailable equal to the replica count) can block node drains and cluster upgrades.
Using KEDA (Kubernetes Event-Driven Autoscaling)
is also supported as an external autoscaler. KEDA should target the FeatureStore's scale sub-resource directly (since it implements the Kubernetes scale API). This is the recommended approach because the operator manages the Deployment's replica count from spec.replicas — targeting the Deployment directly would conflict with the operator's reconciliation.
When using KEDA, do not set scaling.autoscaling or spec.replicas > 1 -- KEDA manages the replica count through the scale sub-resource.
Ensure DB-backed persistence -- The CRD's CEL validation rules automatically enforce DB-backed persistence when KEDA scales spec.replicas above 1 via the scale sub-resource. The operator also automatically switches the deployment strategy to RollingUpdate when replicas > 1.
Configure the FeatureStore with DB-backed persistence:
Create a KEDA ScaledObject targeting the FeatureStore resource:
KEDA-created HPAs are not owned by the Feast operator. The operator will not interfere with them, but it also will not clean them up if the FeatureStore CR is deleted. You must manage the KEDA ScaledObject lifecycle independently.
For the full API reference, see the .
Roadmap
The list below contains the functionality that contributors are planning to develop for Feast.
We welcome contribution to all items in the roadmap!
Natural Language Processing
S3 (s3://) and GCS (gs://) backed registry file persistence is allowed with scaling, since these object stores support concurrent readers.
spec:
replicas: 3
services:
podDisruptionBudgets:
maxUnavailable: 1 # at most 1 pod unavailable during disruptions
# -- OR --
# podDisruptionBudgets:
# minAvailable: "50%" # at least 50% of pods must remain available
# ...
Feast provides robust support for Generative AI applications, enabling teams to build, deploy, and manage feature infrastructure for Large Language Models (LLMs) and other Generative AI (GenAI) applications. With Feast's vector database integrations and feature management capabilities, teams can implement production-ready Retrieval Augmented Generation (RAG) systems and other GenAI applications with the same reliability and operational excellence as traditional ML systems.
3 — Serving & Observability
This guide covers the spec.services.onlineStore.serving block and general server configuration (server block) for all Feast services: per-worker tuning, log levels, Prometheus metrics, offline push batching, and MCP (Model Context Protocol).
Server configuration (server block)
6 — Batch & Jobs
This guide covers two related top-level spec fields:
spec.batchEngine — override the compute engine used for materialization
spec.cronJob — schedule periodic feast materialize-incremental (or any command) as a Kubernetes CronJob
Postgres with PGVector: SQL-based vector operations
Qdrant: Purpose-built vector database integration
These integrations allow you to:
Store embeddings as features
Perform vector similarity search to find relevant context
Retrieve both vector embeddings and traditional features in a single API call
Retrieval Augmented Generation (RAG)
Feast simplifies building RAG applications by providing:
Embedding storage: Store and version embeddings alongside your other features
Vector similarity search: Find the most relevant data/documents for a given query
Feature retrieval: Combine embeddings with structured features for richer context
Versioning and governance: Track changes to your document repository over time
The typical RAG workflow with Feast involves:
Transforming Unstructured Data to Structured Data
Feast provides powerful capabilities for transforming unstructured data (like PDFs, text documents, and images) into structured embeddings that can be used for RAG applications:
Document Processing Pipelines: Integrate with document processing tools like Docling to extract text from PDFs and other document formats
Chunking and Embedding Generation: Process documents into smaller chunks and generate embeddings using models like Sentence Transformers
On-Demand Transformations: Use @on_demand_feature_view decorator to transform raw documents into embeddings in real-time
Batch Processing with Spark: Scale document processing for large datasets using Spark integration
The transformation workflow typically involves:
Raw Data Ingestion: Load documents or other data from various sources (file systems, databases, etc.)
Text Extraction: Extract text content from unstructured documents
Chunking: Split documents into smaller, semantically meaningful chunks
Embedding Generation: Convert text chunks into vector embeddings
Storage: Store embeddings and metadata in Feast's feature store
The DocEmbedder class provides an end-to-end pipeline for ingesting documents into Feast's online vector store. It handles chunking, embedding generation, and writing results -- all in a single step.
Key Components
DocEmbedder: High-level orchestrator that runs the full pipeline: chunk → embed → schema transform → write to online store
BaseChunker / TextChunker: Pluggable chunking layer. TextChunker splits text by word count with configurable chunk_size, chunk_overlap, min_chunk_size, and max_chunk_chars
BaseEmbedder / MultiModalEmbedder: Pluggable embedding layer with modality routing. MultiModalEmbedder supports text (via sentence-transformers) and image (via CLIP) with lazy model loading
SchemaTransformFn: A user-defined function that transforms the chunked + embedded DataFrame into the format expected by the FeatureView schema
Quick Example
Features
Auto-generates FeatureView: Creates a Python file with Entity and FeatureView definitions compatible with feast apply
Auto-applies repo: Registers the generated FeatureView in the registry automatically
Custom schema transform: Provide your own SchemaTransformFn to control how chunked + embedded data maps to your FeatureView schema
Extensible: Subclass BaseChunker or BaseEmbedder to plug in your own chunking or embedding strategies
Feast supports transformations that can be used to:
Process raw text into embeddings
Chunk documents for more effective retrieval
Normalize and preprocess features before serving to LLMs
Apply custom transformations to adapt features for specific LLM requirements
Use Cases
Document Question-Answering
Build document Q&A systems by:
Storing document chunks and their embeddings in Feast
Converting user questions to embeddings
Retrieving relevant document chunks
Providing these chunks as context to an LLM
Knowledge Base Augmentation
Enhance your LLM's knowledge by:
Storing company-specific information as embeddings
Retrieving relevant information based on user queries
Injecting this information into the LLM's context
Semantic Search
Implement semantic search by:
Storing document embeddings in Feast
Converting search queries to embeddings
Finding semantically similar documents using vector search
AI Agents with Context and Memory
Feast can serve as both the context provider and persistent memory layer for AI agents. Unlike stateless RAG pipelines, agents make autonomous decisions about which tools to call and can write state back to the feature store:
Structured context: Retrieve customer profiles, account data, and other entity-keyed features
Knowledge retrieval: Search vector embeddings for relevant documents
Persistent memory: Store and recall per-entity interaction history (last topic, resolution, preferences) using write_to_online_store
Governed access: All reads and writes are subject to the same RBAC, TTL, and audit policies as any other feature
With MCP enabled, agents built with any framework (LangChain, LlamaIndex, CrewAI, AutoGen, or custom) can discover and call Feast tools dynamically. See the Feast-Powered AI Agent example and the blog post Building AI Agents with Feast for a complete walkthrough.
Scaling with Spark Integration
Feast integrates with Apache Spark to enable large-scale processing of unstructured data for GenAI applications:
Spark Data Source: Load data from Spark tables, files, or SQL queries for feature generation
Spark Offline Store: Process large document collections and generate embeddings at scale
Spark Batch Materialization: Efficiently materialize features from offline to online stores
Distributed Processing: Handle gigabytes of documents and millions of embeddings
This integration enables:
Processing large document collections in parallel
Generating embeddings for millions of text chunks
Efficiently materializing features to vector databases
Scaling RAG applications to enterprise-level document repositories
Scaling with Ray Integration
Feast integrates with Ray to enable distributed processing for RAG applications:
Ray Compute Engine: Distributed feature computation using Ray's task and actor model
Ray Offline Store: Process large document collections and generate embeddings at scale
Ray Batch Materialization: Efficiently materialize features from offline to online stores
Distributed Embedding Generation: Scale embedding generation across multiple nodes
This integration enables:
Distributed processing of large document collections
Parallel embedding generation for millions of text chunks
Kubernetes-native scaling for RAG applications
Efficient resource utilization across multiple nodes
Feast supports the Model Context Protocol (MCP), which enables AI agents and applications to interact with your feature store through standardized MCP interfaces. This allows seamless integration with LLMs and AI agents for GenAI applications.
Key Benefits of MCP Support
Standardized AI Integration: Enable AI agents to discover and use features dynamically without hardcoded definitions
Easy Setup: Add MCP support with a simple configuration change and pip install feast[mcp]
Agent-Friendly APIs: Expose feature store capabilities through MCP tools that AI agents can understand and use
Production Ready: Built on top of Feast's proven feature serving infrastructure
By default, Feast uses the SSE-based MCP transport (mcp_transport: sse). Streamable HTTP (mcp_transport: http) is recommended for improved compatibility with some MCP clients.
How It Works
The MCP integration uses the fastapi_mcp library to automatically transform your Feast feature server's FastAPI endpoints into MCP-compatible tools. When you enable MCP support:
Automatic Discovery: The integration scans your FastAPI application and discovers all available endpoints
Tool Generation: Each endpoint becomes an MCP tool with auto-generated schemas and descriptions
Dynamic Access: AI agents can discover and call these tools dynamically without hardcoded definitions
Standard Protocol: Uses the Model Context Protocol for standardized AI-to-API communication
Available MCP Tools
The fastapi_mcp integration automatically exposes your Feast feature server's FastAPI endpoints as MCP tools. This means AI assistants can:
Call /get-online-features to retrieve features from the feature store
Call /retrieve-online-documents to perform vector similarity search
Call /write-to-online-store to persist agent state (memory, notes, interaction history)
from feast import DocEmbedder
import pandas as pd
# Prepare your documents
df = pd.DataFrame({
"id": ["doc1", "doc2"],
"text": ["First document content...", "Second document content..."],
})
# Create DocEmbedder -- automatically generates a FeatureView and applies the repo
embedder = DocEmbedder(
repo_path="feature_repo/",
feature_view_name="text_feature_view",
)
# Embed and ingest documents in one step
result = embedder.embed_documents(
documents=df,
id_column="id",
source_column="text",
column_mapping=("text", "text_embedding"),
)
Every deployable Feast service (online store, offline store, registry, UI) accepts a
server
block under
spec.services.<service>
. The online and offline stores use
ServerConfigs
; the registry uses
RegistryServerConfigs
(adds
restAPI
/
grpc
toggles).
Enable a server
An empty server: {} is enough to deploy a service. Without it the component is not deployed (e.g. the offline store runs as a local process, not as a network server):
Log level
Container image and resources
The operator resolves the feature server image through the following priority chain:
server.image in the CR — per-service override, highest priority
RELATED_IMAGE_FEATURE_SERVER env var on the operator pod — cluster-wide default set by OLM/platform
The same RELATED_IMAGE_FEATURE_SERVER image is used for all server containers (online, offline, registry, UI) and for the init containers (git clone, feast apply). Setting it overrides all of them at once without touching any CR.
The cronJob container uses a separate env var: RELATED_IMAGE_CRON_JOB (default: quay.io/openshift/origin-cli:4.17).
Cluster-wide image override (operator env var) — set this on the operator Deployment to redirect all pods cluster-wide to a different registry (e.g. a private mirror or a pinned digest):
Per-service CR override — for a single FeatureStore or to pin one service to a different image than the cluster default:
Worker configuration (gunicorn)
The online and offline servers run on gunicorn. Tune worker count, connections, and request limits:
Production recommendation: set workers: -1 and registryTtlSec: 60 or higher. See Online Server Performance Tuning for detailed guidance.
Environment variables and secrets
Inject environment variables from Secrets or ConfigMaps into any server:
Volume mounts
Mount additional volumes (ConfigMaps, Secrets, PVCs) into the server containers:
TLS
All servers support TLS termination. Provide a Kubernetes Secret containing the TLS certificate and key, and reference it from tls:
For mTLS (mutual TLS), also set a CA certificate ConfigMap:
Prometheus Metrics
When metrics are enabled the feature server starts an HTTP server on port 8000 for Prometheus scraping. The operator automatically adds a containerPort, a Kubernetes Service port, and a ServiceMonitor for Prometheus discovery.
Two paths — use either or both
Path 1: CLI flag (existing, simple)
Path 2: YAML config (new, granular)
Both paths expose port 8000 and create a ServiceMonitor. When serving.metrics.enabled is true the Python server reads it from feature_store.yaml directly; no --metrics flag is injected. When server.metrics: true is used, the --metrics flag is injected.
SDK note: MetricsConfig uses extra="forbid" in Pydantic. Only use category keys that are recognized by your Feast SDK version.
Verify monitoring is wired:
Offline Push Batching
When features are pushed to the online store via /push, each request also triggers a synchronous offline store write. At high push throughput this causes OOM. Push batching groups these writes into fixed-size batches flushed on a timer.
Field
Type
Description
enabled
bool
Activates batching
batchSize
int
Max rows per batch; flush when reached
MCP (Model Context Protocol)
MCP mounts LLM-agent-compatible tool endpoints alongside the existing REST API on port 6566. The REST API is not replaced — MCP is additive.
The operator writes feature_server.type: mcp into feature_store.yaml only when serving.mcp.enabled: true. Setting enabled: false reverts to type: local.
Field
Type
Default
Description
enabled
bool
—
Must be true; false keeps type: local
serverName
string
MCP is mounted at /mcp on port 6566 — no additional Kubernetes Service is created.
Dependency: the feature server image must include feast[mcp] (fastapi-mcp). Without it the server starts normally but MCP routes are not registered.
services:
onlineStore:
server: {} # deploys the online feature server on port 6566
offlineStore:
server: {} # deploys the offline feature server on port 8815
registry:
local:
server: {} # deploys the registry server on port 6570
ui: {} # deploys the Feast UI
By default, Feast runs materialization using the local batch engine (in-process Python). For large feature sets you can point the operator at a Spark, Ray, or other supported engine via a Kubernetes ConfigMap.
ConfigMap format
Create a ConfigMap whose value is a YAML snippet identical to the batch_engine section of feature_store.yaml. Include the type: key and all engine-specific options:
Reference the ConfigMap from the CR:
Engine types
type
Notes
local
Default; in-process Python, no extra infra
spark
Apache Spark; requires a Spark operator or standalone cluster
ray
Ray cluster; requires a Ray operator
For engine-specific YAML options (Spark conf, Ray address, etc.) see the Feast SDK — Compute Engine docs.
Scheduled Materialization (spec.cronJob)
The operator can deploy a Kubernetes CronJob that runs feast materialize-incremental (or any custom command) on a schedule. This is the recommended way to keep your online store fresh without managing an external job scheduler.
CronJob image resolution
The CronJob container image is resolved through the following priority chain:
cronJob.containerConfigs.image in the CR — per-CronJob override
RELATED_IMAGE_CRON_JOB env var on the operator pod — cluster-wide default set by OLM/platform (default: quay.io/openshift/origin-cli:4.17)
Minimal example — nightly materialization
The CronJob runs the default feast materialize-incremental command using the same feature_store.yaml that the operator generated for this FeatureStore.
Custom command
Override the container command to run any Feast CLI command:
Or run a Python script instead:
Time zone
Concurrency policy
Job history retention
Suspend a CronJob
To temporarily pause scheduled runs without deleting the CronJob:
Resource requests for the job pod
Environment variables and secrets in the job pod
Advanced job spec
Full cronJob field reference
Field
Type
Default
Description
schedule
string
—
Cron expression (required)
timeZone
string
Combining batchEngine + cronJob
Use both together to run scheduled Spark-based materialization:
# Override cluster-wide for all CronJobs
kubectl set env deployment/feast-operator-controller-manager \
RELATED_IMAGE_CRON_JOB=my-registry.example.com/tools/cli:latest \
-n feast-operator-system
apiVersion: feast.dev/v1
kind: FeatureStore
metadata:
name: feast-production
spec:
feastProject: my_project
cronJob:
schedule: "0 2 * * *" # every day at 02:00 UTC
containerConfigs:
image: quay.io/feastdev/feature-server:0.62.0
cronJob:
schedule: "*/30 * * * *"
containerConfigs:
image: quay.io/feastdev/feature-server:0.62.0
commands:
- feast
- materialize-incremental
- "2099-01-01T00:00:00" # materialize up to a fixed end time
The offline store, online store, and registry each need a place to store data. The operator supports two persistence patterns for each:
File persistence — a path on a volume (ephemeral or PVC-backed)
DB/store persistence — an external database or managed store, wired via a Kubernetes Secret
The pattern is the same for all three services; only the store types differ.
Persistence patterns at a glance
Pattern
Best for
Data survives pod restart?
File persistence
Ephemeral (emptyDir)
The default when no persistence block is set. Data lives on the pod's local disk and is lost on restart. Suitable for development only.
PVC — reference an existing PVC
When you already have a PVC provisioned (e.g. by your storage team):
PVC — let the operator create one
Omitting storageClassName uses the cluster default StorageClass. Omitting create entirely creates a PVC with the operator's built-in defaults (1 Gi, default StorageClass).
DB / store persistence
For production, point the operator at an external database. The operator reads connection details from a Kubernetes Secret and writes them into feature_store.yaml.
Secret format
The Secret must contain one key per store component. The key name is the store type (e.g. postgres, sql, redis). The value is a YAML snippet identical to what you would write under the corresponding section in feature_store.yaml, minus the type: key (the operator inserts it from persistence.store.type).
Reference the Secret from the CR:
Key lookup rule: the operator looks up the Secret key that matches persistence.store.type (e.g. type: postgres → key postgres). Keep all stores in one Secret or split across multiple — both work.
Injecting DB credentials into the server pod
The Secret key values in the example above hard-code passwords. For production, keep credentials in a separate Secret and inject them as environment variables using envFrom, then reference them with ${VAR} substitution in the data-stores Secret value:
Store types by component
Online store
For all store-specific YAML keys (connection options, pool sizes, etc.) see the linked SDK docs — the Secret value accepts the same keys.
Offline store
For external DB-backed offline stores (BigQuery, Snowflake, Spark, Trino, etc.), use persistence.store.type and a Secret with the matching key. See in the SDK docs.
Registry
For sql, the Secret value is a path: (SQLAlchemy URL) plus optional cache_ttl_seconds and sqlalchemy_config_kwargs:
services:
onlineStore:
server: {} # no persistence block → emptyDir
services:
onlineStore:
persistence:
file:
path: online_store.db # filename relative to mountPath
pvc:
ref:
name: online-pvc # name of the existing PVC
mountPath: /data/online # where the PVC is mounted in the pod
Alpha Feature: The dbt integration is currently in early development and subject to change.
Current Limitations:
Supported data sources: BigQuery, Snowflake, and File-based sources only
Manual entity column specification required
Breaking changes may occur in future releases.
This guide explains how to use Feast's dbt integration to automatically import dbt models as Feast FeatureViews. This enables you to leverage your existing dbt transformations as feature definitions without manual duplication.
Overview
is a popular tool for transforming data in your warehouse. Many teams already use dbt to create feature tables. Feast's dbt integration allows you to:
Discover dbt models tagged for feature engineering
Import model metadata (columns, types, descriptions) as Feast objects
Generate Python code for Entity, DataSource, and FeatureView definitions
This eliminates the need to manually define Feast objects that mirror your dbt models.
Prerequisites
A dbt project with compiled artifacts (target/manifest.json)
Feast installed with dbt support:
Or install the parser directly:
Quick Start
1. Tag your dbt models
In your dbt project, add a feast tag to models you want to import:
2. Define column types in schema.yml
Feast uses column metadata from your schema.yml to determine feature types:
3. Compile your dbt project
This generates target/manifest.json which Feast will read.
4. List available models
Use the Feast CLI to discover tagged models:
Output:
5. Import models as Feast definitions
Generate a Python file with Feast object definitions:
This generates:
Multiple Entity Support
The dbt integration supports feature views with multiple entities, useful for modeling relationships involving multiple keys.
Usage
Specify multiple entity columns using repeated -e flags:
This creates a FeatureView with both user_id and merchant_id as entities, useful for:
Transaction features keyed by both user and merchant
Interaction features keyed by multiple parties
Association tables in many-to-many relationships
Single entity usage:
Requirements
All specified entity columns must exist in each dbt model being imported. Models missing any entity column will be skipped with a warning.
Generated Code
The --output flag generates code like:
CLI Reference
feast dbt list
Discover dbt models available for import.
Arguments:
manifest_path: Path to dbt's manifest.json file
Options:
--tag-filter, -t: Filter models by dbt tag (e.g., feast)
--model, -m: Filter to specific model name(s)
feast dbt import
Import dbt models as Feast object definitions.
Arguments:
manifest_path: Path to dbt's manifest.json file
Options:
Option
Description
Default
Type Mapping
Feast automatically maps dbt/warehouse column types to Feast types:
dbt/SQL Type
Feast Type
Snowflake NUMBER(precision, scale) types are handled specially:
Scale > 0: Float64
Precision <= 9: Int32
Precision <= 18: Int64
Data Source Configuration
BigQuery
Generates BigQuerySource with the full table path from dbt metadata:
Snowflake
Generates SnowflakeSource with database, schema, and table:
File
Generates FileSource with a placeholder path:
For file sources, update the generated path to point to your actual data files.
Best Practices
1. Use consistent tagging
Create a standard tagging convention in your dbt project:
2. Document your columns
Column descriptions from schema.yml are preserved in the generated Feast definitions, making your feature catalog self-documenting.
3. Review before committing
Use --dry-run to preview what will be generated:
4. Version control generated code
Commit the generated Python files to your repository. This allows you to:
Track changes to feature definitions over time
Review dbt-to-Feast mapping in pull requests
Customize generated code if needed
5. Integrate with CI/CD
Add dbt import to your CI pipeline:
Limitations
Single entity support: Currently supports one entity column per import. For multi-entity models, run multiple imports or manually adjust the generated code.
No incremental updates: Each import generates a complete file. Use version control to track changes.
Column types required: Models without data_type in schema.yml default to String
Troubleshooting
"manifest.json not found"
Run dbt compile or dbt run first to generate the manifest file.
"No models found with tag"
Check that your models have the correct tag in their config:
"Missing entity column"
Ensure your dbt model includes the entity column specified with --entity-column. Models missing this column are skipped with a warning.
"Missing timestamp column"
By default, Feast looks for event_timestamp. Use --timestamp-field to specify a different column name.
Table formats
Overview
Table formats are metadata and transaction layers built on top of data storage formats (like Parquet). They provide advanced capabilities for managing large-scale data lakes, including ACID transactions, time travel, schema evolution, and efficient data management.
Feast supports modern table formats to enable data lakehouse architectures with your feature store.
Supported Table Formats
Apache Iceberg
is an open table format designed for huge analytic datasets. It provides:
ACID transactions: Atomic commits with snapshot isolation
Time travel: Query data as of any snapshot
Schema evolution: Add, drop, rename, or reorder columns safely
Basic Configuration
Configuration Options
Parameter
Type
Description
Common Properties
Time Travel Example
Delta Lake
is an open-source storage layer that brings ACID transactions to Apache Spark and big data workloads. It provides:
ACID transactions: Serializable isolation for reads and writes
Time travel: Access and revert to earlier versions
Schema enforcement: Prevent bad data from corrupting tables
Basic Configuration
Configuration Options
Parameter
Type
Description
Common Properties
Time Travel Example
Apache Hudi
(Hadoop Upserts Deletes and Incrementals) is a data lake storage framework for simplifying incremental data processing. It provides:
Upserts and deletes: Efficient record-level updates
Incremental queries: Process only changed data
Time travel: Query historical versions
Basic Configuration
Configuration Options
Parameter
Type
Description
Table Types
COPY_ON_WRITE (COW)
Stores data in columnar format (Parquet)
Updates create new file versions
Best for read-heavy workloads
Lower query latency
MERGE_ON_READ (MOR)
Uses columnar + row-based formats
Updates written to delta logs
Best for write-heavy workloads
Lower write latency
Common Properties
Incremental Query Example
Table Format vs File Format
It's important to understand the distinction:
Aspect
File Format
Table Format
Can be used together
Benefits of Table Formats
Reliability
ACID transactions: Ensure data consistency across concurrent operations
feast dbt list target/manifest.json --tag-filter feast
Found 1 model(s) with tag 'feast':
driver_features
Description: Driver aggregated features for ML models
Columns: driver_id, event_timestamp, avg_rating, total_trips, is_active
Tags: feast
# Read from a specific snapshot
iceberg_format = IcebergFormat(
catalog="spark_catalog",
namespace="lakehouse"
)
iceberg_format.set_property("snapshot-id", "7896524153287651133")
# Or read as of a timestamp
iceberg_format.set_property("as-of-timestamp", "1609459200000")
from feast.table_format import DeltaFormat
delta_format = DeltaFormat()
# Read from a specific version
delta_format = DeltaFormat()
delta_format.set_property("versionAsOf", "10")
# Or read as of a timestamp
delta_format = DeltaFormat()
delta_format.set_property("timestampAsOf", "2024-01-15 12:00:00")
# Process only new/changed data
hudi_format = HudiFormat(
table_type="COPY_ON_WRITE",
record_key="id",
precombine_field="timestamp",
properties={
"hoodie.datasource.query.type": "incremental",
"hoodie.datasource.read.begin.instanttime": "20240101000000",
"hoodie.datasource.read.end.instanttime": "20240102000000"
}
)
# Table format (metadata layer) built on top of file format (storage layer)
from feast.infra.offline_stores.contrib.spark_offline_store.spark_source import SparkSource
from feast.table_format import IcebergFormat
iceberg = IcebergFormat(catalog="my_catalog", namespace="db")
source = SparkSource(
name="features",
path="catalog.db.table",
file_format="parquet", # Underlying storage format
table_format=iceberg, # Table metadata format
timestamp_field="event_timestamp"
)
# Iceberg - hidden partitioning
iceberg_format.set_property("partition-spec", "days(event_timestamp)")
# Delta - explicit partitioning in data source
# Hudi - configure via properties
hudi_format.set_property("hoodie.datasource.write.partitionpath.field", "date")
# Regularly clean up old snapshots/versions
# For Iceberg: Use expire_snapshots() procedure
# For Delta: Use VACUUM command
# For Hudi: Configure retention policies
# Always test schema changes in non-production first
# Ensure backward compatibility
# Use proper migration procedures
is defined as a
collection of features
.
In the online settings, this is a stateful collection of features that are read when the get_online_features method is called.
In the offline setting, this is a stateless collection of features that are created when the get_historical_features method is called.
A feature view is an object representing a logical group of time-series feature data as it is found in a data source. Depending on the kind of feature view, it may contain some lightweight (experimental) feature transformations (see [Beta] On demand feature views).
If the features are not related to a specific object, the feature view might not have entities; see feature views without entities below.
a name to uniquely identify this feature view in the project.
(optional, but recommended) a schema specifying one or more (without this, Feast will infer the schema by reading from the data source)
(optional, but recommended) metadata (for example, description, or other free-form metadata via tags)
(optional) owner: the email of the primary maintainer
(optional) org: the organizational unit that owns the feature view (e.g. "ads", "search"); useful for grouping feature views by team or product area
(optional) a TTL, which limits how far back Feast will look when generating historical datasets
(optional) enable_validation=True, which enables schema validation during materialization (see below)
Feature views allow Feast to model your existing feature data in a consistent way in both an offline (training) and online (serving) environment. Feature views generally contain features that are properties of a specific object, in which case that object is defined as an entity and included in the feature view.
The generation of training datasets by querying the data source of feature views in order to find historical feature values. A single training dataset may consist of features from multiple feature views.
Loading of feature values into an online store. Feature views determine the storage schema in the online store. Feature values can be loaded from batch sources or from stream sources.
Retrieval of features from the online store. Feature views provide the schema definition to Feast in order to look up features from the online store.
Feature views without entities
If a feature view contains features that are not related to a specific entity, the feature view can be defined without entities (only timestamps are needed for this feature view).
from feast import BigQuerySource, FeatureView, Field
from feast.types import Int64
global_stats_fv = FeatureView(
name="global_stats",
entities=[],
schema=[
Field(name="total_trips_today_by_all_drivers", dtype=Int64),
],
source=BigQuerySource(
table="feast-oss.demo_data.global_stats"
)
)
Feature inferencing
If the schema parameter is not specified in the creation of the feature view, Feast will infer the features during feast apply by creating a Field for each column in the underlying data source except the columns corresponding to the entities of the feature view or the columns corresponding to the timestamp columns of the feature view's data source. The names and value types of the inferred features will use the names and data types of the columns from which the features were inferred.
Entity aliasing
"Entity aliases" can be specified to join entity_dataframe columns that do not match the column names in the source table of a FeatureView.
This could be used if a user has no control over these column names or if there are multiple entities are a subclass of a more general entity. For example, "spammer" and "reporter" could be aliases of a "user" entity, and "origin" and "destination" could be aliases of a "location" entity as shown below.
It is suggested that you dynamically specify the new FeatureView name using .with_name and join_key_map override using .with_join_key_map instead of needing to register each new copy.
from feast import BigQuerySource, Entity, FeatureView, Field
from feast.types import Int32, Int64
location = Entity(name="location", join_keys=["location_id"])
location_stats_fv= FeatureView(
name="location_stats",
entities=[location],
schema=[
Field(name="temperature", dtype=Int32),
Field(name="location_id", dtype=Int64),
],
source=BigQuerySource(
table="feast-oss.demo_data.location_stats"
),
)
A field or feature is an individual measurable property. It is typically a property observed on a specific entity, but does not have to be associated with an entity. For example, a feature of a customer entity could be the number of transactions they have made on an average month, while a feature that is not observed on a specific entity could be the total number of posts made by all users in the last month. Supported types for fields in Feast can be found in sdk/python/feast/types.py.
Fields are defined as part of feature views. Since Feast does not transform data, a field is essentially a schema that only contains a name and a type:
Together with data sources, they indicate to Feast where to find your feature values, e.g., in a specific parquet file or BigQuery table. Feature definitions are also used when reading features from the feature store, using feature references.
Feature names must be unique within a feature view.
Each field can have additional metadata associated with it, specified as key-value tags.
[Alpha] Versioning
Feature views support automatic version tracking. Every time feast apply detects a schema or UDF change, a versioned snapshot is saved to the registry. This enables auditing what changed, reverting to a prior version, querying specific versions via @v<N> syntax, and staging new versions without promoting them.
Version history tracking is always active with no configuration needed. The version parameter is fully optional — omitting it preserves existing behavior.
For full details on version pinning, version-qualified reads, staged publishing (--no-promote), online store support, and known limitations, see the [Alpha] Feature View Versioning reference page.
Schema Validation
Feature views support an optional enable_validation parameter that enables schema validation during materialization and historical feature retrieval. When enabled, Feast verifies that:
All declared feature columns are present in the input data.
Column data types match the expected Feast types (mismatches are logged as warnings).
This is useful for catching data quality issues early in the pipeline. To enable it:
JSON vs Map vs Struct: These three complex types serve different purposes:
Map: Schema-free dictionary with string keys. Use when the keys and values are dynamic.
Json: Opaque JSON data stored as a string. Backends use native JSON types (jsonb, VARIANT). Use for configuration blobs or API responses where you don't need field-level typing.
Struct: Schema-aware structured type with named, typed fields. Persisted through the registry via Field tags. Use when you know the exact structure and want type safety.
Validation is supported in all compute engines (Local, Spark, and Ray). When a required column is missing, a ValueError is raised. Type mismatches are logged as warnings but do not block execution, allowing for safe gradual adoption.
The enable_validation parameter is also available on BatchFeatureView and StreamFeatureView, as well as their respective decorators (@batch_feature_view and @stream_feature_view).
[Alpha] On demand feature views
On demand feature views allows data scientists to use existing features and request time data (features only available at request time) to transform and create new features. Users define python transformation logic which is executed in both the historical retrieval and online retrieval paths.
Currently, these transformations are executed locally. This is fine for online serving, but does not scale well to offline retrieval.
Why use on demand feature views?
This enables data scientists to easily impact the online feature retrieval path. For example, a data scientist could
Call get_historical_features to generate a training dataframe
Iterate in notebook on feature engineering in Pandas
Copy transformation logic into on demand feature views and commit to a dev branch of the feature repository
Verify with get_historical_features (on a small dataset) that the transformation gives expected output over historical data
Verify with get_online_features on dev branch that the transformation correctly outputs online features
Submit a pull request to the staging / prod branches which impact production traffic
[Alpha] Stream feature views
A stream feature view is an extension of a normal feature view. The primary difference is that stream feature views have both stream and batch data sources, whereas a normal feature view only has a batch data source.
Stream feature views should be used instead of normal feature views when there are stream data sources (e.g. Kafka and Kinesis) available to provide fresh features in an online setting. Like regular feature views, stream feature views support the optional org field for grouping by organizational unit. Here is an example definition of a stream feature view with an attached transformation:
See here for a example of how to use stream feature views to register your own streaming data pipelines in Feast.
from feast import Field
from feast.types import Float32
trips_today = Field(
name="trips_today",
dtype=Float32
)
# Pin to a specific version (reverts the active definition to v2's snapshot)
driver_stats = FeatureView(
name="driver_stats",
entities=[driver],
schema=[...],
source=my_source,
version="v2",
)
from feast import Field, RequestSource
from feast.on_demand_feature_view import on_demand_feature_view
from feast.types import Float64
# Define a request data source which encodes features / information only
# available at request time (e.g. part of the user initiated HTTP request)
input_request = RequestSource(
name="vals_to_add",
schema=[
Field(name="val_to_add", dtype=PrimitiveFeastType.INT64),
Field(name="val_to_add_2": dtype=PrimitiveFeastType.INT64),
]
)
# Use the input data and feature view features to create new features
@on_demand_feature_view(
sources=[
driver_hourly_stats_view,
input_request
],
schema=[
Field(name='conv_rate_plus_val1', dtype=Float64),
Field(name='conv_rate_plus_val2', dtype=Float64)
]
)
def transformed_conv_rate(features_df: pd.DataFrame) -> pd.DataFrame:
df = pd.DataFrame()
df['conv_rate_plus_val1'] = (features_df['conv_rate'] + features_df['val_to_add'])
df['conv_rate_plus_val2'] = (features_df['conv_rate'] + features_df['val_to_add_2'])
return df
Offline feature retrieval for batch scoring (via feature_store.get_historical_features(...))
Online feature retrieval for real-time model predictions
via the SDK: feature_store.get_online_features(...)
via deployed feature server endpoints: requests.post('http://localhost:6566/get-online-features', data=json.dumps(online_request))
Each of these retrieval mechanisms accept:
some way of specifying entities (to fetch features for)
some way to specify the features to fetch (either via feature services, which group features needed for a model version, or feature references)
Before beginning, you need to instantiate a local FeatureStore object that knows how to parse the registry (see more details)
For code examples of how the below work, inspect the generated repository from feast init -t [YOUR TEMPLATE] (gcp, snowflake, and aws are the most fully fleshed).
Concepts
Before diving into how to retrieve features, we need to understand some high level concepts in Feast.
Feature Services
A feature service is an object that represents a logical group of features from one or more feature views. Feature Services allows features from within a feature view to be used as needed by an ML model. Users can expect to create one feature service per model version, allowing for tracking of the features used by models.
from driver_ratings_feature_view import driver_ratings_fv
from driver_trips_feature_view import driver_stats_fv
driver_stats_fs = FeatureService(
name="driver_activity",
features=[driver_stats_fv, driver_ratings_fv[["lifetime_rating"]]]
)
Feature services are used during
The generation of training datasets when querying feature views in order to find historical feature values. A single training dataset may consist of features from multiple feature views.
Retrieval of features for batch scoring from the offline store (e.g. with an entity dataframe where all timestamps are now())
Retrieval of features from the online store for online inference (with smaller batch sizes). The features retrieved from the online store may also belong to multiple feature views.
Applying a feature service does not result in an actual service being deployed.
Feature services enable referencing all or some features from a feature view.
Retrieving from the online store with a feature service
Retrieving from the offline store with a feature service
Feature References
This mechanism of retrieving features is only recommended as you're experimenting. Once you want to launch experiments or serve models, feature services are recommended.
Feature references uniquely identify feature values in Feast. The structure of a feature reference in string form is as follows: <feature_view>[@version]:<feature>
The @version part is optional. When omitted, the latest (active) version is used. You can specify a version like @v2 to read from a specific historical version snapshot.
Feature references are used for the retrieval of features from Feast:
Version-qualified reads (@v<N>) require enable_online_feature_view_versioning: true in your registry config and are currently supported only on the SQLite online store. See the feature view versioning docs for details.
It is possible to retrieve features from multiple feature views with a single request, and Feast is able to join features from multiple tables in order to build a training dataset. However, it is not possible to reference (or retrieve) features from multiple projects at the same time.
Note, if you're using Feature views without entities, then those features can be added here without additional entity values in the entity_rows parameter.
Event timestamp
The timestamp on which an event occurred, as found in a feature view's data source. The event timestamp describes the event time at which a feature was observed or generated.
Event timestamps are used during point-in-time joins to ensure that the latest feature values are joined from feature views onto entity rows. Event timestamps are also used to ensure that old feature values aren't served to models during online serving.
Why event_timestamp is required in the entity dataframe
When calling get_historical_features(), the entity_df must include an event_timestamp column. This timestamp acts as the upper bound (inclusive) for which feature values are allowed to be retrieved for each entity row. Feast performs a point-in-time join (also called a "last known good value" temporal join): for each entity row, it retrieves the latest feature values with a timestamp at or before the entity row's event_timestamp.
This ensures point-in-time correctness, which is critical to prevent data leakage during model training. Without this constraint, features generated after the prediction time could leak into training data—effectively letting the model "see the future"—leading to inflated offline metrics that do not translate to real-world performance.
For example, if you want to predict whether a driver will be rated well on April 12 at 10:00 AM, the entity dataframe row should have event_timestamp = datetime(2021, 4, 12, 10, 0, 0). Feast will then only join feature values observed on or before that time, excluding any data generated after 10:00 AM.
Retrieving features without an entity dataframe
While the entity dataframe is the standard way to retrieve historical features, Feast also supports entity-less historical feature retrieval by datetime range. This is useful when:
You are training time-series or population-level models and don't have a pre-defined list of entity IDs.
You want all features in a time window for exploratory analysis or batch training on full history.
Constructing an entity dataframe upfront is unnecessarily complex or expensive.
Instead of passing entity_df, you specify a time window with start_date and/or end_date:
If start_date is omitted, it defaults to end_date minus the feature view TTL. If end_date is omitted, it defaults to the current time. Point-in-time correctness is still preserved.
Entity-less retrieval is currently supported for the Postgres, Dask, Spark, and Ray offline stores. You cannot mix entity_df with start_date/end_date in the same call.
A dataset is a collection of rows that is produced by a historical retrieval from Feast in order to train a model. A dataset is produced by a join from one or more feature views onto an entity dataframe. Therefore, a dataset may consist of features from multiple feature views.
Dataset vs Feature View: Feature views contain the schema of data and a reference to where data can be found (through its data source). Datasets are the actual data manifestation of querying those data sources.
Dataset vs Data Source: Datasets are the output of historical retrieval, whereas data sources are the inputs. One or more data sources can be used in the creation of a dataset.
Retrieving historical features (for training data or batch scoring)
Feast abstracts away point-in-time join complexities with the get_historical_features API.
We go through the major steps, and also show example code. Note that the quickstart templates generally have end-to-end working examples for all these cases.
Full example: retrieve offline features for batch scoring
The main difference here compared to training data generation is how to handle timestamps in the entity dataframe. You want to pass in the current time to get the latest feature values for all your entities.
from feast import FeatureStore
store = FeatureStore(repo_path=".")
# Get the latest feature values for unique entities
entity_sql = f"""
SELECT
driver_id,
CURRENT_TIMESTAMP() as event_timestamp
FROM {store.get_data_source("driver_hourly_stats_source").get_table_query_string()}
WHERE event_timestamp BETWEEN '2021-01-01' and '2021-12-31'
GROUP BY driver_id
"""
batch_scoring_features = store.get_historical_features(
entity_df=entity_sql,
features=store.get_feature_service("model_v2"),
).to_df()
# predictions = model.predict(batch_scoring_features)
Step 1: Specifying Features
Feast accepts either:
feature services, which group features needed for a model version
Feast-native wrapper with engine metadata; preferred for Feast-internal tooling
Persistence methods
Method
Effect
When to use
.persist(storage)
Writes result to offline storage
Save a training dataset for later reuse or auditing
.to_remote_storage()
Exports result to S3/GCS as Parquet files
Hand off to external systems or data pipelines
Retrieving as a Ray Dataset
to_ray_dataset() is a first-class method on every RetrievalJob. When the underlying offline store is a RayOfflineStore, the dataset is returned directly without a copy through Arrow. For all other offline stores, a zero-copy Arrow → Ray Dataset conversion is used as a fallback.
Note:to_ray_dataset() requires feast[ray] to be installed.
Retrieving online features (for model inference)
Feast will ensure the latest feature values for registered features are available. At retrieval time, you need to supply a list of entities and the corresponding features to be retrieved. Similar to get_historical_features, we recommend using feature services as a mechanism for grouping features in a model version.
Note: unlike get_historical_features, the entity_rowsdo not need timestamps since you only want one feature value per entity key.
There are several options for retrieving online features: through the SDK, or through a feature server
Full example: retrieve online features for real-time model inference (Python SDK)
from feast import RepoConfig, FeatureStore
from feast.repo_config import RegistryConfig
repo_config = RepoConfig(
registry=RegistryConfig(path="gs://feast-test-gcs-bucket/registry.pb"),
project="feast_demo_gcp",
provider="gcp",
)
store = FeatureStore(config=repo_config)
features = store.get_online_features(
features=[
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_daily_features:daily_miles_driven",
],
entity_rows=[
{
"driver_id": 1001,
}
],
).to_dict()
Full example: retrieve online features for real-time model inference (Feature Server)
entity_sql = f"""
SELECT
driver_id,
event_timestamp
FROM {store.get_data_source("driver_hourly_stats_source").get_table_query_string()}
WHERE event_timestamp BETWEEN '2021-01-01' and '2021-12-31'
"""
training_df = store.get_historical_features(
entity_df=entity_sql,
features=[
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_daily_features:daily_miles_driven"
],
).to_df()
from feast import FeatureStore
store = FeatureStore(".")
# to_ray_dataset() is a first-class method on the RetrievalJob — chain it
# directly after get_historical_features().
ray_ds = store.get_historical_features(
entity_df=entity_df,
features=["driver_hourly_stats:conv_rate", "driver_hourly_stats:acc_rate"],
).to_ray_dataset()
# Use with Ray Train
import ray.train
trainer = ray.train.torch.TorchTrainer(
train_loop_per_worker=...,
datasets={"train": ray_ds},
)
Feast-Powered AI Agent
This example demonstrates an AI agent with persistent memory that uses Feast as both a feature store and a context memory layer through the Model Context Protocol (MCP). This demo uses Milvus as the vector-capable online store, but Feast supports multiple vector backends -- including Milvus, Elasticsearch, Qdrant, PGVector, and FAISS -- swappable via configuration.
Why Feast for Agents?
Agents need more than just access to data -- they need to remember what happened in prior interactions. Feast's online store is entity-keyed, low-latency, governed, and supports both reads and writes, making it a natural fit for agent context and memory.
Capability
How Feast Provides It
Architecture
Tools (backed by Feast)
The agent has four tools. Feast is both the read path (context) and the write path (memory):
Tool
Direction
What it does
When the LLM calls it
Memory is auto-saved after each agent turn (not as an LLM tool call). This follows the same pattern used by production frameworks -- see below.
Feast as Context Memory
The agent_memory feature view stores per-customer interaction state:
This gives agents persistent, governed, entity-keyed memory that survives across sessions, is versioned, and lives under the same RBAC as every other feature -- unlike an ad-hoc Redis cache or an in-process dict.
Memory as Infrastructure
Production agent frameworks treat memory as infrastructure, not an LLM decision. The framework auto-saves state after each step - the LLM never needs to "decide" to persist:
Framework
Memory mechanism
How it works
This demo follows the same pattern: the agent's three read tools (recall_memory, lookup_customer, search_knowledge_base) are exposed to the LLM for reasoning, while memory persistence is handled by the framework after each turn via _auto_save_memory. This ensures consistent, reliable memory regardless of LLM behaviour - no risk of the LLM forgetting to save, double-saving, or writing inconsistent state.
Feast is a natural fit for this checkpoint layer because it already provides:
Entity-keyed storage: memory is keyed by customer ID (or any entity)
TTL management: memory auto-expires via declarative feature view TTL
OpenAI API key (for live tool-calling; demo mode works without it)
Quickstart
One command
The script installs dependencies, generates sample data, starts the Feast server, runs the agent, and cleans up on exit.
Step by step
1. Install dependencies
2. Generate sample data and apply the registry
This creates:
3 customer profiles with attributes like plan tier, spend, and satisfaction score
6 knowledge-base articles with 384-dimensional vector embeddings
Empty agent memory scaffold (populated as the agent runs)
3. Start the Feast MCP Feature Server
4. Run the agent
In a new terminal:
To run with a real LLM, set the API key and (optionally) the base URL:
Demo mode output
Without an API key, the agent simulates the decision-making process with memory:
Scene 4 demonstrates memory continuity -- the agent recalls the SSO conversation from Scene 1 without the customer re-explaining.
Live mode output (with API key)
With an API key, the LLM autonomously decides which tools to use:
How It Works
Why a raw loop? This example builds the agent from scratch using the OpenAI tool-calling API and the MCP Python SDK to keep dependencies minimal and make every Feast call visible. All Feast interactions go through the MCP protocol -- the agent connects to Feast's MCP endpoint, discovers tools dynamically, and invokes them via session.call_tool(). In production, you would use a framework like LangChain/LangGraph, LlamaIndex, CrewAI, or AutoGen -- Feast's MCP endpoint lets any of them auto-discover the tools with zero custom wiring (see below).
The Agent Loop (agent.py)
The LLM sees the tool definitions (JSON Schema) and decides:
Which tools to call (can call zero, one, or multiple per round)
What arguments to pass (e.g., which customer ID to look up)
When to stop (once it has enough information to answer)
All Feast calls go through MCP (session.call_tool()), not direct REST. Memory is saved automatically after each turn by the framework, not by the LLM. This mirrors how production frameworks handle persistence (see ).
Feature Definitions (feature_repo/features.py)
customer_profile: Structured data (name, plan, spend, tickets, satisfaction)
knowledge_base: Support articles with 384-dim vector embeddings (Milvus in this demo; swappable to Elasticsearch, Qdrant, PGVector, or FAISS)
MCP Integration
The Feast Feature Server exposes all endpoints as MCP tools at http://localhost:6566/mcp. Any MCP-compatible framework can connect:
Building the same agent with a framework: The examples above show the Feast-specific part -- connecting to the MCP endpoint and getting the tools. Once you have the tools, building the agent follows each framework's standard patterns. The key difference from this demo's raw loop: frameworks handle the tool-calling loop, message threading, and (with LangGraph checkpointers or CrewAI memory=True) automatic state persistence natively. Feast's MCP endpoint means zero custom integration code -- the tools are discovered and callable immediately.
Adapting to your use case: The demo's system prompt, tool wrappers (lookup_customer, recall_memory), and feature views are all specific to customer support. For your own agent, you define your feature views in Feast (e.g., product_catalog, order_history, fraud_signals), run feast apply, and start the server. The same three generic MCP tools -- get_online_features, retrieve_online_documents, and write_to_online_store -- serve any domain. With a framework like LangChain or LlamaIndex, you don't even need custom tool wrappers -- the LLM calls the generic Feast tools directly with your feature view names and entities.
Production Deployment
For production, Feast fits into a layered platform architecture:
This demo uses Milvus Lite (embedded). For production, swap to any supported vector-capable backend by updating feature_store.yaml:
Milvus cluster: Deploy via the and set host/port instead of path.
Elasticsearch: Set online_store: type: elasticsearch with your cluster URL.
=================================================================
Scene 1: Enterprise customer (C1001) asks about SSO
Customer: C1001 | Query: "How do I set up SSO for my team?"
=================================================================
[Demo mode] Simulating agent reasoning
Round 1 | recall_memory(customer_id=C1001)
-> No prior interactions found
Round 1 | lookup_customer(customer_id=C1001)
-> Alice Johnson | enterprise plan | $24,500 spend | 1 open tickets
Round 1 | search_knowledge_base(query="How do I set up SSO for my team?...")
-> Best match: "Configuring single sign-on (SSO)"
Round 2 | Generating personalised response...
─────────────────────────────────────────────────────────────
Agent Response:
─────────────────────────────────────────────────────────────
Hi Alice!
Since you're on our Enterprise plan, SSO is available for your
team. Go to Settings > Security > SSO and enter your Identity
Provider metadata URL. We support SAML 2.0 and OIDC...
[Checkpoint] Memory saved: topic="SSO setup"
=================================================================
Scene 4: C1001 returns -- does the agent remember Scene 1?
Customer: C1001 | Query: "I'm back about my SSO question from earlier."
=================================================================
[Demo mode] Simulating agent reasoning
Round 1 | recall_memory(customer_id=C1001)
-> Previous topic: SSO setup
-> Open issue: none
-> Interaction count: 1
Round 1 | lookup_customer(customer_id=C1001)
-> Alice Johnson | enterprise plan | $24,500 spend | 1 open tickets
Round 2 | Generating personalised response...
─────────────────────────────────────────────────────────────
Agent Response:
─────────────────────────────────────────────────────────────
Welcome back, Alice! I can see from our records that we last
discussed "SSO setup". How can I help you today?
[Checkpoint] Memory saved: topic="SSO setup"
=================================================================
Scene 1: Enterprise customer (C1001) asks about SSO
Customer: C1001 | Query: "How do I set up SSO for my team?"
=================================================================
[Round 1] Tool call: recall_memory({'customer_id': 'C1001'})
[Round 1] Tool call: lookup_customer({'customer_id': 'C1001'})
[Round 1] Tool call: search_knowledge_base({'query': 'SSO setup'})
Agent finished after 2 round(s)
─────────────────────────────────────────────────────────────
Agent Response:
─────────────────────────────────────────────────────────────
Hi Alice! Since you're on our Enterprise plan, SSO is available
for your team. Go to Settings > Security > SSO and enter your
Identity Provider metadata URL. We support SAML 2.0 and OIDC...
[Checkpoint] Memory saved: topic="SSO setup"
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
async with streamablehttp_client("http://localhost:6566/mcp") as (r, w, _):
async with ClientSession(r, w) as session:
await session.initialize()
tools = await session.list_tools() # discover Feast tools
for round in range(MAX_ROUNDS):
# 1. Send messages + read tools to LLM
response = call_llm(messages, tools=[...])
# 2. If LLM says "stop" -> return the answer
if response.finish_reason == "stop":
break
# 3. Execute tool calls via MCP
for tool_call in response.tool_calls:
result = await session.call_tool(name, args)
messages.append(tool_result(result))
# 4. Framework-style checkpoint: auto-save via MCP
await session.call_tool("write_to_online_store", {...})
# LangChain / LangGraph
from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.prebuilt import create_react_agent
async with MultiServerMCPClient(
{"feast": {"url": "http://localhost:6566/mcp", "transport": "streamable_http"}}
) as client:
tools = client.get_tools()
agent = create_react_agent(llm, tools)
result = await agent.ainvoke({"messages": "How do I set up SSO?"})
# LlamaIndex
from llama_index.tools.mcp import aget_tools_from_mcp_url
from llama_index.core.agent.function_calling import FunctionCallingAgent
from llama_index.llms.openai import OpenAI
tools = await aget_tools_from_mcp_url("http://localhost:6566/mcp")
agent = FunctionCallingAgent.from_tools(tools, llm=OpenAI(model="gpt-4o-mini"))
response = await agent.achat("How do I set up SSO?")
how to extend the test suite to test new functionality
how to use the existing test suite to test a new custom offline / online store
Test suite overview
Unit tests are contained in sdk/python/tests/unit. Integration tests are contained in sdk/python/tests/integration. Let's inspect the structure of sdk/python/tests/integration:
feature_repos has setup files for most tests in the test suite.
conftest.py (in the parent directory) contains the most common fixtures, which are designed as an abstraction on top of specific offline/online stores, so tests do not need to be rewritten for different stores. Individual test files also contain more specific fixtures.
The tests are organized by which Feast component(s) they test.
Structure of the test suite
Universal feature repo
The universal feature repo refers to a set of fixtures (e.g. environment and universal_data_sources) that can be parametrized to cover various combinations of offline stores, online stores, and providers. This allows tests to run against all these various combinations without requiring excess code. The universal feature repo is constructed by fixtures in conftest.py with help from the various files in feature_repos.
Integration vs. unit tests
Tests in Feast are split into integration and unit tests. If a test requires external resources (e.g. cloud resources on GCP or AWS), it is an integration test. If a test can be run purely locally (where locally includes Docker resources), it is a unit test.
Integration tests test non-local Feast behavior. For example, tests that require reading data from BigQuery or materializing data to DynamoDB are integration tests. Integration tests also tend to involve more complex Feast functionality.
Unit tests test local Feast behavior. For example, tests that only require registering feature views are unit tests. Unit tests tend to only involve simple Feast functionality.
Main types of tests
Integration tests
E2E tests
E2E tests test end-to-end functionality of Feast over the various codepaths (initialize a feature store, apply, and materialize).
The main codepaths include:
basic e2e tests for offline stores
test_universal_e2e.py
go feature server
test_go_feature_server.py
python http server
test_python_feature_server.py
data quality monitoring feature validation
test_validation.py
Offline and Online Store Tests
Offline and online store tests mainly test for the offline and online retrieval functionality.
The various specific functionalities that are tested include:
Registration Tests
The registration folder contains all of the registry tests and some universal cli tests. This includes:
CLI Apply and Materialize tests tested against on the universal test suite
Miscellaneous Tests
AWS Lambda Materialization Tests (Currently do not work)
test_lambda.py
Unit tests
Registry Diff Tests
These are tests for the infrastructure and registry diff functionality that Feast uses to determine if changes to the registry or infrastructure is needed.
Local CLI Tests and Local Feast Tests
These tests test all of the cli commands against the local file offline store.
Infrastructure Unit Tests
DynamoDB tests with dynamo mocked out
Repository configuration tests
Feature Store Validation Tests
These test mainly contain class level validation like hashing tests, protobuf and class serialization, and error and warning handling.
Data source unit tests
Docstring tests
Docstring tests are primarily smoke tests to make sure imports and setup functions can be executed without errors.
Understanding the test suite with an example test
Example test
Let's look at a sample test using the universal repo:
@pytest.mark.integration
@pytest.mark.universal_offline_stores
@pytest.mark.parametrize("full_feature_names", [True, False], ids=lambda v: f"full:{v}")
def test_historical_features(environment, universal_data_sources, full_feature_names):
store = environment.feature_store
(entities, datasets, data_sources) = universal_data_sources
feature_views = construct_universal_feature_views(data_sources)
entity_df_with_request_data = datasets.entity_df.copy(deep=True)
entity_df_with_request_data["val_to_add"] = [
i for i in range(len(entity_df_with_request_data))
]
entity_df_with_request_data["driver_age"] = [
i + 100 for i in range(len(entity_df_with_request_data))
]
feature_service = FeatureService(
name="convrate_plus100",
features=[feature_views.driver[["conv_rate"]], feature_views.driver_odfv],
)
feature_service_entity_mapping = FeatureService(
name="entity_mapping",
features=[
feature_views.location.with_name("origin").with_join_key_map(
{"location_id": "origin_id"}
),
feature_views.location.with_name("destination").with_join_key_map(
{"location_id": "destination_id"}
),
],
)
store.apply(
[
driver(),
customer(),
location(),
feature_service,
feature_service_entity_mapping,
*feature_views.values(),
]
)
# ... more test code
job_from_df = store.get_historical_features(
entity_df=entity_df_with_request_data,
features=[
"driver_stats:conv_rate",
"driver_stats:avg_daily_trips",
"customer_profile:current_balance",
"customer_profile:avg_passenger_count",
"customer_profile:lifetime_trip_count",
"conv_rate_plus_100:conv_rate_plus_100",
"conv_rate_plus_100:conv_rate_plus_100_rounded",
"conv_rate_plus_100:conv_rate_plus_val_to_add",
"order:order_is_success",
"global_stats:num_rides",
"global_stats:avg_ride_length",
"field_mapping:feature_name",
],
full_feature_names=full_feature_names,
)
if job_from_df.supports_remote_storage_export():
files = job_from_df.to_remote_storage()
print(files)
assert len(files) > 0 # This test should be way more detailed
start_time = datetime.utcnow()
actual_df_from_df_entities = job_from_df.to_df()
# ... more test code
validate_dataframes(
expected_df,
table_from_df_entities,
sort_by=[event_timestamp, "order_id", "driver_id", "customer_id"],
event_timestamp = event_timestamp,
)
# ... more test code
The key fixtures are the environment and universal_data_sources fixtures, which are defined in the feature_repos directories and the conftest.py file. This by default pulls in a standard dataset with driver and customer entities (that we have pre-defined), certain feature views, and feature values.
The environment fixture sets up a feature store, parametrized by the provider and the online/offline store. It allows the test to query against that feature store without needing to worry about the underlying implementation or any setup that may be involved in creating instances of these datastores.
Each fixture creates a different integration test with its own IntegrationTestRepoConfig which is used by pytest to generate a unique test testing one of the different environments that require testing.
Feast tests also use a variety of markers:
The @pytest.mark.integration marker is used to designate integration tests which will cause the test to be run when you call make test-python-integration.
The @pytest.mark.universal_offline_stores
Writing a new test or reusing existing tests
To add a new test to an existing test file
Use the same function signatures as an existing test (e.g. use environment and universal_data_sources as an argument) to include the relevant test fixtures.
If possible, expand an individual test instead of writing a new test, due to the cost of starting up offline / online stores.
Use the universal_offline_stores and universal_online_store markers to parametrize the test against different offline store and online store combinations. You can also designate specific online and offline stores to test by using the only parameter on the marker.
To test a new offline / online store from a plugin repo
Install Feast in editable mode with pip install -e.
The core tests for offline / online store behavior are parametrized by the FULL_REPO_CONFIGS variable defined in feature_repos/repo_configuration.py. To overwrite this variable without modifying the Feast repo, create your own file that contains a FULL_REPO_CONFIGS (which will require adding a new IntegrationTestRepoConfig or two) and set the environment variable FULL_REPO_CONFIGS_MODULE to point to that file. Then the core offline / online store tests can be run with make test-python-universal.
See the and the for examples.
What are some important things to keep in mind when adding a new offline / online store?
Type mapping/Inference
Many problems arise when implementing your data store's type conversion to interface with Feast datatypes.
You will need to correctly update inference.py so that Feast can infer your datasource schemas
You also need to update type_map.py so that Feast knows how to convert your datastores types to Feast-recognized types in feast/types.py.
Historical and online retrieval
The most important functionality in Feast is historical and online retrieval. Most of the e2e and universal integration test test this functionality in some way. Making sure this functionality works also indirectly asserts that reading and writing from your datastore works as intended.
To include a new offline / online store in the main Feast repo
Extend data_source_creator.py for your offline store.
In repo_configuration.py add a new IntegrationTestRepoConfig or two (depending on how many online stores you want to test).
Generally, you should only need to test against sqlite. However, if you need to test against a production online store, then you can also test against Redis or dynamodb.
Run the full test suite with make test-python-integration.
Including a new offline / online store in the main Feast repo from external plugins with community maintainers.
This folder is for plugins that are officially maintained with community owners. Place the APIs in feast/infra/offline_stores/contrib/.
Extend data_source_creator.py for your offline store and implement the required APIs.
In contrib_repo_configuration.py add a new IntegrationTestRepoConfig (depending on how many online stores you want to test).
Run the test suite on the contrib test suite with make test-python-contrib-universal.
To include a new online store
In repo_configuration.py add a new config that maps to a serialized version of configuration you need in feature_store.yaml to setup the online store.
In repo_configuration.py, add new IntegrationTestRepoConfig for online stores you want to test.
Run the full test suite with make test-python-integration
To use custom data in a new test
Check test_universal_types.py for an example of how to do this.
Running your own Redis cluster for testing
Install Redis on your computer. If you are a mac user, you should be able to brew install redis.
Running redis-server --help and redis-cli --help should show corresponding help menus.
Run ./infra/scripts/redis-cluster.sh start then ./infra/scripts/redis-cluster.sh create to start the Redis cluster locally. You should see output that looks like this:
You should be able to run the integration tests and have the Redis cluster tests pass.
If you would like to run your own Redis cluster, you can run the above commands with your own specified ports and connect to the newly configured cluster.
To stop the cluster, run ./infra/scripts/redis-cluster.sh stop and then ./infra/scripts/redis-cluster.sh clean.
Feature service, feature view, and feature validation tests
Protobuf/json tests for Feast ValueTypes
Serialization tests
Type mapping
Feast types
Serialization tests due to this
marker will parametrize the test on all of the universal offline stores including file, redshift, bigquery and snowflake.
The full_feature_names parametrization defines whether or not the test should reference features as their full feature name (fully qualified path) or just the feature name itself.
Feast makes adding support for a new offline store easy. Developers can simply implement the interface to add support for a new store (other than the existing stores like Parquet files, Redshift, and Bigquery).
In this guide, we will show you how to extend the existing File offline store and use in a feature repo. While we will be implementing a specific store, this guide should be representative for adding support for any new offline store.
The full working code for this guide can be found at .
The process for using a custom offline store consists of 8 steps:
Defining an OfflineStore class.
Defining an OfflineStoreConfig class.
Defining a RetrievalJob class for this offline store.
Defining a DataSource class for the offline store
Referencing the OfflineStore in a feature repo's feature_store.yaml file.
Testing the OfflineStore class.
Updating dependencies.
Adding documentation.
1. Defining an OfflineStore class
OfflineStore class names must end with the OfflineStore suffix!
Contrib offline stores
New offline stores go in sdk/python/feast/infra/offline_stores/contrib/.
What is a contrib plugin?
Not guaranteed to implement all interface methods
Not guaranteed to be stable.
Should have warnings for users to indicate this is a contrib plugin that is not maintained by the maintainers.
How do I make a contrib plugin an "official" plugin?
To move an offline store plugin out of contrib, you need:
GitHub actions (i.e make test-python-integration) is setup to run all tests against the offline store and pass.
At least two contributors own the plugin (ideally tracked in our OWNERS / CODEOWNERS file).
Define the offline store class
The OfflineStore class contains a couple of methods to read features from the offline store. Unlike the OnlineStore class, Feast does not manage any infrastructure for the offline store.
To fully implement the interface for the offline store, you will need to implement these methods:
pull_latest_from_table_or_query is invoked when running materialization (using the feast materialize or feast materialize-incremental commands, or the corresponding FeatureStore.materialize() method. This method pull data from the offline store, and the FeatureStore class takes care of writing this data into the online store.
get_historical_features is invoked when reading values from the offline store using the FeatureStore.get_historical_features() method. Typically, this method is used to retrieve features when training ML models.
(optional) offline_write_batch is a method that supports directly pushing a pyarrow table to a feature view. Given a feature view with a specific schema, this function should write the pyarrow table to the batch source defined. More details about the push api can be found . This method only needs implementation if you want to support the push api in your offline store.
(optional) pull_all_from_table_or_query is a method that pulls all the data from an offline store from a specified start date to a specified end date. This method is only used for SavedDatasets as part of data quality monitoring validation.
(optional) write_logged_features is a method that takes a pyarrow table or a path that points to a parquet file and writes the data to a defined source defined by LoggingSource and LoggingConfig. This method is only used internally for SavedDatasets.
1.1 Type Mapping
Most offline stores will have to perform some custom mapping of offline store datatypes to feast value types.
The function to implement here are source_datatype_to_feast_value_type and get_column_names_and_types in your DataSource class.
source_datatype_to_feast_value_type is used to convert your DataSource's datatypes to feast value types.
get_column_names_and_types retrieves the column names and corresponding datasource types.
Add any helper functions for type conversion to sdk/python/feast/type_map.py.
Be sure to implement correct type mapping so that Feast can process your feature columns without casting incorrectly that can potentially cause loss of information or incorrect data.
2. Defining an OfflineStoreConfig class
Additional configuration may be needed to allow the OfflineStore to talk to the backing store. For example, Redshift needs configuration information like the connection information for the Redshift instance, credentials for connecting to the database, etc.
To facilitate configuration, all OfflineStore implementations are required to also define a corresponding OfflineStoreConfig class in the same file. This OfflineStoreConfig class should inherit from the FeastConfigBaseModel class, which is defined here.
The FeastConfigBaseModel is a pydantic class, which parses yaml configuration into python objects. Pydantic also allows the model classes to define validators for the config classes, to make sure that the config classes are correctly defined.
This config class must container a type field, which contains the fully qualified class name of its corresponding OfflineStore class.
Additionally, the name of the config class must be the same as the OfflineStore class, with the Config suffix.
An example of the config class for the custom file offline store :
This configuration can be specified in the feature_store.yaml as follows:
This configuration information is available to the methods of the OfflineStore, via the config: RepoConfig parameter which is passed into the methods of the OfflineStore interface, specifically at the config.offline_store field of the config parameter. This fields in the feature_store.yaml should map directly to your OfflineStoreConfig class that is detailed above in Section 2.
3. Defining a RetrievalJob class
The offline store methods aren't expected to perform their read operations eagerly. Instead, they are expected to execute lazily, and they do so by returning a RetrievalJob instance, which represents the execution of the actual query against the underlying store.
Custom offline stores may need to implement their own instances of the RetrievalJob interface.
The RetrievalJob interface exposes two methods - to_df and to_arrow. The expectation is for the retrieval job to be able to return the rows read from the offline store as a parquet DataFrame, or as an Arrow table respectively.
Users who want to have their offline store support scalable batch materialization for online use cases (detailed in this RFC) will also need to implement to_remote_storage to distribute the reading and writing of offline store records to blob storage (such as S3). This may be used by a custom Materialization Engine to parallelize the materialization of data by processing it in chunks. If this is not implemented, Feast will default to local materialization (pulling all records into memory to materialize).
4. Defining a DataSource class for the offline store
Before this offline store can be used as the batch source for a feature view in a feature repo, a subclass of the DataSourcebase class needs to be defined. This class is responsible for holding information needed by specific feature views to support reading historical values from the offline store. For example, a feature view using Redshift as the offline store may need to know which table contains historical feature values.
The data source class should implement two methods - from_proto, and to_proto.
For custom offline stores that are not being implemented in the main feature repo, the custom_options field should be used to store any configuration needed by the data source. In this case, the implementer is responsible for serializing this configuration into bytes in the to_proto method and reading the value back from bytes in the from_proto method.
5. Using the custom offline store
After implementing these classes, the custom offline store can be used by referencing it in a feature repo's feature_store.yaml file, specifically in the offline_store field. The value specified should be the fully qualified class name of the OfflineStore.
As long as your OfflineStore class is available in your Python environment, it will be imported by Feast dynamically at runtime.
To use our custom file offline store, we can use the following feature_store.yaml:
If additional configuration for the offline store is not required, then we can omit the other fields and only specify the type of the offline store class as the value for the offline_store.
Finally, the custom data source class can be use in the feature repo to define a data source, and refer to in a feature view definition.
6. Testing the OfflineStore class
Integrating with the integration test suite and unit test suite.
Even if you have created the OfflineStore class in a separate repo, you can still test your implementation against the Feast test suite, as long as you have Feast as a submodule in your repo.
In order to test against the test suite, you need to create a custom DataSourceCreator that implement our testing infrastructure methods, create_data_source and optionally, created_saved_dataset_destination.
create_data_source should create a datasource based on the dataframe passed in. It may be implemented by uploading the contents of the dataframe into the offline store and returning a datasource object pointing to that location. See BigQueryDataSourceCreator for an implementation of a data source creator.
created_saved_dataset_destination is invoked when users need to save the dataset for use in data validation. This functionality is still in alpha and is optional.
Make sure that your offline store doesn't break any unit tests first by running:
Next, set up your offline store to run the universal integration tests. These are integration tests specifically intended to test offline and online stores against Feast API functionality, to ensure that the Feast APIs works with your offline store.
Feast parametrizes integration tests using the FULL_REPO_CONFIGS variable defined in sdk/python/tests/universal/feature_repos/repo_configuration.py which stores different offline store classes for testing.
You should swap out the FULL_REPO_CONFIGS environment variable and run the integration tests against your offline store. In the example repo, the file that overwrites FULL_REPO_CONFIGS is feast_custom_offline_store/feast_tests.py, so you would run:
If the integration tests fail, this indicates that there is a mistake in the implementation of this offline store!
Remember to add your datasource to repo_config.py similar to how we added spark, trino, etc, to the dictionary OFFLINE_STORE_CLASS_FOR_TYPE. This will allow Feast to load your class from the feature_store.yaml.
Finally, add a Makefile target to the Makefile to run your datastore specific tests by setting the FULL_REPO_CONFIGS_MODULE and PYTEST_PLUGINS environment variable. The PYTEST_PLUGINS environment variable allows pytest to load in the DataSourceCreator for your datasource. You can remove certain tests that are not relevant or still do not work for your datastore using the -k option.
7. Dependencies
Add any dependencies for your offline store to pyproject.toml under [project.optional-dependencies] as a new extra (e.g. <offline_store> = ["package1>=1.0", "package2"]). These packages should be defined as extras so that they are not installed by users by default. You will need to regenerate our requirements lock files:
8. Add Documentation
Remember to add documentation for your offline store.
Add a new markdown file to docs/reference/offline-stores/ and docs/reference/data-sources/. Use these files to document your offline store functionality similar to how the other offline stores are documented.
You should also add a reference in docs/reference/data-sources/README.md and docs/SUMMARY.md to these markdown files.
NOTE: Be sure to document the following things about your offline store:
How to create the datasource and most what configuration is needed in the feature_store.yaml file in order to create the datasource.
Make sure to flag that the datasource is in alpha development.
Add some documentation on what the data model is for the specific offline store for more clarity.
Finally, generate the python code docs by running:
# Only prints out runtime warnings once.
warnings.simplefilter("once", RuntimeWarning)
def get_historical_features(self,
config: RepoConfig,
feature_views: List[FeatureView],
feature_refs: List[str],
entity_df: Union[pd.DataFrame, str],
registry: Registry, project: str,
full_feature_names: bool = False) -> RetrievalJob:
""" Perform point-in-time correct join of features onto an entity dataframe(entity key and timestamp). More details about how this should work at https://docs.feast.dev/v/v0.6-branch/user-guide/feature-retrieval#3.-historical-feature-retrieval.
print("Getting historical features from my offline store")."""
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
# Implementation here.
pass
def pull_latest_from_table_or_query(self,
config: RepoConfig,
data_source: DataSource,
join_key_columns: List[str],
feature_name_columns: List[str],
timestamp_field: str,
created_timestamp_column: Optional[str],
start_date: datetime,
end_date: datetime) -> RetrievalJob:
""" Pulls data from the offline store for use in materialization."""
print("Pulling latest features from my offline store")
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
# Implementation here.
pass
def pull_all_from_table_or_query(
config: RepoConfig,
data_source: DataSource,
join_key_columns: List[str],
feature_name_columns: List[str],
timestamp_field: str,
start_date: datetime,
end_date: datetime,
) -> RetrievalJob:
""" Optional method that returns a Retrieval Job for all join key columns, feature name columns, and the event timestamp columns that occur between the start_date and end_date."""
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
# Implementation here.
pass
def write_logged_features(
config: RepoConfig,
data: Union[pyarrow.Table, Path],
source: LoggingSource,
logging_config: LoggingConfig,
registry: BaseRegistry,
):
""" Optional method to have Feast support logging your online features."""
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
# Implementation here.
pass
def offline_write_batch(
config: RepoConfig,
feature_view: FeatureView,
table: pyarrow.Table,
progress: Optional[Callable[[int], Any]],
):
""" Optional method to have Feast support the offline push api for your offline store."""
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
# Implementation here.
pass
feast_custom_offline_store/file.py
class CustomFileOfflineStoreConfig(FeastConfigBaseModel):
""" Custom offline store config for local (file-based) store """
type: Literal["feast_custom_offline_store.file.CustomFileOfflineStore"] \
= "feast_custom_offline_store.file.CustomFileOfflineStore"
uri: str # URI for your offline store(in this case it would be a path)
def get_historical_features(self,
config: RepoConfig,
feature_views: List[FeatureView],
feature_refs: List[str],
entity_df: Union[pd.DataFrame, str],
registry: Registry, project: str,
full_feature_names: bool = False) -> RetrievalJob:
warnings.warn(
"This offline store is an experimental feature in alpha development. "
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
offline_store_config = config.offline_store
assert isinstance(offline_store_config, CustomFileOfflineStoreConfig)
store_type = offline_store_config.type
feast_custom_offline_store/file.py
class CustomFileRetrievalJob(RetrievalJob):
def __init__(self, evaluation_function: Callable):
"""Initialize a lazy historical retrieval job"""
# The evaluation function executes a stored procedure to compute a historical retrieval.
self.evaluation_function = evaluation_function
def to_df(self):
# Only execute the evaluation function to build the final historical retrieval dataframe at the last moment.
print("Getting a pandas DataFrame from a File is easy!")
df = self.evaluation_function()
return df
def to_arrow(self):
# Only execute the evaluation function to build the final historical retrieval dataframe at the last moment.
print("Getting a pandas DataFrame from a File is easy!")
df = self.evaluation_function()
return pyarrow.Table.from_pandas(df)
def to_remote_storage(self):
# Optional method to write to an offline storage location to support scalable batch materialization.
pass
feast_custom_offline_store/file.py
class CustomFileDataSource(FileSource):
"""Custom data source class for local files"""
def __init__(
self,
timestamp_field: Optional[str] = "",
path: Optional[str] = None,
field_mapping: Optional[Dict[str, str]] = None,
created_timestamp_column: Optional[str] = "",
date_partition_column: Optional[str] = "",
):
"Some functionality may still be unstable so functionality can change in the future.",
RuntimeWarning,
)
super(CustomFileDataSource, self).__init__(
timestamp_field=timestamp_field,
created_timestamp_column,
field_mapping,
date_partition_column,
)
self._path = path
@staticmethod
def from_proto(data_source: DataSourceProto):
custom_source_options = str(
data_source.custom_options.configuration, encoding="utf8"
)
path = json.loads(custom_source_options)["path"]
return CustomFileDataSource(
field_mapping=dict(data_source.field_mapping),
path=path,
timestamp_field=data_source.timestamp_field,
created_timestamp_column=data_source.created_timestamp_column,
date_partition_column=data_source.date_partition_column,
)
def to_proto(self) -> DataSourceProto:
config_json = json.dumps({"path": self.path})
data_source_proto = DataSourceProto(
type=DataSourceProto.CUSTOM_SOURCE,
custom_options=DataSourceProto.CustomSourceOptions(
configuration=bytes(config_json, encoding="utf8")
),
)
data_source_proto.timestamp_field = self.timestamp_field
data_source_proto.created_timestamp_column = self.created_timestamp_column
data_source_proto.date_partition_column = self.date_partition_column
return data_source_proto
feature_repo/feature_store.yaml
project: test_custom
registry: data/registry.db
provider: local
offline_store:
# Make sure to specify the type as the fully qualified path that Feast can import.
type: feast_custom_offline_store.file.CustomFileOfflineStore
feature_repo/feature_store.yaml
project: test_custom
registry: data/registry.db
provider: local
offline_store: feast_custom_offline_store.file.CustomFileOfflineStore
test-python-universal-spark:
PYTHONPATH='.' \
FULL_REPO_CONFIGS_MODULE=sdk.python.feast.infra.offline_stores.contrib.spark_repo_configuration \
PYTEST_PLUGINS=feast.infra.offline_stores.contrib.spark_offline_store.tests \
IS_TEST=True \
python -m pytest -n 8 --integration \
-k "not test_historical_retrieval_fails_on_validation and \
not test_historical_retrieval_with_validation and \
not test_historical_features_persisting and \
not test_historical_retrieval_fails_on_validation and \
not test_universal_cli and \
not test_go_feature_server and \
not test_feature_logging and \
not test_reorder_columns and \
not test_logged_features_validation and \
not test_lambda_materialization_consistency and \
not test_offline_write and \
not test_push_features_to_offline_store.py and \
not gcs_registry and \
not s3_registry and \
not test_universal_types" \
sdk/python/tests
make lock-python-dependencies-all
make build-sphinx
To overwrite the default configurations to use your own offline store, you can simply create your own file that contains a FULL_REPO_CONFIGS dictionary, and point Feast to that file by setting the environment variable FULL_REPO_CONFIGS_MODULE to point to that file. The module should add new IntegrationTestRepoConfig classes to the AVAILABLE_OFFLINE_STORES by defining an offline store that you would like Feast to test with.
A sample FULL_REPO_CONFIGS_MODULE looks something like this:
export FULL_REPO_CONFIGS_MODULE='feast_custom_offline_store.feast_tests'
make test-python-universal
# Should go in sdk/python/feast/infra/offline_stores/contrib/postgres_repo_configuration.py
from feast.infra.offline_stores.contrib.postgres_offline_store.tests.data_source import (
PostgreSQLDataSourceCreator,
)
AVAILABLE_OFFLINE_STORES = [("local", PostgreSQLDataSourceCreator)]
Quickstart
What is Feast?
Feast (Feature Store) is an open-source feature store designed to facilitate the management and serving of machine learning features in a way that supports both batch and real-time applications.
For Data Scientists: Feast is a a tool where you can easily define, store, and retrieve your features for both model development and model deployment. By using Feast, you can focus on what you do best: build features that power your AI/ML models and maximize the value of your data.
For MLOps Engineers: Feast is a library that allows you to connect your existing infrastructure (e.g., online database, application server, microservice, analytical database, and orchestration tooling) that enables your Data Scientists to ship features for their models to production using a friendly SDK without having to be concerned with software engineering challenges that occur from serving real-time production systems. By using Feast, you can focus on maintaining a resilient system, instead of implementing features for Data Scientists.
For Data Engineers: Feast provides a centralized catalog for storing feature definitions allowing one to maintain a single source of truth for feature data. It provides the abstraction for reading and writing to many different types of offline and online data stores. Using either the provided python SDK or the feature server service, users can write data to the online and/or offline stores and then read that data out again in either low-latency online scenarios for model inference, or in batch scenarios for model training.
For AI Engineers: Feast provides a platform designed to scale your AI applications by enabling seamless integration of richer data and facilitating fine-tuning. With Feast, you can optimize the performance of your AI models while ensuring a scalable and efficient data pipeline.
For more info refer to
Prerequisites
Ensure that you have Python (3.9 or above) installed.
It is recommended to create and work in a virtual environment:
Overview
In this tutorial we will:
Deploy a local feature store with a Parquet file offline store and Sqlite online store.
Build a training dataset using our time series features from our Parquet files.
Ingest batch features ("materialization") and streaming features (via a Push API) into the online store.
Note - Feast provides a python SDK as well as an optional for reading and writing feature data to the online and offline data stores. The latter might be useful when non-python languages are required.
For this tutorial, we will be using the python SDK.
In this tutorial, we'll use Feast to generate training data and power online model inference for a ride-sharing driver satisfaction prediction model. Feast solves several common issues in this flow:
Training-serving skew and complex data joins: Feature values often exist across multiple tables. Joining these datasets can be complicated, slow, and error-prone.
Feast joins these tables with battle-tested logic that ensures point-in-time correctness so future feature values do not leak to models.
Online feature availability: At inference time, models often need access to features that aren't readily available and need to be precomputed from other data sources.
Step 1: Install Feast
Install the Feast SDK and CLI using pip:
In this tutorial, we focus on a local deployment. For a more in-depth guide on how to use Feast with Snowflake / GCP / AWS deployments, see
Step 2: Create a feature repository
Bootstrap a new feature repository using feast init from the command line.
Let's take a look at the resulting demo repo itself. It breaks down into
The feature_store.yaml file configures the key overall architecture of the feature store.
The provider value sets default offline and online stores.
The offline store provides the compute layer to process historical data (for generating training data & feature values for serving).
The online store is a low latency store of the latest feature values (for powering real-time inference).
Valid values for provider in feature_store.yaml are:
local: use a SQL registry or local file registry. By default, use a file / Dask based offline store + SQLite online store
gcp: use a SQL registry or GCS file registry. By default, use BigQuery (offline store) + Google Cloud Datastore (online store)
aws: use a SQL registry or S3 file registry. By default, use Redshift (offline store) + DynamoDB (online store)
Note that there are many other offline / online stores Feast works with, including Spark, Azure, Hive, Trino, and PostgreSQL via community plugins. See for all supported data sources.
A custom setup can also be made by following .
Inspecting the raw data
The raw feature data we have in this demo is stored in a local parquet file. The dataset captures hourly stats of a driver in a ride-sharing app.
Step 3: Run sample workflow
There's an included test_workflow.py file which runs through a full sample workflow:
Register feature definitions through feast apply
Generate a training dataset (using get_historical_features)
Generate features for batch scoring (using get_historical_features
We'll walk through some snippets of code below and explain
Step 4: Register feature definitions and deploy your feature store
The apply command scans python files in the current directory for feature view/entity definitions, registers the objects, and deploys infrastructure. In this example, it reads feature_definitions.py and sets up SQLite online store tables. Note that we had specified SQLite as the default online store by configuring online_store in feature_store.yaml.
Step 5: Generating training data or powering batch scoring models
To train a model, we need features and labels. Often, this label data is stored separately (e.g. you have one table storing user survey results and another set of tables with feature values). Feast can help generate the features that map to these labels.
Feast needs a list of entities (e.g. driver ids) and timestamps. Feast will intelligently join relevant tables to create the relevant feature vectors. There are two ways to generate this list:
The user can query that table of labels with timestamps and pass that into Feast as an entity dataframe for training data generation.
The user can also query that table with a SQL query which pulls entities. See the documentation on for details
Note that we include timestamps because we want the features for the same driver at various timestamps to be used in a model.
Generating training data
Run offline inference (batch scoring)
To power a batch model, we primarily need to generate features with the get_historical_features call, but using the current timestamp
Step 6: Ingest batch features into your online store
We now serialize the latest values of features since the beginning of time to prepare for serving. Note, materialize_incremental serializes all new features since the last materialize call, or since the time provided minus the ttl timedelta. In this case, this will be CURRENT_TIME - 1 day (ttl was set on the FeatureView instances in feature_definitions.py).
Step 7: Fetching feature vectors for inference
At inference time, we need to quickly read the latest feature values for different drivers (which otherwise might have existed only in batch sources) from the online feature store using get_online_features(). These feature vectors can then be fed to the model.
Step 8: Using a feature service to fetch online features instead.
You can also use feature services to manage multiple features, and decouple feature view definitions and the features needed by end applications. The feature store can also be used to fetch either online or historical features using the same API below. More information can be found .
The driver_activity_v1 feature service pulls all features from the driver_hourly_stats feature view:
Step 9: Browse your features with the Web UI (experimental)
View all registered features, data sources, entities, and feature services with the Web UI.
One of the ways to view this is with the feast ui command.
Step 10: Re-examine test_workflow.py
Take a look at test_workflow.py again. It showcases many sample flows on how to interact with Feast. You'll see these show up in the upcoming concepts + architecture + tutorial pages as well.
Next steps
Read the page to understand the Feast data model.
Read the page.
Check out our section for more examples on how to use Feast.
Read the latest features from the offline store for batch scoring
Read the latest features from the online store for real-time inference.
Explore the (experimental) Feast UI
Feast manages deployment to a variety of online stores (e.g. DynamoDB, Redis, Google Cloud Datastore) and ensures necessary features are consistently available and freshly computed at inference time.
Feature and model versioning: Different teams within an organization are often unable to reuse features across projects, resulting in duplicate feature creation logic. Models have data dependencies that need to be versioned, for example when running A/B tests on model versions.
Feast enables discovery of and collaboration on previously used features and enables versioning of sets of features (via feature services).
(Experimental) Feast enables light-weight feature transformations so users can re-use transformation logic across online / offline use cases and across models.
contains a demo setup configuring where data sources are
test_workflow.py showcases how to run all key Feast commands, including defining, retrieving, and pushing features. You can run this with python test_workflow.py.
)
Ingest batch features into an online store (using materialize_incremental)
Fetch online features to power real time inference (using get_online_features)
Ingest streaming features into offline / online stores (using push)
Creating a new Feast repository in /home/Jovyan/my_project.
project: my_project
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
registry: data/registry.db
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 3
# This is an example feature definition file
from datetime import timedelta
import pandas as pd
from feast import (
Entity,
FeatureService,
FeatureView,
Field,
FileSource,
Project,
PushSource,
RequestSource,
)
from feast.on_demand_feature_view import on_demand_feature_view
from feast.types import Float32, Float64, Int64
# Define a project for the feature repo
project = Project(name="my_project", description="A project for driver statistics")
# Define an entity for the driver. You can think of an entity as a primary key used to
# fetch features.
driver = Entity(name="driver", join_keys=["driver_id"])
# Read data from parquet files. Parquet is convenient for local development mode. For
# production, you can use your favorite DWH, such as BigQuery. See Feast documentation
# for more info.
driver_stats_source = FileSource(
name="driver_hourly_stats_source",
path="%PARQUET_PATH%",
timestamp_field="event_timestamp",
created_timestamp_column="created",
)
# Our parquet files contain sample data that includes a driver_id column, timestamps and
# three feature column. Here we define a Feature View that will allow us to serve this
# data to our model online.
driver_stats_fv = FeatureView(
# The unique name of this feature view. Two feature views in a single
# project cannot have the same name, and names must be unique across
# all feature view types (regular, stream, on-demand) to avoid conflicts
# during `feast apply`.
name="driver_hourly_stats",
entities=[driver],
ttl=timedelta(days=1),
# The list of features defined below act as a schema to both define features
# for both materialization of features into a store, and are used as references
# during retrieval for building a training dataset or serving features
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
Field(name="avg_daily_trips", dtype=Int64, description="Average daily trips"),
],
online=True,
source=driver_stats_source,
# Tags are user defined key/value pairs that are attached to each
# feature view
tags={"team": "driver_performance"},
)
# Define a request data source which encodes features / information only
# available at request time (e.g. part of the user initiated HTTP request)
input_request = RequestSource(
name="vals_to_add",
schema=[
Field(name="val_to_add", dtype=Int64),
Field(name="val_to_add_2", dtype=Int64),
],
)
# Define an on demand feature view which can generate new features based on
# existing feature views and RequestSource features
@on_demand_feature_view(
sources=[driver_stats_fv, input_request],
schema=[
Field(name="conv_rate_plus_val1", dtype=Float64),
Field(name="conv_rate_plus_val2", dtype=Float64),
],
)
def transformed_conv_rate(inputs: pd.DataFrame) -> pd.DataFrame:
df = pd.DataFrame()
df["conv_rate_plus_val1"] = inputs["conv_rate"] + inputs["val_to_add"]
df["conv_rate_plus_val2"] = inputs["conv_rate"] + inputs["val_to_add_2"]
return df
# This groups features into a model version
driver_activity_v1 = FeatureService(
name="driver_activity_v1",
features=[
driver_stats_fv[["conv_rate"]], # Sub-selects a feature from a feature view
transformed_conv_rate, # Selects all features from the feature view
],
)
driver_activity_v2 = FeatureService(
name="driver_activity_v2", features=[driver_stats_fv, transformed_conv_rate]
)
# Defines a way to push data (to be available offline, online or both) into Feast.
driver_stats_push_source = PushSource(
name="driver_stats_push_source",
batch_source=driver_stats_source,
)
# Defines a slightly modified version of the feature view from above, where the source
# has been changed to the push source. This allows fresh features to be directly pushed
# to the online store for this feature view.
driver_stats_fresh_fv = FeatureView(
name="driver_hourly_stats_fresh",
entities=[driver],
ttl=timedelta(days=1),
schema=[
Field(name="conv_rate", dtype=Float32),
Field(name="acc_rate", dtype=Float32),
Field(name="avg_daily_trips", dtype=Int64),
],
online=True,
source=driver_stats_push_source, # Changed from above
tags={"team": "driver_performance"},
)
# Define an on demand feature view which can generate new features based on
# existing feature views and RequestSource features
@on_demand_feature_view(
sources=[driver_stats_fresh_fv, input_request], # relies on fresh version of FV
schema=[
Field(name="conv_rate_plus_val1", dtype=Float64),
Field(name="conv_rate_plus_val2", dtype=Float64),
],
)
def transformed_conv_rate_fresh(inputs: pd.DataFrame) -> pd.DataFrame:
df = pd.DataFrame()
df["conv_rate_plus_val1"] = inputs["conv_rate"] + inputs["val_to_add"]
df["conv_rate_plus_val2"] = inputs["conv_rate"] + inputs["val_to_add_2"]
return df
driver_activity_v3 = FeatureService(
name="driver_activity_v3",
features=[driver_stats_fresh_fv, transformed_conv_rate_fresh],
)
import pandas as pd
pd.read_parquet("data/driver_stats.parquet")
feast apply
Created entity driver
Created feature view driver_hourly_stats
Created feature view driver_hourly_stats_fresh
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v3
Created feature service driver_activity_v1
Created feature service driver_activity_v2
Created sqlite table my_project_driver_hourly_stats_fresh
Created sqlite table my_project_driver_hourly_stats
from datetime import datetime
import pandas as pd
from feast import FeatureStore
# Note: see https://docs.feast.dev/getting-started/concepts/feature-retrieval for
# more details on how to retrieve for all entities in the offline store instead
entity_df = pd.DataFrame.from_dict(
{
# entity's join key -> entity values
"driver_id": [1001, 1002, 1003],
# "event_timestamp" (reserved key) -> timestamps
# Each timestamp acts as the upper bound for the point-in-time join:
# Feast retrieves the latest feature values at or before this time,
# preventing data leakage from future events.
"event_timestamp": [
datetime(2021, 4, 12, 10, 59, 42),
datetime(2021, 4, 12, 8, 12, 10),
datetime(2021, 4, 12, 16, 40, 26),
],
# (optional) label name -> label values. Feast does not process these
"label_driver_reported_satisfaction": [1, 5, 3],
# values we're using for an on-demand transformation
"val_to_add": [1, 2, 3],
"val_to_add_2": [10, 20, 30],
}
)
store = FeatureStore(repo_path=".")
training_df = store.get_historical_features(
entity_df=entity_df,
features=[
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_hourly_stats:avg_daily_trips",
"transformed_conv_rate:conv_rate_plus_val1",
"transformed_conv_rate:conv_rate_plus_val2",
],
).to_df()
print("----- Feature schema -----\n")
print(training_df.info())
print()
print("----- Example features -----\n")
print(training_df.head())
# Alternative: Materialize all data using current timestamp (for data without event timestamps)
feast materialize --disable-event-timestamp
Materializing 2 feature views to 2024-04-19 10:59:58-04:00 into the sqlite online store.
driver_hourly_stats from 2024-04-18 15:00:46-04:00 to 2024-04-19 10:59:58-04:00:
100%|████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 370.32it/s]
driver_hourly_stats_fresh from 2024-04-18 15:00:46-04:00 to 2024-04-19 10:59:58-04:00:
100%|███████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 1046.64it/s]
Materializing 2 feature views to 2024-04-19 10:59:58-04:00 into the sqlite online store.
from pprint import pprint
from feast import FeatureStore
store = FeatureStore(repo_path=".")
feature_vector = store.get_online_features(
features=[
"driver_hourly_stats:conv_rate",
"driver_hourly_stats:acc_rate",
"driver_hourly_stats:avg_daily_trips",
],
entity_rows=[
# {join_key: entity_value}
{"driver_id": 1004},
{"driver_id": 1005},
],
).to_dict()
pprint(feature_vector)
INFO: Started server process [66664]
08/17/2022 01:25:49 PM uvicorn.error INFO: Started server process [66664]
INFO: Waiting for application startup.
08/17/2022 01:25:49 PM uvicorn.error INFO: Waiting for application startup.
INFO: Application startup complete.
08/17/2022 01:25:49 PM uvicorn.error INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
08/17/2022 01:25:49 PM uvicorn.error INFO: Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
Type System
Motivation
Feast uses an internal type system to provide guarantees on training and serving data. Feast supports primitive types, array types, set types, map types, JSON, and struct types for feature values. Null types are not supported, although the UNIX_TIMESTAMP type is nullable. The type system is controlled by Value.proto in protobuf and by types.py in Python. Type conversion logic can be found in type_map.py.
Supported Types
Feast supports the following data types:
Primitive Types
Feast Type
Python Type
Description
Domain-Specific Primitive Types
These types are semantic aliases over Bytes for domain-specific use cases (e.g., RAG pipelines, image processing). They are stored as bytes at the proto level.
Feast Type
Python Type
Description
PdfBytes and ImageBytes are not natively supported by any backend's type inference. You must explicitly declare them in your feature view schema. Backend storage treats them as raw bytes.
Array Types
All primitive types have corresponding array (list) types:
Feast Type
Python Type
Description
Set Types
All primitive types (except Map and Json) have corresponding set types for storing unique values:
Feast Type
Python Type
Description
Note: Set types automatically remove duplicate values. When converting from lists or other iterables to sets, duplicates are eliminated.
Backend limitations for Set types:
No backend infers Set types from schema. No offline store (BigQuery, Snowflake, Redshift, PostgreSQL, Spark, Athena, MSSQL) maps its native types to Feast Set types. You must explicitly declare Set types in your feature view schema.
Nested Collection Types
Feast supports arbitrarily nested collections using a recursive VALUE_LIST / VALUE_SET design. The outer container determines the proto enum (VALUE_LIST for Array(…), VALUE_SET for Set(…)), while the full inner type structure is persisted via a mandatory feast:nested_inner_type Field tag.
Feast Type
Python Type
ValueType
Description
Where T is any supported primitive type (Int32, Int64, Float32, Float64, String, Bytes, Bool, UnixTimestamp) or another nested collection type.
Notes:
Nesting depth is unlimited. Array(Array(Array(T))), Set(Array(Set(T))), etc. are all supported.
Inner type information is preserved via Field tags (feast:nested_inner_type) and restored during deserialization. This tag is mandatory for nested collection types.
Map Types
Map types allow storing dictionary-like data structures:
Feast Type
Python Type
Description
Note:Map keys must always be strings. ScalarMap supports non-string scalar keys — Feast infers ScalarMap automatically when the first key of a dict is not a string. Map values can be any supported Feast type, including primitives, arrays, or nested maps at the proto level. However, the PyArrow representation is map<string, string>, which means backends that rely on PyArrow schemas (e.g., during materialization) treat Map as string-to-string.
ScalarMap is not inferred from any backend schema. You must declare it explicitly in your feature view schema. It is best suited for online serving use cases where the online store serializes proto bytes directly (e.g., Redis, DynamoDB, SQLite).
Backend support for Map:
Backend
Native Type
Notes
JSON Type
The Json type represents opaque JSON data. Unlike Map, which is schema-free key-value storage, Json is stored as a string at the proto level but backends use native JSON types where available.
Feast Type
Python Type
Description
Backend support for Json:
Backend
Native Type
When a backend's native type is ambiguous (e.g., PostgreSQL jsonb could be Map or Json), the schema-declared Feast type takes precedence. The backend-to-Feast mappings are only used during schema inference when no explicit type is provided.
Struct Type
The Struct type represents a schema-aware structured type with named, typed fields. Unlike Map (which is schema-free), a Struct declares its field names and their types, enabling schema validation.
Feast Type
Python Type
Description
Example:
Backend support for Struct:
Backend
Native Type
Complete Feature View Example
Below is a complete example showing how to define a feature view with all supported types:
Set Type Usage Examples
Sets store unique values and automatically remove duplicates:
UUID Type Usage Examples
UUID types store universally unique identifiers natively, with support for both random UUIDs and time-based UUIDs:
Decimal Type Usage Examples
The Decimal type stores arbitrary-precision decimal numbers using Python's decimal.Decimal. Values are stored as strings in the proto to preserve full precision — no floating-point rounding occurs.
Decimal is not inferred from any backend schema. You must declare it explicitly in your feature view schema. The pandas dtype for Decimal columns is object (holding decimal.Decimal instances), not a numeric dtype.
Nested Collection Type Usage Examples
Limitation: Empty inner collections round-trip as None:
Map Type Usage Examples
Maps can store complex nested data structures:
ScalarMap Type Usage Examples
ScalarMap supports non-string keys. Feast infers it automatically when the first dict key is not a string:
ScalarMap must be explicitly declared in your feature view schema — it is never inferred from backend type schemas. It is best suited for online serving via stores that use proto byte serialization (e.g., Redis, DynamoDB, SQLite). Materialization paths that use PyArrow (e.g., BigQuery, Snowflake, Redshift, Spark) do not have native ScalarMap support.
JSON Type Usage Examples
Feast's Json type stores values as JSON strings at the proto level. You can pass either a pre-serialized JSON string or a Python dict/list — Feast will call json.dumps() automatically when the value is not already a string:
Struct Type Usage Examples
Type System in Practice
The sections below explain how Feast uses its type system in different contexts.
Feature inference
During feast apply, Feast runs schema inference on the data sources underlying feature views. For example, if the schema parameter is not specified for a feature view, Feast will examine the schema of the underlying data source to determine the event timestamp column, feature columns, and entity columns. Each of these columns must be associated with a Feast type, which requires conversion from the data source type system to the Feast type system.
The feature inference logic calls _infer_features_and_entities.
Types that cannot be inferred:Set, Json, Struct, Decimal, ScalarMap, PdfBytes, and ImageBytes types are never inferred from backend schemas. If you use these types, you must declare them explicitly in your feature view schema.
Materialization
Feast serves feature values as proto objects, which have a type corresponding to Feast types. Thus Feast must materialize feature values into the online store as Value proto objects.
The local materialization engine first pulls the latest historical features and converts it to pyarrow.
Then it calls _convert_arrow_to_proto to convert the pyarrow table to proto format.
This calls python_values_to_proto_values in type_map.py
Historical feature retrieval
The Feast type system is typically not necessary when retrieving historical features. A call to get_historical_features will return a RetrievalJob object, which allows the user to export the results to one of several possible locations: a Pandas dataframe, a pyarrow table, a data lake (e.g. S3 or GCS), or the offline store (e.g. a Snowflake table). In all of these cases, the type conversion is handled natively by the offline store. For example, a BigQuery query exposes a to_dataframe method that will automatically convert the result to a dataframe, without requiring any conversions within Feast.
Feature serving
As mentioned above in the section on , Feast persists feature values into the online store as Value proto objects. A call to get_online_features will return an OnlineResponse object, which essentially wraps a bunch of Value protos with some metadata. The OnlineResponse object can then be converted into a Python dictionary, which calls feast_value_type_to_python_type from type_map.py, a utility that converts the Feast internal types to Python native types.
float
32-bit floating point
Float64
float
64-bit floating point
String
str
String/text value
Bytes
bytes
Binary data
Bool
bool
Boolean value
UnixTimestamp
datetime
Unix timestamp (nullable)
Uuid
uuid.UUID
UUID (any version)
TimeUuid
uuid.UUID
Time-based UUID (version 1)
Decimal
decimal.Decimal
Arbitrary-precision decimal number
List[float]
List of 32-bit floats
Array(Float64)
List[float]
List of 64-bit floats
Array(String)
List[str]
List of strings
Array(Bytes)
List[bytes]
List of binary data
Array(Bool)
List[bool]
List of booleans
Array(UnixTimestamp)
List[datetime]
List of timestamps
Array(Uuid)
List[uuid.UUID]
List of UUIDs
Array(TimeUuid)
List[uuid.UUID]
List of time-based UUIDs
Array(Decimal)
List[decimal.Decimal]
List of arbitrary-precision decimals
Set[float]
Set of unique 32-bit floats
Set(Float64)
Set[float]
Set of unique 64-bit floats
Set(String)
Set[str]
Set of unique strings
Set(Bytes)
Set[bytes]
Set of unique binary data
Set(Bool)
Set[bool]
Set of unique booleans
Set(UnixTimestamp)
Set[datetime]
Set of unique timestamps
Set(Uuid)
Set[uuid.UUID]
Set of unique UUIDs
Set(TimeUuid)
Set[uuid.UUID]
Set of unique time-based UUIDs
Set(Decimal)
Set[decimal.Decimal]
Set of unique arbitrary-precision decimals
No native PyArrow set type. Feast converts Sets to pyarrow.list_() internally, but feast_value_type_to_pa() in type_map.py does not include Set mappings, which can cause errors in some code paths.
Online stores that serialize proto bytes (e.g., SQLite, Redis, DynamoDB) handle Sets correctly.
Offline stores may not handle Set types correctly during retrieval. For example, the Ray offline store only special-cases _LIST types, not _SET.
Set types are best suited for online serving use cases where feature values are written as Python sets and retrieved via get_online_features.
List of sets
Set(Array(T))
List[List[T]]
VALUE_SET
Set of lists
Set(Set(T))
List[List[T]]
VALUE_SET
Set of sets
Array(Array(Array(T)))
List[List[List[T]]]
VALUE_LIST
3-level nesting
Empty inner collections ([]) are stored as empty proto values and round-trip as None. For example, [[1, 2], [], [3]] becomes [[1, 2], None, [3]] after a write-read cycle.
Dict[Any, Any]
Dictionary with non-string scalar keys (int, float, bool, UUID, Decimal, bytes, datetime) and values of any supported Feast type
SUPER
Inferred as Map
Spark
map<string,string>
map<> → Map, array<map<>> → Array(Map)
Athena
map
Inferred as Map
MSSQL
nvarchar(max)
Serialized as string
DynamoDB / Redis
Proto bytes
Full proto Map and ScalarMap support
JSON
Spark
Not natively distinguished from String
MSSQL
nvarchar(max)
VARIANT (serialized)
MSSQL
nvarchar(max) (serialized)
DynamoDB / Redis
Proto bytes
source_datatype_to_feast_value_type calls the appropriate method in type_map.py. For example, if a SnowflakeSource is being examined, snowflake_python_type_to_feast_value_type from type_map.py will be called.
to perform the type conversion.
Int32
int
32-bit signed integer
Int64
int
64-bit signed integer
PdfBytes
bytes
PDF document binary data (used in RAG / document processing pipelines)
ImageBytes
bytes
Image binary data (used in image processing / multimodal pipelines)
Array(Int32)
List[int]
List of 32-bit integers
Array(Int64)
List[int]
List of 64-bit integers
Set(Int32)
Set[int]
Set of unique 32-bit integers
Set(Int64)
Set[int]
Set of unique 64-bit integers
Array(Array(T))
List[List[T]]
VALUE_LIST
List of lists
Array(Set(T))
List[List[T]]
Map
Dict[str, Any]
Dictionary with string keys and values of any supported Feast type (including nested maps)
# Simple set
visited_pages = {"home", "products", "checkout", "products"} # "products" appears twice
# Feast will store this as: {"home", "products", "checkout"}
# Integer set
unique_categories = {1, 2, 3, 2, 1} # duplicates will be removed
# Feast will store this as: {1, 2, 3}
# Converting a list with duplicates to a set
tag_list = [100, 200, 300, 100, 200]
tag_ids = set(tag_list) # {100, 200, 300}
import uuid
# Random UUID (version 4) — use Uuid type
session_id = uuid.uuid4() # e.g., UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')
# Time-based UUID (version 1) — use TimeUuid type
event_id = uuid.uuid1() # e.g., UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8')
# UUID values are returned as uuid.UUID objects from get_online_features()
response = store.get_online_features(
features=["user_features:session_id"],
entity_rows=[{"user_id": 1}],
)
result = response.to_dict()
# result["session_id"][0] is a uuid.UUID object
# UUID lists
related_sessions = [uuid.uuid4(), uuid.uuid4(), uuid.uuid4()]
# UUID sets (unique values)
unique_devices = {uuid.uuid4(), uuid.uuid4()}
import decimal
# Scalar decimal — e.g., a financial price
price = decimal.Decimal("19.99")
# High-precision value — all digits preserved
tax_rate = decimal.Decimal("0.08750000000000000000")
# Decimal values are returned as decimal.Decimal objects from get_online_features()
response = store.get_online_features(
features=["product_features:price"],
entity_rows=[{"product_id": 42}],
)
result = response.to_dict()
# result["price"][0] is a decimal.Decimal object
# Decimal lists — e.g., a history of prices
historical_prices = [
decimal.Decimal("18.50"),
decimal.Decimal("19.00"),
decimal.Decimal("19.99"),
]
# Decimal sets — unique price points seen
unique_prices = {decimal.Decimal("9.99"), decimal.Decimal("19.99"), decimal.Decimal("29.99")}
# List of lists — e.g., weekly score history per user
weekly_scores = [[85.0, 90.5, 78.0], [92.0, 88.5], [95.0, 91.0, 87.5]]
# List of sets — e.g., unique tags assigned per category
unique_tags_per_category = [["python", "ml"], ["rust", "systems"], ["python", "web"]]
# 3-level nesting — e.g., multi-dimensional matrices
Field(name="tensor", dtype=Array(Array(Array(Float64))))
# Mixed nesting
Field(name="grouped_tags", dtype=Array(Set(Array(String))))
import uuid
import decimal
# Integer keys — e.g., category ID → item count
event_counts = {1001: 5, 1002: 12, 1003: 0}
# UUID keys — e.g., session ID → score
import uuid
session_scores = {
uuid.UUID("6ba7b810-9dad-11d1-80b4-00c04fd430c8"): 0.95,
uuid.UUID("a8098c1a-f86e-11da-bd1a-00112444be1e"): 0.87,
}
# Decimal keys — e.g., price bucket → product name
price_tier = {
decimal.Decimal("9.99"): "budget",
decimal.Decimal("49.99"): "standard",
decimal.Decimal("99.99"): "premium",
}
# Type inference: Feast automatically picks SCALAR_MAP when the key is non-string
from feast.type_map import python_type_to_feast_value_type
from feast.value_type import ValueType
python_type_to_feast_value_type({1: "a"}) # → ValueType.SCALAR_MAP
python_type_to_feast_value_type({"a": 1}) # → ValueType.MAP
python_type_to_feast_value_type({}) # → ValueType.MAP (empty dict defaults to MAP)
import json
# Option 1: pass a Python dict — Feast calls json.dumps() internally during proto conversion
raw_event = {"type": "click", "target": "button_1", "metadata": {"page": "home"}}
# Option 2: pass an already-serialized JSON string — Feast validates it via json.loads()
raw_event = '{"type": "click", "target": "button_1", "metadata": {"page": "home"}}'
# When building a DataFrame for store.push(), values must be strings since
# Pandas/PyArrow columns expect uniform types:
import pandas as pd
event_df = pd.DataFrame({
"user_id": ["user_1"],
"event_timestamp": [datetime.now()],
"raw_event": [json.dumps({"type": "click", "target": "button_1"})],
})
store.push("event_push_source", event_df)
# Struct — schema-aware, fields and types are declared
from feast.types import Struct, String, Int32
address = Struct({"street": String, "city": String, "zip": Int32})
# Value: {"street": "123 Main St", "city": "Springfield", "zip": 62704}
Validating historical features with Great Expectations
In this tutorial, we will use the public dataset of Chicago taxi trips to present data validation capabilities of Feast.
The original dataset is stored in BigQuery and consists of raw data for each taxi trip (one row per trip) since 2013.
We will generate several training datasets (aka historical features in Feast) for different periods and evaluate expectations made on one dataset against another.
Types of features we're ingesting and generating:
Features that aggregate raw data with daily intervals (eg, trips per day, average fare or speed for a specific day, etc.).
Features using SQL while pulling data from BigQuery (like total trips time or total miles travelled).
Features calculated on the fly when requested using Feast's on-demand transformations
Our plan:
Prepare environment
Pull data from BigQuery (optional)
Declare & apply features and feature views in Feast
The original notebook and datasets for this tutorial can be found on .
0. Setup
Install Feast Python SDK and great expectations:
1. Dataset preparation (Optional)
You can skip this step if you don't have GCP account. Please use parquet files that are coming with this tutorial instead
Running some basic aggregations while pulling data from BigQuery. Grouping by taxi_id and day:
2. Declaring features
Read more about feature views in
Read more about on demand feature views
3. Generating training (reference) dataset
Generating range of timestamps with daily frequency:
Cross merge (aka relation multiplication) produces entity dataframe with each taxi_id repeated for each timestamp:
taxi_id
event_timestamp
156984 rows × 2 columns
Retrieving historical features for resulting entity dataframe and persisting output as a saved dataset:
4. Developing dataset profiler
Dataset profiler is a function that accepts dataset and generates set of its characteristics. This charasteristics will be then used to evaluate (validate) next datasets.
Important: datasets are not compared to each other! Feast use a reference dataset and a profiler function to generate a reference profile. This profile will be then used during validation of the tested dataset.
Loading saved dataset first and exploring the data:
total_earned
avg_trip_seconds
taxi_id
total_miles_travelled
trip_count
earned_per_hour
event_timestamp
total_trip_seconds
avg_fare
avg_speed
156984 rows × 10 columns
Feast uses as a validation engine and as a dataset's profile. Hence, we need to develop a function that will generate ExpectationSuite. This function will receive instance of (wrapper around pandas.DataFrame) so we can utilize both Pandas DataFrame API and some helper functions from PandasDataset during profiling.
Testing our profiler function:
Verify that all expectations that we coded in our profiler are present here. Otherwise (if you can't find some expectations) it means that it failed to pass on the reference dataset (do it silently is default behavior of Great Expectations).
Now we can create validation reference from dataset and profiler function:
and test it against our existing retrieval job
Validation successfully passed as no exception were raised.
5. Validating new historical retrieval
Creating new timestamps for Dec 2020:
taxi_id
event_timestamp
35448 rows × 2 columns
Execute retrieval job with validation reference:
Validation failed since several expectations didn't pass:
Trip count (mean) decreased more than 10% (which is expected when comparing Dec 2020 vs June 2019)
Average Fare increased - all quantiles are higher than expected
Earn per hour (mean) increased more than 10% (most probably due to increased fare)
Generate reference dataset
Develop & test profiler function
Run validation on different dataset using reference dataset & profiler
import pyarrow.parquet
from google.cloud.bigquery import Client
bq_client = Client(project='kf-feast')
data_query = """SELECT
taxi_id,
TIMESTAMP_TRUNC(trip_start_timestamp, DAY) as day,
SUM(trip_miles) as total_miles_travelled,
SUM(trip_seconds) as total_trip_seconds,
SUM(fare) as total_earned,
COUNT(*) as trip_count
FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips`
WHERE
trip_miles > 0 AND trip_seconds > 60 AND
trip_start_timestamp BETWEEN '2019-01-01' and '2020-12-31' AND
trip_total < 1000
GROUP BY taxi_id, TIMESTAMP_TRUNC(trip_start_timestamp, DAY)"""
def entities_query(year):
return f"""SELECT
distinct taxi_id
FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips`
WHERE
trip_miles > 0 AND trip_seconds > 0 AND
trip_start_timestamp BETWEEN '{year}-01-01' and '{year}-12-31'
"""
import pyarrow.parquet
import pandas as pd
from feast import FeatureView, Entity, FeatureStore, Field, BatchFeatureView
from feast.types import Float64, Int64
from feast.value_type import ValueType
from feast.data_format import ParquetFormat
from feast.on_demand_feature_view import on_demand_feature_view
from feast.infra.offline_stores.file_source import FileSource
from feast.infra.offline_stores.file import SavedDatasetFileStorage
from datetime import timedelta
batch_source = FileSource(
timestamp_field="day",
path="trips_stats.parquet", # using parquet file that we created on previous step
file_format=ParquetFormat()
)
import numpy as np
from feast.dqm.profilers.ge_profiler import ge_profiler
from great_expectations.core.expectation_suite import ExpectationSuite
from great_expectations.dataset import PandasDataset
This guide defines three production-ready deployment topologies for Feast on Kubernetes using the . Each topology addresses a different stage of organizational maturity, from getting started safely to running large-scale multi-tenant deployments.
Topology
Target audience
Key traits
Beyond the core topologies, this guide also covers:
— securing resources with policies
— store choices for AWS, GCP, and on-premise
— routing to multiple backends from a single deployment
Prerequisites: All topologies assume a running Kubernetes cluster with the . Familiarity with Feast and is recommended.
1. Minimal Production
When to use
Small teams with a single ML use case
POCs graduating to production
Low-traffic, non-critical workloads where simplicity is more important than availability
Architecture
Components
Component
Configuration
Notes
Sample FeatureStore CR
Limitations
No high availability — a single replica failure causes downtime
No automatic failover — manual intervention required on failure
2. Standard Production (Recommended)
When to use
Most production ML workloads
Teams with moderate traffic that need reliability
Environments that require TLS, RBAC, and automated scaling
Architecture
Components
Core
Component
Configuration
Notes
Storage
Component
Configuration
Notes
Networking & Security
Component
Configuration
Notes
Sample FeatureStore CR
Key features:
High availability — multi-replica deployment with auto-injected pod anti-affinity and topology spread constraints
Each team gets its own registry and online feature server in a dedicated namespace. This provides the strongest isolation but has notable trade-offs: feature discovery is siloed per team (no cross-project visibility), and each registry requires its own deployment — you cannot view multiple projects in a single UI instance.
Architecture — Shared Registry (cross-namespace)
Alternatively, a single centralized registry server can serve multiple tenant namespaces. Tenant online feature servers connect to the shared registry via the gRPC client. This reduces operational overhead, enables cross-team feature discovery, and allows a single deployment to browse all projects — while Feast enforce tenant isolation at the data level.
Shared registry client configuration — each tenant's feature_store.yaml points to the centralized registry:
Shared vs isolated registries:
Use a shared registry when teams need to discover and reuse features across projects, and rely on Feast permissions for access control. Use isolated registries when regulatory or compliance requirements demand physical separation of metadata.
Shared Registry
Isolated Registries
Components
Multi-tenancy
Aspect
Configuration
Notes
Storage
Component
Configuration
Notes
Scaling
Component
Configuration
Notes
Security
Component
Configuration
Notes
Observability
Component
Purpose
Notes
Reliability & Disaster Recovery
Aspect
Configuration
Notes
Recovery priority guidance
Not all Feast components carry the same recovery urgency. The table below ranks components by restoration priority and provides guidance for RPO (Recovery Point Objective — maximum acceptable data loss) and RTO (Recovery Time Objective — maximum acceptable downtime). Specific targets depend on your backing store SLAs and organizational requirements.
Priority
Component
RPO guidance
RTO guidance
Rationale
Key insight: The online store is reconstructible — it can always be rebuilt from the offline store by re-running materialization. This means its RPO is effectively zero (no unique data to lose), but RTO depends on how long full materialization takes for your dataset volume. For large datasets, consider maintaining Redis persistence (RDB snapshots or AOF) to reduce recovery time.
Backup recommendations by topology
Topology
Registry
Online Store
Offline Store
Sample FeatureStore CR (per tenant)
Feast Permissions and RBAC
Feast provides a built-in permissions framework that secures resources at the application level, independently of Kubernetes RBAC. Permissions are defined as Python objects in your feature repository and registered via feast apply.
For full details, see the and docs.
How it works
Permission enforcement happens on the server side (Online Feature Server, Offline Feature Server, Registry Server). There is no enforcement when using the Feast SDK with a local provider.
Actions
Feast defines eight granular actions:
Action
Description
Convenience aliases are provided:
Alias
Includes
Protected resource types
Permissions can be applied to any of these Feast object types:
The constant ALL_RESOURCE_TYPES includes all of the above. ALL_FEATURE_VIEW_TYPES includes all feature view subtypes.
Policy types
Policy
Match criteria
Use case
Example: Role-based permissions
This is the most common pattern — separate admin and read-only roles:
Example: Namespace-based isolation for multi-tenant deployments
Use NamespaceBasedPolicy to restrict access based on the Kubernetes namespace of the calling service account — ideal for the shared-registry enterprise topology.
Each team gets two permissions: full access to its own resources (matched by team tag), and read-only access to resources any team has explicitly published as shared (matched by visibility: shared tag). The two required_tags target different resources — a feature view tagged team: team-b, visibility: shared matches only the second permission for Team A, enabling cross-team discovery without granting write access:
Example: Combined group + namespace policy
For organizations that use both OIDC groups and Kubernetes namespaces for identity — ideal when platform engineers lack a dedicated namespace but need cross-team visibility, or when OIDC group membership and namespace ownership should independently grant access:
Example: Fine-grained resource filtering
Permissions support name_patterns (regex) and required_tags for targeting specific resources:
Authorization configuration
Enable auth enforcement in feature_store.yaml:
For OIDC:
Permission granting order: Feast uses an affirmative decision strategy — if any matching permission grants access, the request is allowed. Access is denied only when all matching permissions deny the user. If no permission matches a resource + action combination, access is denied. Resources that do not match any configured permission are unsecured. Always define explicit coverage for all critical resources.
Recommended RBAC by topology
Topology
Auth type
Policy type
Guidance
Infrastructure-Specific Recommendations
Choosing the right online store, offline store, and registry backend depends on your cloud environment and existing infrastructure. The table below maps common deployment environments to recommended Feast components.
Recommendation matrix
AWS / EKS / ROSA
Component
Recommended
Alternative
Notes
ROSA (Red Hat OpenShift on AWS): Same store recommendations as EKS. Use OpenShift Routes instead of Ingress for TLS termination. Leverage OpenShift's built-in OAuth for auth.type: kubernetes integration.
GCP / GKE
Component
Recommended
Alternative
Notes
On-Premise / OpenShift / Self-Managed Kubernetes
Component
Recommended
Alternative
Notes
Multi-replica constraint: When scaling any Feast service to multiple replicas (via the Feast Operator), you must use database-backed persistence for all enabled services. File-based stores (SQLite, DuckDB, registry.db) are incompatible with multi-replica deployments. See for details.
Air-Gapped / Disconnected Environment Deployments
Production environments in regulated industries (finance, government, defense) often have no outbound internet access from the Kubernetes cluster. The Feast Operator supports air-gapped deployments through custom container images, init container controls, and standard Kubernetes image-pull mechanisms.
Default init container behavior
When feastProjectDir is set on the FeatureStore CR, the operator creates up to two init containers:
feast-init — bootstraps the feature repository by running either git clone (if feastProjectDir.git is set) or feast init (if feastProjectDir.init is set), then writes the generated feature_store.yaml into the repo directory.
In air-gapped environments, git clone will fail because the cluster cannot reach external Git repositories. The solution is to pre-bake the feature repository into a custom container image and disable the init containers entirely.
Air-gapped deployment workflow
Steps:
Build a custom container image that bundles the feature repository and all Python dependencies into the Feast base image.
Push the image to your internal container registry.
Set services.disableInitContainers: true on the FeatureStore CR to skip git clone / feast init
Sample FeatureStore CR (air-gapped)
Pre-populating the registry: With init containers disabled, feast apply does not run on pod startup. You can populate the registry by:
Running feast apply from your CI/CD pipeline that has network access to the registry DB.
Air-gapped deployment checklist
Pre-stage the following artifacts before deploying Feast in an air-gapped environment:
Container images — Feast feature server image (with bundled feature repo) pushed to internal registry
Hybrid Store Configuration
The hybrid store feature allows a single Feast deployment to route feature operations to multiple backends based on tags or data sources. This is useful when different feature views have different latency, cost, or compliance requirements.
Hybrid online store
The HybridOnlineStore routes online operations to different backends based on a configurable tag on the FeatureView.
feature_store.yaml configuration:
Feature view with routing tag:
The tag value must match the online store type name (e.g. dynamodb, redis, bigtable).
Hybrid offline store
The HybridOfflineStore routes offline operations to different backends based on the batch_source type of each FeatureView.
feature_store.yaml configuration:
Feature views with different sources:
Hybrid offline store constraint:get_historical_features requires all requested feature views to share the same batch_source type within a single call. You cannot join features across different offline engines in one retrieval request.
Performance Considerations
For detailed server-level tuning (worker counts, timeouts, keep-alive, etc.), see the guide.
Online feature server sizing
Traffic tier
Replicas
CPU (per pod)
Memory (per pod)
Notes
Online store latency guidelines
Store
p50 latency
p99 latency
Best for
Connection pooling for remote online store
When using the (client-server architecture), connection pooling significantly reduces latency by reusing TCP/TLS connections:
Tuning by workload:
Registry performance
SQL registry (PostgreSQL, MySQL) is required for concurrent materialization jobs writing to the registry simultaneously.
File-based registries (S3, GCS, local) serialize the entire registry on each write — suitable only for single-writer scenarios.
For read-heavy workloads, scale the Registry Server to multiple replicas (all connecting to the same database).
Registry cache tuning at scale
Each Feast server pod maintains its own in-memory copy of the registry metadata. With multiple Gunicorn workers per pod, the total number of independent registry copies is replicas x workers. For example, 5 replicas with 4 workers each means 20 copies of the registry in memory, each refreshing independently.
With the default cache_mode: sync, the refresh is synchronous — when the TTL expires, the next request blocks until the full registry is re-downloaded. At scale, this causes periodic latency spikes across multiple pods simultaneously.
Recommendation: Use cache_mode: thread with a higher TTL in production to avoid refresh storms:
For the server-side refresh interval, set registryTTLSeconds on the CR:
registryTTLSeconds on the CR controls the server-side refresh interval. cache_ttl_seconds in the registry secret controls the SDK client refresh. In Operator deployments, the CR field is what matters for serving performance. For a deep dive into sync vs thread mode trade-offs, memory impact, and freshness considerations, see the section in the performance tuning guide.
Materialization performance
Data volume
Recommended engine
Notes
For detailed engine configuration, see .
Redis sizing guidelines
Metric
Guideline
Design Principles
Understanding the following principles helps you choose and customize the right topology.
Control plane vs data plane
Control plane (Operator + Registry Server) manages feature definitions, metadata, and lifecycle. It changes infrequently and should be highly available.
Data plane (Online Feature Server + Offline Feature Server) handles the actual feature reads/writes at request time. It must scale with traffic.
Backing stores (databases, object storage) hold the actual data. These are stateful and managed independently.
Stateless vs stateful components
The Feast Operator deploys all Feast services (Online Feature Server, Offline Feature Server, Registry Server) in a single shared Deployment. When scaling (spec.replicas > 1 or HPA autoscaling), all services scale together.
Scaling requires DB-backed persistence for all enabled services. The operator enforces this via CRD validation:
Online Store — must use DB persistence (e.g. type: redis, type: dynamodb, type: postgres
Component
Type
Scaling
DB-backed requirement
Scalability guidelines
Read scaling — increase Online Feature Server replicas; they are stateless and scale linearly.
Write scaling — use a distributed compute engine (, , or ) for materialization.
Storage scaling — scale online and offline stores independently based on data volume and query patterns.
For detailed scaling configuration, see .
Topology Comparison
Capability
Minimal
Standard
Enterprise
Next Steps
— install the Feast Operator and deploy your first FeatureStore CR
— detailed HPA, registry scaling, and materialization engine configuration
— worker counts, timeouts, keep-alive, and server-level tuning
SQLite is simplest for development; Redis for production. See for all options
Offline Store
File-based or MinIO
DuckDB or file-based for development; MinIO/S3 for production. See for all options
Compute Engine
In-process (default)
Suitable for small datasets and development; use Spark, Ray, or Snowflake Engine for larger workloads
Manual scaling — no HPA configured
Limited security — no TLS, no ingress, no RBAC by default
HPA (min 2 replicas, max based on peak load)
Separate container — serves online features from the online store
Offline Feature Server
Scales with the same Deployment
Separate container — serves historical features and materialization source reads from the offline store
Spark, Ray (KubeRay), or Snowflake Engine
Distributed compute for materialization and historical retrieval at scale
Kubernetes Secrets + ${ENV_VAR} substitution
Store credentials via secretRef / envFrom in the FeatureStore CR; inject into feature_store.yaml with
— HPA adjusts the shared deployment replicas (all services scale together) based on demand
Secure external access — TLS-terminated ingress with RBAC
Persistent storage — Online Store (Redis Cluster shown as example; see supported online stores for all options) + Offline Store (PostgreSQL shown as example; see supported offline stores for all options) for durability
User must match at least one group or one namespace
Flexible cross-cutting policies
AllowAll
Always grants access
Development / unsecured resources
K8s service account roles
Enterprise (isolated)
oidc or kubernetes
RoleBasedPolicy + GroupBasedPolicy
Per-team OIDC groups
Enterprise (shared registry)
kubernetes
NamespaceBasedPolicy or CombinedGroupNamespacePolicy
Namespace isolation with tag-based resource scoping
Redshift is the core AWS offline store. Use Snowflake if it's already your warehouse. Athena for S3-native query patterns.
Registry
SQL (RDS PostgreSQL)
S3
SQL registry required for concurrent materialization writers. S3 registry is simpler but limited to single-writer.
Compute Engine
Snowflake Engine
Spark on EMR,
Snowflake engine when your offline/online stores are Snowflake. Spark for S3-based pipelines. Ray with KubeRay for Kubernetes-native distributed processing.
BigQuery is the core GCP offline store with full feature support.
Registry
SQL (Cloud SQL PostgreSQL)
GCS
SQL for multi-writer. GCS for simple single-writer setups.
Compute Engine
Snowflake Engine
Spark on Dataproc,
Use Snowflake engine if your offline store is Snowflake. Spark for BigQuery + GCS pipelines. Ray with KubeRay for Kubernetes-native distributed processing.
Spark for scale. PostgreSQL for simpler setups. Oracle for enterprise customers with existing Oracle infrastructure. DuckDB for development only.
Registry
SQL (PostgreSQL)
—
Always use SQL registry in production on-prem. File-based registries do not support concurrent writers.
Compute Engine
Spark
Run Spark on Kubernetes or standalone. Ray with KubeRay for Kubernetes-native distributed DAG execution.
feast-apply
— runs
feast apply
to register feature definitions in the registry. Controlled by
runFeastApplyOnInit
(defaults to
true
). Skipped when
disableInitContainers
is
true
.
and
feast apply
.
Override the image on each service using the per-service image field.
Set imagePullPolicy: IfNotPresent (or Never if images are pre-loaded on nodes).
Configure imagePullSecrets on the namespace's ServiceAccount — the FeatureStore CRD does not expose an imagePullSecrets field, so use the standard Kubernetes approach of attaching secrets to the ServiceAccount that the pods run under.
Using the FeatureStore CR's built-in CronJob (spec.cronJob) — the operator creates a Kubernetes CronJob that runs feast apply and feast materialize-incremental on a schedule. The CronJob runs inside the cluster (no external access needed) and can use a custom image just like the main deployment. This is the recommended approach for air-gapped environments.
Running feast apply manually from the build environment before deploying the CR.
CRD manifests
— Feast Operator CRDs and operator deployment manifests available locally
Store credentials — Kubernetes Secrets for online store, offline store, and registry DB connections created in the target namespace
Python packages (if using custom on-demand transforms) — bundled into the custom image or available from an internal PyPI mirror
ServiceAccount configuration — imagePullSecrets attached to the ServiceAccount used by the Feast deployment
2–5 (HPA)
1–2
1–2Gi
Standard production
High (>1000 RPS)
5–20 (HPA)
2–4
2–4Gi
Enterprise, per-tenant
High availability + low latency
DynamoDB
<5ms
<20ms
Serverless, variable traffic
PostgreSQL
<5ms
<30ms
On-prem, simplicity
Remote (HTTP)
<10ms
<50ms
Client-server separation
0 (never close)
3
Resource-constrained edge
10
60
2
300
300
Production (frequent schema changes)
thread
60
60
Spark on Kubernetes / EMR / Dataproc, or Ray via KubeRay
Full cluster-scale materialization with distributed DAG execution
Use Redis Cluster for >25GB datasets or >10K connections.
)
Offline Store — if enabled, must use DB persistence (e.g. type: redshift, type: bigquery, type: spark, type: postgres)
Registry — must use SQL persistence (type: sql), a remote registry, or S3/GCS file-backed registry
File-based stores (SQLite, DuckDB, registry.db) are rejected when replicas > 1 or autoscaling is configured.
Offline store must use DB persistence (e.g. Redshift, BigQuery, Spark, PostgreSQL)
Registry Server
Stateless (server)
Scales with the shared Deployment (HPA or spec.replicas)
Registry must use SQL, remote, or S3/GCS persistence
Database-backed for consistency and concurrent access
Online Store
Redis Cluster (example)
Multi-node for availability and low latency; other production stores are also supported — see supported online stores
Offline Store
PostgreSQL (example)
Platform-agnostic DB-backed store; use Redshift/Athena for AWS, BigQuery for GCP, Spark for S3/MinIO pipelines — see supported offline stores for all options
All services (Online Feature Server, Registry, Offline Feature Server) scale together per tenant; set maxReplicas based on your peak load. Independent scaling across tenants.
Cluster
Multi-zone node pools
Zone-aware scheduling with auto-injected topology spread constraints
User must belong to at least one of the listed groups
LDAP/OIDC group membership
from feast.feast_object import ALL_RESOURCE_TYPES
from feast.permissions.action import READ, ALL_ACTIONS, AuthzedAction
from feast.permissions.permission import Permission
from feast.permissions.policy import RoleBasedPolicy
admin_perm = Permission(
name="feast_admin_permission",
types=ALL_RESOURCE_TYPES,
policy=RoleBasedPolicy(roles=["feast-admin-role"]),
actions=ALL_ACTIONS,
)
user_perm = Permission(
name="feast_user_permission",
types=ALL_RESOURCE_TYPES,
policy=RoleBasedPolicy(roles=["feast-user-role"]),
actions=[AuthzedAction.DESCRIBE] + READ,
)
from feast.feast_object import ALL_RESOURCE_TYPES
from feast.permissions.action import ALL_ACTIONS, READ, AuthzedAction
from feast.permissions.permission import Permission
from feast.permissions.policy import NamespaceBasedPolicy
# Team A: full access to its own resources
team_a_own = Permission(
name="team_a_full_access",
types=ALL_RESOURCE_TYPES,
required_tags={"team": "team-a"}, # matches only Team A's resources
policy=NamespaceBasedPolicy(namespaces=["team-a"]),
actions=ALL_ACTIONS,
)
# Team A: read-only access to shared resources published by ANY team
# e.g. a Team B feature view tagged {team: team-b, visibility: shared}
# satisfies required_tags here but NOT team_a_own above
team_a_read_shared = Permission(
name="team_a_read_shared",
types=ALL_RESOURCE_TYPES,
required_tags={"visibility": "shared"}, # matches shared resources from any team
policy=NamespaceBasedPolicy(namespaces=["team-a"]),
actions=[AuthzedAction.DESCRIBE] + READ,
)
# Team B: mirror of the above — full access to its own, read-only to shared
team_b_own = Permission(
name="team_b_full_access",
types=ALL_RESOURCE_TYPES,
required_tags={"team": "team-b"},
policy=NamespaceBasedPolicy(namespaces=["team-b"]),
actions=ALL_ACTIONS,
)
team_b_read_shared = Permission(
name="team_b_read_shared",
types=ALL_RESOURCE_TYPES,
required_tags={"visibility": "shared"},
policy=NamespaceBasedPolicy(namespaces=["team-b"]),
actions=[AuthzedAction.DESCRIBE] + READ,
)
from feast.feast_object import ALL_RESOURCE_TYPES
from feast.permissions.action import ALL_ACTIONS, READ, AuthzedAction
from feast.permissions.permission import Permission
from feast.permissions.policy import CombinedGroupNamespacePolicy
# Platform engineers (OIDC group) OR any team namespace can read shared features.
# This covers platform engineers who have no dedicated K8s namespace of their own
# but need cross-team feature discovery.
platform_read_shared = Permission(
name="platform_read_shared",
types=ALL_RESOURCE_TYPES,
required_tags={"visibility": "shared"},
policy=CombinedGroupNamespacePolicy(
groups=["ml-platform"], # OIDC group for platform/infra engineers
namespaces=["team-a", "team-b"], # team namespaces from enterprise topology
),
actions=[AuthzedAction.DESCRIBE] + READ,
)
# ML engineers (OIDC) OR team namespace owners have full write access.
# Either identity alone is sufficient — useful during namespace migration or
# when the same person holds both the OIDC role and the team namespace.
ml_engineer_write = Permission(
name="ml_engineer_full_access",
types=ALL_RESOURCE_TYPES,
policy=CombinedGroupNamespacePolicy(
groups=["ml-engineers"],
namespaces=["team-a", "team-b"],
),
actions=ALL_ACTIONS,
)
from feast.feature_view import FeatureView
from feast.data_source import DataSource
from feast.permissions.action import AuthzedAction, READ
from feast.permissions.permission import Permission
from feast.permissions.policy import RoleBasedPolicy
sensitive_fv_perm = Permission(
name="sensitive_feature_reader",
types=[FeatureView],
name_patterns=[".*sensitive.*", ".*pii.*"],
policy=RoleBasedPolicy(roles=["trusted-reader"]),
actions=[AuthzedAction.READ_OFFLINE],
)
high_risk_ds_writer = Permission(
name="high_risk_ds_writer",
types=[DataSource],
required_tags={"risk_level": "high"},
policy=RoleBasedPolicy(roles=["admin", "data_team"]),
actions=[AuthzedAction.WRITE_ONLINE, AuthzedAction.WRITE_OFFLINE],
)
