Welcome to the Feast blog! Here you'll find articles about feature store development, new features, and community updates.

Feast uses online stores to serve features at low latency. Feature values are loaded from data sources into the online store through materialization, which can be triggered through the materialize command (either with specific timestamps or using --disable-event-timestamp to materialize all data with current timestamps).

The storage schema of features within the online store mirrors that of the original data source. One key difference is that for each entity key, only the latest feature values are stored. No historical values are stored.

Here is an example batch data source:

Once the above data source is materialized into Feast (using feast materialize with timestamps or feast materialize --disable-event-timestamp), the feature values will be stored as follows:

Features can also be written directly to the online store via .

Push vs Pull Model

Feast uses a Push Model, i.e., Data Producers push data to the feature store and Feast stores the feature values in the online store, to serve features in real-time.

In a Pull Model, Feast would pull data from the data producers at request time and store the feature values in the online store before serving them (storing them would actually be unnecessary). This approach would incur additional network latency as Feast would need to orchestrate a request to each data producer, which would mean the latency would be at least as long as your slowest call. So, in order to serve features as fast as possible, we push data to Feast and store the feature values in the online store.

The trade-off with the Push Model is that strong consistency is not guaranteed out of the box. Instead, strong consistency has to be explicitly designed for in orchestrating the updates to Feast and the client usage.

The significant advantage with this approach is that Feast is read-optimized for low-latency feature retrieval.

How to Push

Implicit in the Push model are decisions about how and when to push feature values to the online store.

From a developer's perspective, there are three ways to push feature values to the online store with different tradeoffs.

They are discussed further in the section.

Feature repository

Feast users use Feast to manage two important sets of configuration:

• Configuration about how to run Feast on your infrastructure

• Feature definitions

With Feast, the above configuration can be written declaratively and stored as code in a central location. This central location is called a feature repository. The feature repository is the declarative source of truth for what the desired state of a feature store should be.

A feature repository is the collection of python files that define entities, feature views and data sources. Feature Repos also have a feature_store.yaml file at their root.

Users can collaborate by making and reviewing changes to Feast object definitions (feature views, entities, etc.) in the feature repo. But, these objects must be applied, either through API, or the CLI, for them to be available by downstream Feast actions (such as materialization, or retrieving online features). Internally, Feast only looks at the registry when performing these actions, and not at the feature repo directly.

When using the CLI to apply changes (via feast apply), the CLI determines the state of the feature repo from the source files and updates the registry state to reflect the definitions in the feature repo files. This means that new feature views are added to the registry, existing feature views are updated as necessary, and Feast objects removed from the source files are deleted from the registry.

For more details, see the reference.

An offline store is an interface for working with historical time-series feature values that are stored in data sources. The OfflineStore interface has several different implementations, such as the BigQueryOfflineStore, each of which is backed by a different storage and compute engine. For more details on which offline stores are supported, please see Offline Stores.

Offline stores are primarily used for two reasons:

1. Building training datasets from time-series features.

2. Materializing (loading) features into an online store to serve those features at low-latency in a production setting.

Offline stores are configured through the . When building training datasets or materializing features into an online store, Feast will use the configured offline store with your configured data sources to execute the necessary data operations.

Only a single offline store can be used at a time. Moreover, offline stores are not compatible with all data sources; for example, the BigQuery offline store cannot be used to query a file-based data source.

Please see for more details on how to push features directly to the offline store in your feature store.

• • As a part of the Linux Foundation, we ask community members to adhere to the

• : Find the complete Feast codebase on GitHub.

Feast's architecture is designed to be flexible and scalable. It is composed of several components that work together to provide a feature store that can be used to serve features for training and inference.

• Feast uses a to ingest data from different sources and store feature values in the online store. This allows Feast to serve features in real-time with low latency.

• Feast supports for On Demand and Streaming data sources and will support Batch transformations in the future. For Streaming and Batch data sources, Feast requires a separate (in the batch case, this is typically your Offline Store). We are exploring adding a default streaming engine to Feast.

Role-Based Access Control (RBAC) is a security mechanism that restricts access to resources based on the roles/groups/namespaces of individual users within an organization. In the context of the Feast, RBAC ensures that only authorized users or groups/namespaces can access or modify specific resources, thereby maintaining data security and operational integrity.

The RBAC implementation in Feast is designed to:

• Assign Permissions: Allow administrators to assign permissions for various operations and resources to users or groups/namespaces.

• Seamless Integration: Integrate smoothly with existing business code without requiring significant modifications.

Projects provide complete isolation of feature stores at the infrastructure level. This is accomplished through resource namespacing, e.g., prefixing table names with the associated project. Each project should be considered a completely separate universe of entities and features. It is not possible to retrieve features from multiple projects in a single request. We recommend having a single feature store and a single project per environment (dev, staging, prod).

Users define one or more within a project. Each feature view contains one or more . These features typically relate to one or more . A feature view must always have a , which in turn is used during the generation of training and when materializing feature values into the online store.

The concept of a "project" provide the following benefits:

Logical Grouping: Projects group related features together, making it easier to manage and track them.

A data source in Feast refers to raw underlying data that users own (e.g. in a table in BigQuery). Feast does not manage any of the raw underlying data but instead, is in charge of loading this data and performing different operations on the data to retrieve or serve features.

Feast uses a time-series data model to represent data. This data model is used to interpret feature data in data sources in order to build training datasets or materialize features into an online store.

Below is an example data source with a single entity column (driver) and two feature columns (trips_today, and rating).

Feast supports primarily time-stamped tabular data as data sources. There are many kinds of possible data sources:

Tags in Feast allow for efficient filtering of Feast objects when listing them in the UI, CLI, or querying the registry directly.

The way to define tags on the feast objects is through the definition file or directly in the object that will be applied to the feature store.

In this example we define a Feature View in a definition file that has a tag:

In this example we define a Stream Feature View that has a tag, in the code:

An example of filtering feature-views with the tag team:driver_performance:

The same example of listing feature-views without tag filtering:

Use Python to serve your features.

Why should you use Python to Serve features for Machine Learning?

Python has emerged as the primary language for machine learning, and this extends to feature serving and there are five main reasons Feast recommends using a microservice written in Python.

1. Python is the language of Machine Learning

You should meet your users where they are. Python’s popularity in the machine learning community is undeniable. Its simplicity and readability make it an ideal language for writing and understanding complex algorithms. Python boasts a rich ecosystem of libraries such as TensorFlow, PyTorch, XGBoost, and scikit-learn, which provide robust support for developing and deploying machine learning models and we want Feast in this ecosystem.

2. Precomputation is The Way

Precomputing features is the recommended optimal path to ensure low latency performance. Reducing feature serving to a lightweight database lookup is the ideal pattern, which means the marginal overhead of Python should be tolerable. Precomputation ensures product experiences for downstream services are also fast. Slow user experiences are bad user experiences. Precompute and persist data as much as you can.

3. Serving features in another language can lead to skew

Ensuring that features used during model training (offline serving) and online serving are available in production to make real-time predictions is critical. When features are initially developed, they are typically written in Python. This is due to the convenience and efficiency provided by Python's data manipulation libraries. However, in a production environment, there is often interest or pressure to rewrite these features in a different language, like Java, Go, or C++, for performance reasons. This reimplementation introduces a significant risk: training and serving skew. Note that there will always be some minor exceptions (e.g., any Time Since Last Event types of features) but this should not be the rule.

Training and serving skew occurs when there are discrepancies between the features used during model training and those used during prediction. This can lead to degraded model performance, unreliable predictions, and reduced velocity in releasing new features and new models. The process of rewriting features in another language is prone to errors and inconsistencies, which exacerbate this issue.

Rewriting features in another language is not only risky but also resource-intensive. It requires significant time and effort from engineers to ensure that the features are correctly translated. This process can introduce bugs and inconsistencies, further increasing the risk of training and serving skew. Additionally, maintaining two versions of the same feature codebase adds unnecessary complexity and overhead. More importantly, the opportunity cost of this work is high and requires twice the amount of resourcing. Reimplementing code should only be done when the performance gains are worth the investment. Features should largely be precomputed so the latency performance gains should not be the highest impact work that your team can accomplish.

Rather than switching languages, it is more efficient to optimize the performance of your feature store while keeping Python as the primary language. Optimization is a two step process.

Use tools like to understand latency bottlenecks in your code. This will help you prioritize the biggest inefficiencies first. When we initially launched Python native transformations in Python, helped us identify that Pandas resulted in a 10x overhead due to type conversion.

As mentioned, precomputation is the recommended path. In some cases, you may want fully synchronous writes from your data producer to your online feature store, in which case you will want your feature computations and writes to be very fast. In this case, we recommend optimizing the feature calculation code first.

You should optimize your code using libraries, tools, and caching. For example, identify whether your feature calculations can be optimized through vectorized calculations in NumPy; explore tools like Numba for faster execution; and cache frequently accessed data using tools like an lru_cache.

Lastly, Feast will continue to optimize serving in Python and making the overall infrastructure more performant. This will better serve the community.

So we recommend focusing on optimizing your feature-specific code, reporting latency bottlenecks to the maintainers, and contributing to help the infrastructure be more performant.

By keeping features in Python and optimizing performance, you can ensure consistency between training and serving, reduce the risk of errors, and focus on launching more product experiences for your customers.

Embrace Python for feature serving, and leverage its strengths to build robust and reliable machine learning systems.

An entity is a collection of semantically related features. Users define entities to map to the domain of their use case. For example, a ride-hailing service could have customers and drivers as their entities, which group related features that correspond to these customers and drivers.

driver = Entity(name='driver', join_keys=['driver_id'])

The entity name is used to uniquely identify the entity (for example to show in the experimental Web UI). The join key is used to identify the physical primary key on which feature values should be joined together to be retrieved during feature retrieval.

Entities are used by Feast in many contexts, as we explore below:

Use case #1: Defining and storing features

Feast's primary object for defining features is a feature view, which is a collection of features. Feature views map to 0 or more entities, since a feature can be associated with:

• zero entities (e.g. a global feature like num_daily_global_transactions)

• one entity (e.g. a user feature like user_age or last_5_bought_items)

• multiple entities, aka a composite key (e.g. a user + merchant category feature like num_user_purchases_in_merchant_category)

Feast refers to this collection of entities for a feature view as an entity key.

Entities should be reused across feature views. This helps with discovery of features, since it enables data scientists understand how other teams build features for the entity they are most interested in.

Feast will use the feature view concept to then define the schema of groups of features in a low-latency online store.

At training time, users control what entities they want to look up, for example corresponding to train / test / validation splits. A user specifies a list of entity keys + timestamps they want to fetch correct features for to generate a training dataset.

At serving time, users specify entity key(s) to fetch the latest feature values which can power real-time model prediction (e.g. a fraud detection model that needs to fetch the latest transaction user's features to make a prediction).

Feast datasets allow for conveniently saving dataframes that include both features and entities to be subsequently used for data analysis and model training. Data Quality Monitoring was the original motivation for creating the dataset concept.

Dataset's metadata is stored in the Feast registry and raw data (features, entities, additional input keys and timestamp) is stored in the offline store.

Dataset can be created from:

1. Results of historical retrieval

2. [planned] Logging request (including input for on demand transformation) and response during feature serving

3. [planned] Logging features during writing to online store (from batch source or stream)

To create a saved dataset from historical features for later retrieval or analysis, a user needs to call get_historical_features method first and then pass the returned retrieval job to create_saved_dataset method. create_saved_dataset will trigger the provided retrieval job (by calling .persist() on it) to store the data using the specified storage behind the scenes. Storage type must be the same as the globally configured offline store (e.g it's impossible to persist data to a different offline source). create_saved_dataset will also create a SavedDataset object with all of the related metadata and will write this object to the registry.

Saved dataset can be retrieved later using the get_saved_dataset method in the feature store:

Check out our to see how this concept can be applied in a real-world use case.

What is Feast?

Feast (Feature Store) is an open-source feature store that helps teams operate production ML systems at scale by allowing them to define, manage, validate, and serve features for production AI/ML.

Feast's feature store is composed of two foundational components: (1) an offline store for historical feature extraction used in model training and (2) an online store for serving features at low-latency in production systems and applications.

Feast is a configurable operational data system that re-uses existing infrastructure to manage and serve machine learning features to real-time models. For more details, please review our architecture.

Concretely, Feast provides:

• A Python SDK for programmatically defining features, entities, sources, and (optionally) transformations

• A Python SDK for reading and writing features to configured offline and online data stores

• An for reading and writing features (useful for non-python languages)

• A for viewing and exploring information about features defined in the project

• A for viewing and updating feature information

Feast allows ML platform teams to:

• Make features consistently available for training and low-latency serving by managing an offline store (to process historical data for scale-out batch scoring or model training), a low-latency online store (to power real-time prediction), and a battle-tested feature server (to serve pre-computed features online).

• Avoid data leakage by generating point-in-time correct feature sets so data scientists can focus on feature engineering rather than debugging error-prone dataset joining logic. This ensures that future feature values do not leak to models during training.

• Decouple ML from data infrastructure

Feast helps ML platform/MLOps teams with DevOps experience productionize real-time models. Feast also helps these teams build a feature platform that improves collaboration between data engineers, software engineers, machine learning engineers, and data scientists.

• For Data Scientists: Feast is a tool where you can easily define, store, and retrieve your features for both model development and model deployment. By using Feast, you can focus on what you do best: build features that power your AI/ML models and maximize the value of your data.

• For MLOps Engineers: Feast is a library that allows you to connect your existing infrastructure (e.g., online database, application server, microservice, analytical database, and orchestration tooling) that enables your Data Scientists to ship features for their models to production using a friendly SDK without having to be concerned with software engineering challenges that occur from serving real-time production systems. By using Feast, you can focus on maintaining a resilient system, instead of implementing features for Data Scientists.

• For Data Engineers: Feast provides a centralized catalog for storing feature definitions, allowing one to maintain a single source of truth for feature data. It provides the abstraction for reading and writing to many different types of offline and online data stores. Using either the provided Python SDK or the feature server service, users can write data to the online and/or offline stores and then read that data out again in either low-latency online scenarios for model inference, or in batch scenarios for model training.

• An / system. Feast is not a general purpose data pipelining system. Users often leverage tools like to manage upstream data transformations. Feast does support some .

• A data orchestration tool: Feast does not manage or orchestrate complex workflow DAGs. It relies on upstream data pipelines to produce feature values and integrations with tools like to make features consistently available.

• A data warehouse: Feast is not a replacement for your data warehouse or the source of truth for all transformed data in your organization. Rather, Feast is a lightweight downstream layer that can serve data from an existing data warehouse (or other data sources) to models in production.

• reproducible model training / model backtesting / experiment management: Feast captures feature and model metadata, but does not version-control datasets / labels or manage train / test splits. Other tools like , , and are better suited for this.

• batch feature engineering: Feast supports on-demand and streaming transformations. Feast is also investing in supporting batch transformations.

• native streaming feature integration: Feast enables users to push streaming features, but does not pull from streaming sources or manage streaming pipelines.

Many companies have used Feast to power real-world ML use cases such as:

• Personalizing online recommendations by leveraging pre-computed historical user or item features.

• Online fraud detection, using features that compare against (pre-computed) historical transaction patterns

• Churn prediction (an offline model), generating feature values for all users at a fixed cadence in batch

Explore the following resources to get started with Feast:

• is the fastest way to get started with Feast

• describes all important Feast API concepts

• describes Feast's overall architecture.

Functionality

• Create Batch Features: ELT/ETL systems like Spark and SQL are used to transform data in the batch store.

• Create Stream Features: Stream features are created from streaming services such as Kafka or Kinesis, and can be pushed directly into Feast via the Push API.

• Feast Apply: The user (or CI) publishes versioned controlled feature definitions using feast apply. This CLI command updates infrastructure and persists definitions in the object store registry.

• Feast Materialize: The user (or scheduler) executes feast materialize (with timestamps or --disable-event-timestamp to materialize all data with current timestamps) which loads features from the offline store into the online store.

• Model Training: A model training pipeline is launched. It uses the Feast Python SDK to retrieve a training dataset that can be used for training models.

• Get Historical Features: Feast exports a point-in-time correct training dataset based on the list of features and entity dataframe provided by the model training pipeline.

• Deploy Model: The trained model binary (and list of features) are deployed into a model serving system. This step is not executed by Feast.

• Prediction: A backend system makes a request for a prediction from the model serving service.

• Get Online Features: The model serving service makes a request to the Feast Online Serving service for online features using a Feast SDK.

• Feature Retrieval: The online serving service retrieves the latest feature values from the online store and returns them to the model serving service.

A complete Feast deployment contains the following components:

• Feast Registry: An object store (GCS, S3) based registry used to persist feature definitions that are registered with the feature store. Systems can discover feature data by interacting with the registry through the Feast SDK.

• Feast Python SDK/CLI: The primary user facing SDK. Used to:

  • Manage version controlled feature definitions.

Feast project structure

The top-level namespace within Feast is a project. Users define one or more feature views within a project. Each feature view contains one or more features. These features typically relate to one or more entities. A feature view must always have a data source, which in turn is used during the generation of training datasets and when materializing feature values into the online store. You can read more about Feast projects in the project page.

Data ingestion

For offline use cases that only rely on batch data, Feast does not need to ingest data and can query your existing data (leveraging a compute engine, whether it be a data warehouse or (experimental) Spark / Trino). Feast can help manage pushing streaming features to a batch source to make features available for training.

For online use cases, Feast supports ingesting features from batch sources to make them available online (through a process called materialization), and pushing streaming features to make them available both offline / online. We explore this more in the next concept page (Data ingestion)

Feature registration and retrieval

Features are registered as code in a version controlled repository, and tie to data sources + model versions via the concepts of entities, feature views, and feature services. We explore these concepts more in the upcoming concept pages. These features are then stored in a registry, which can be accessed across users and services. The features can then be retrieved via SDK API methods or via a deployed feature server which exposes endpoints to query for online features (to power real time models).

Feast supports several patterns of feature retrieval.

A feature transformation is a function that takes some set of input data and returns some set of output data. Feature transformations can happen on either raw data or derived data.

Feature transformations can be executed by three types of "transformation engines":

1. The Feast Feature Server

2. An Offline Store (e.g., Snowflake, BigQuery, DuckDB, Spark, etc.)

The Feature Server is a core architectural component in Feast, designed to provide low-latency feature retrieval and updates for machine learning applications.

It is a REST API server built using and exposes a limited set of endpoints to serve features, push data, and support materialization operations. The server is scalable, flexible, and designed to work seamlessly with various deployment environments, including local setups and cloud-based systems.

In machine learning workflows, real-time access to feature values is critical for enabling low-latency predictions. The Feature Server simplifies this requirement by:

1. Serving Features: Allowing clients to retrieve feature values for specific entities in real-time, reducing the complexity of direct interactions with the online store.

Production machine learning systems can choose from four approaches to serving machine learning predictions (the output of model inference):

1. Online model inference with online features

2. Offline mode inference without online features

3. Online model inference with online features and cached predictions

A Stream Processor is responsible for consuming data from stream sources (such as Kafka, Kinesis, etc.) and loading it directly into the online (and optionally the offline store).

A Stream Processor abstracts over specific technologies or frameworks that are used to materialize data. An experimental Spark Processor for Kafka is available in Feast.

If the built-in processor is not sufficient, you can create your own custom processor. Please see for more details.

StreamFeatureView is a type of feature view in Feast that allows you to define features that are continuously updated from a streaming source. It is designed to handle real-time data ingestion and feature generation, making it suitable for use cases where features need to be updated frequently as new data arrives.

• LocalComputeEngine

• SparkComputeEngine

• FlinkComputeEngine

A provider is an implementation of a feature store using specific feature store components (e.g. offline store, online store) targeting a specific environment (e.g. GCP stack).

Providers orchestrate various components (offline store, online store, infrastructure, compute) inside an environment. For example, the gcp provider supports as an offline store and as an online store, ensuring that these components can work together seamlessly. Feast has three built-in providers (local, gcp, and aws) with default configurations that make it easy for users to start a feature store in a specific environment. These default configurations can be overridden easily. For instance, you can use the gcp provider but use Redis as the online store instead of Datastore.

If the built-in providers are not sufficient, you can create your own custom provider. Please see

These Feast tutorials showcase how to use Feast to simplify end to end model training / serving.

Feast is highly pluggable and configurable:

• One can use existing plugins (offline store, online store, batch materialization engine, providers) and configure those using the built in options. See reference documentation for details.

• The other way to customize Feast is to build your own custom components, and then point Feast to delegate to them.

Below are some guides on how to add new custom components:

Feast supports registering streaming feature views and Kafka and Kinesis streaming sources. It also provides an interface for stream processing called the Stream Processor. An example Kafka/Spark StreamProcessor is implemented in the contrib folder. For more details, please see the RFC for more details.

Please see here for a tutorial on how to build a versioned streaming pipeline that registers your transformations, features, and data sources in Feast.

Install Feast using pip:

pip install feast

Install Feast with Snowflake dependencies (required when using Snowflake):

pip install 'feast[snowflake]'

Install Feast with GCP dependencies (required when using BigQuery or Firestore):

pip install 'feast[gcp]'

Install Feast with AWS dependencies (required when using Redshift or DynamoDB):

Install Feast with Redis dependencies (required when using Redis, either through AWS Elasticache or independently):

Entity Key Serialization version 2 is now deprecated. All new and existing Feast deployments should shift to using serialization version 3.

However, a challenge arises when existing FeatureViews in online or offline stores have written features with version 2. A version 2 serialized entity key cannot be retrieved using the version 3 deserialization algorithm.

Reserialize the Feature Views entity Keys to version 3

The solution is to reserialize the entity keys from version 2 to version 3.

Follow the following procedures to reserialize the entity key to version 3 in feature View in an offline / online store.

In short, you need to iterate through all the feature views in your Feast repository, retrieve their serialized entity keys (if stored in version 2), reserialize them to version 3, and then update the online/offline store or wherever the serialized keys are stored.

1. Initialize the Feature Store

2. Iterate Through Feature Views

3. Retrieve Serialized Entity Keys

4. Reserialize Entity Keys

5. Update the Online/offline Store

Description

File data sources are files on disk or on S3. Currently only Parquet and Delta formats are supported.

Example

The full set of configuration options is available here.

Supported Types

File data sources support all eight primitive types and their corresponding array types. For a comparison against other batch data sources, please see here.

Feast uses a Push Model to push features to the online store.

This has two important consequences: (1) communication patterns between the Data Producer (i.e., the client) and Feast (i.e,. the server) and (2) feature computation and feature value write patterns to Feast's online store.

Data Producers (i.e., services that generate data) send data to Feast so that Feast can write feature values to the online store. That data can be either raw data where Feast computes and stores the feature values or precomputed feature values.

Communication Patterns

There are two ways a client (or Data Producer) can send data to the online store:

1. Synchronously

   • Using a synchronous API call for a small number of entities or a single entity (e.g., using the ) or the Feature Server's )

2. Asynchronously

   • Using an asynchronous API call for a small number of entities or a single entity (e.g., using the ) or the Feature Server's )

   • Using a "batch job" for a large number of entities (e.g., using a )

Note, in some contexts, developers may "batch" a group of entities together and write them to the online store in a single API call. This is a common pattern when writing data to the online store to reduce write loads but we would not qualify this as a batch job.

Writing feature values to the online store (i.e., the server) can be done in two ways: Precomputing the transformations client-side or Computing the transformations On Demand server-side.

In some scenarios, a combination of Precomputed and On Demand transformations may be optimal.

When selecting feature value write patterns, one must consider the specific requirements of your application, the acceptable correctness of the data, the latency tolerance, and the computational resources available. Making deliberate choices can help the performance and reliability of your service.

There are two ways the client can write feature values to the online store:

1. Precomputing transformations

2. Computing transformations On Demand

3. Hybrid (Precomputed + On Demand)

Precomputed transformations can happen outside of Feast (e.g., via some batch job or streaming application) or inside of the Feast feature server when writing to the online store via the push or write-to-online-store api.

On Demand transformations can only happen inside of Feast at either (1) the time of the client's request or (2) when the data producer writes to the online store. With the transform_on_write parameter, you can control whether transformations are applied during write operations, allowing you to skip transformations for pre-processed data while still enabling transformations during API calls.

The hybrid approach allows for precomputed transformations to happen inside or outside of Feast and have the On Demand transformations happen at client request time. This is particularly convenient for "Time Since Last" types of features (e.g., time since purchase).

When deciding between synchronous and asynchronous data writes, several tradeoffs should be considered:

• Data Consistency: Asynchronous writes allow Data Producers to send data without waiting for the write operation to complete, which can lead to situations where the data in the online store is stale. This might be acceptable in scenarios where absolute freshness is not critical. However, for critical operations, such as calculating loan amounts in financial applications, stale data can lead to incorrect decisions, making synchronous writes essential.

• Correctness: The risk of data being out-of-date must be weighed against the operational requirements. For instance, in a lending application, having up-to-date feature data can be crucial for correctness (depending upon the features and raw data), thus favoring synchronous writes. In less sensitive contexts, the eventual consistency offered by asynchronous writes might be sufficient.

• Service Coupling: Synchronous writes result in tighter coupling between services. If a write operation fails, it can cause the dependent service operation to fail as well, which might be a significant drawback in systems requiring high reliability and independence between services.

The table below can help guide the most appropriate data write and feature computation strategies based on specific application needs and data sensitivity.

The Feast feature registry is a central catalog of all feature definitions and their related metadata. Feast uses the registry to store all applied Feast objects (e.g. Feature views, entities, etc). It allows data scientists to search, discover, and collaborate on new features. The registry exposes methods to apply, list, retrieve and delete these objects, and is an abstraction with multiple implementations.

Feast comes with built-in file-based and sql-based registry implementations. By default, Feast uses a file-based registry, which stores the protobuf representation of the registry as a serialized file in the local file system. For more details on which registries are supported, please see Registries.

Updating the registry

We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD automatically stays synced with the registry. Users will often also want multiple registries to correspond to different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write access since they can impact real user traffic. See Running Feast in Production for details on how to set this up.

Deleting objects from the registry

Using the CLI

The simplest way to delete objects is using the feast delete command:

# Delete any Feast object by name
feast delete my_feature_view
feast delete my_entity
feast delete my_feature_service

See the CLI documentation for more details.

Using the Python SDK

To delete objects programmatically, use the explicit delete methods provided by the FeatureStore class:

For entities, data sources, and other registry objects, you can use the registry methods directly:

The following snippet shows the full lifecycle of deleting a feature view from the registry:

The same pattern works for other registry objects: list/verify the object, call the corresponding delete_* method, then list again to confirm the deletion.

Users can specify the registry through a feature_store.yaml config file, or programmatically. We often see teams preferring the programmatic approach because it makes notebook driven development very easy:

Instantiating a FeatureStore object can then point to this:

Overview

The Feast permissions model allows to configure granular permission policies to all the resources defined in a feature store.

The configured permissions are stored in the Feast registry and accessible through the CLI and the registry APIs.

The permission authorization enforcement is performed when requests are executed through one of the Feast (Python) servers

• The online feature server (REST)

• The offline feature server (Arrow Flight)

• The registry server (gRPC)

Note that there is no permission enforcement when accessing the Feast API with a local provider.

The permission model is based on the following components:

• A resource is a Feast object that we want to secure against unauthorized access.

  • We assume that the resource has a name attribute and optional dictionary of associated key-value tags.

• An action

The Permission class identifies a single permission configured on the feature store and is identified by these attributes:

• name: The permission name.

• types: The list of protected resource types. Defaults to all managed types, e.g. the ALL_RESOURCE_TYPES alias. All sub-classes are included in the resource match.

• name_patterns: A list of regex patterns to match resource names. If any regex matches, the Permission

To simplify configuration, several constants are defined to streamline the permissions setup:

• In module feast.feast_object:

  • ALL_RESOURCE_TYPES is the list of all the FeastObject types.

  • ALL_FEATURE_VIEW_TYPES is the list of all the feature view types, including those not inheriting from

Given the above definitions, the feature store can be configured with granular control over each resource, enabling partitioned access by teams to meet organizational requirements for service and data sharing, and protection of sensitive information.

The feast CLI includes a new permissions command to list the registered permissions, with options to identify the matching resources for each configured permission and the existing resources that are not covered by any permission.

This permission definition grants access to the resource state and the ability to read all of the stores for any feature view or feature service to all users with the role super-reader:

This example grants permission to write on all the data sources with risk_level tag set to high only to users with role admin or data_team:

The following permission grants authorization to read the offline store of all the feature views including risky in the name, to users with role trusted:

When mixing and matching policies in permissions script, the permission granting order is as follows:

1. The first matching policy wins in the list of policies and the permission is granted based on the matching policy rules and rest policies are ignored.

2. If any policy matches from the list of policies, the permission is granted based on the matching policy rules and rest policies are ignored

3. If no policy matches, the permission is denied

In order to leverage the permission functionality, the auth section is needed in the feature_store.yaml configuration. Currently, Feast supports OIDC and Kubernetes RBAC authorization protocols.

The default configuration, if you don't specify the auth configuration section, is no_auth, indicating that no permission enforcement is applied.

The auth section includes a type field specifying the actual authorization protocol, and protocol-specific fields that are specified in .

This page covers common use cases for Feast and how a feature store can benefit your AI/ML workflows.

Recommendation Engines

Recommendation engines require personalized feature data related to users, items, and their interactions. Feast can help by:

• Managing feature data: Store and serve user preferences, item characteristics, and interaction history

• Low-latency serving: Provide real-time features for dynamic recommendations

• Point-in-time correctness: Ensure training and serving data are consistent to avoid data leakage

• Feature reuse: Allow different recommendation models to share the same feature definitions

A typical recommendation engine might need features such as:

• User features: demographics, preferences, historical behavior

• Item features: categories, attributes, popularity scores

• Interaction features: past user-item interactions, ratings

Feast allows you to define these features once and reuse them across different recommendation models, ensuring consistency between training and serving environments.

Risk scorecards (such as credit risk, fraud risk, and marketing propensity models) require a comprehensive view of entity data with historical contexts. Feast helps by:

• Feature consistency: Ensure all models use the same feature definitions

• Historical feature retrieval: Generate training datasets with correct point-in-time feature values

• Feature monitoring: Track feature distributions to detect data drift

• Governance: Maintain an audit trail of features used in regulated environments

Credit risk models might use features like:

• Transaction history patterns

• Account age and status

• Payment history features

• External credit bureau data

Feast enables you to combine these features from disparate sources while maintaining data consistency and freshness.

Natural Language Processing (NLP) and Retrieval Augmented Generation (RAG) applications require efficient storage and retrieval of text embeddings. Feast supports these use cases by:

• Vector storage: Store and index embedding vectors for efficient similarity search

• Document metadata: Associate embeddings with metadata for contextualized retrieval

• Scaling retrieval: Serve vectors with low latency for real-time applications

• Versioning: Track changes to embedding models and document collections

RAG systems can leverage Feast to:

• Store document embeddings and chunks in a vector database

• Retrieve contextually relevant documents for user queries

• Combine document retrieval with entity-specific features

• Scale to large document collections

Feast makes it remarkably easy to make data available for retrieval by providing a simple API for both storing and querying vector embeddings.

Time series forecasting for demand planning, inventory management, and anomaly detection benefits from Feast through:

• Temporal feature management: Store and retrieve time-bound features

• Feature engineering: Create time-based aggregations and transformations

• Consistent feature retrieval: Ensure training and inference use the same feature definitions

• Backfilling capabilities: Generate historical features for model training

Demand forecasting applications typically use features such as:

• Historical sales data with temporal patterns

• Seasonal indicators and holiday flags

• Weather data

• Price changes and promotions

Feast allows you to combine these diverse data sources and make them available for both batch training and online inference.

While Feast was initially built for structured data, it can also support multi-modal applications by:

• Storing feature metadata: Keep track of image paths, embeddings, and metadata

• Vector embeddings: Store image embeddings for similarity search

• Feature fusion: Combine image features with structured data features

Across all these use cases, Feast provides several core benefits:

1. Consistency between training and serving: Eliminate training-serving skew by using the same feature definitions

2. Feature reuse: Define features once and use them across multiple models

3. Scalable feature serving: Serve features at low latency for production applications

4. Feature governance: Maintain a central registry of feature definitions with metadata

By implementing a feature store with Feast, teams can focus on model development rather than data engineering challenges, accelerating the delivery of ML applications to production.

Feature values in Feast are modeled as time-series records. Below is an example of a driver feature view with two feature columns (trips_today, and earnings_today):

The above table can be registered with Feast through the following feature view:

from feast import Entity, FeatureView, Field, FileSource
from feast.types import Float32, Int64
from datetime import timedelta

driver = Entity(name="driver", join_keys=["driver_id"])

driver_stats_fv = FeatureView(
    name="driver_hourly_stats",
    entities=[driver],
    schema=[
        Field(name="trips_today", dtype=Int64),
        Field(name="earnings_today", dtype=Float32),
    ],
    ttl=timedelta(hours=2),
    source=FileSource(
        path="driver_hourly_stats.parquet"
    )
)

Feast is able to join features from one or more feature views onto an entity dataframe in a point-in-time correct way. This means Feast is able to reproduce the state of features at a specific point in the past.

Given the following entity dataframe, imagine a user would like to join the above driver_hourly_stats feature view onto it, while preserving the trip_success column:

The timestamps within the entity dataframe above are the events at which we want to reproduce the state of the world (i.e., what the feature values were at those specific points in time). In order to do a point-in-time join, a user would load the entity dataframe and run historical retrieval:

For each row within the entity dataframe, Feast will query and join the selected features from the appropriate feature view data source. Feast will scan backward in time from the entity dataframe timestamp up to a maximum of the TTL time specified.

Below is the resulting joined training dataframe. It contains both the original entity rows and joined feature values:

Three feature rows were successfully joined to the entity dataframe rows. The first row in the entity dataframe was older than the earliest feature rows in the feature view and could not be joined. The last row in the entity dataframe was outside of the TTL window (the event happened 11 hours after the feature row) and also couldn't be joined.

By default, point-in-time joins only constrain the feature's event timestamp. If a data source also has a created_timestamp_column, it is used to deduplicate rows that share an event timestamp (the row with the highest created timestamp wins), but it is not otherwise filtered. This means a value that was backfilled or corrected after an entity dataframe timestamp can still be returned for it.

To restrict retrieval to feature values that were already available at each entity row's timestamp, pass filter_by_created_timestamp=True:

This adds a created_timestamp <= entity_timestamp condition to the join, so each entity dataframe row only sees feature values whose created timestamp is at or before its own timestamp. This is useful to keep backfilled values from leaking into training data, and to reproduce what the online store would have served at each event time (assuming the created timestamp reflects when the value became available online).

When individuals apply for loans from banks and other credit providers, the decision to approve a loan application is often made through a statistical model. This model uses information about a customer to determine the likelihood that they will repay or default on a loan, in a process called credit scoring.

In this example, we will demonstrate how a real-time credit scoring system can be built using Feast and Scikit-Learn on AWS, using feature data from S3.

This real-time system accepts a loan request from a customer and responds within 100ms with a decision on whether their loan has been approved or rejected.

This end-to-end tutorial will take you through the following steps:

• Deploying S3 with Parquet as your primary data source, containing both loan features and zip code features

• Deploying Redshift as the interface Feast uses to build training datasets

• Registering your features with Feast and configuring DynamoDB for online serving

• Building a training dataset with Feast to train your credit scoring model

• Loading feature values from S3 into DynamoDB

• Making online predictions with your credit scoring model using features from DynamoDB

In this example, you'll learn how to use some of the key functionality in Feast. The tutorial runs in both local mode and on the Google Cloud Platform (GCP). For GCP, you must have access to a GCP project already, including read and write permissions to BigQuery.

This tutorial guides you on how to use Feast with Scikit-learn. You will learn how to:

• Train a model locally (on your laptop) using data from BigQuery

• Test the model for online inference using SQLite (for fast iteration)

• Test the model for online inference using (for production use)

Try it and let us know what you think!

We integrate with a wide set of tools and technologies so you can make Feast work in your existing stack. Many of these integrations are maintained as plugins to the main Feast repo.

Integrations

See Functionality and Roadmap

Standards

In order for a plugin integration to be highlighted, it must meet the following requirements:

1. The plugin must have tests. Ideally it would use the Feast universal tests (see this guide for an example), but custom tests are fine.

2. The plugin must have some basic documentation on how it should be used.

3. The author must work with a maintainer to pass a basic code review (e.g. to ensure that the implementation roughly matches the core Feast implementations).

In order for a plugin integration to be merged into the main Feast repo, it must meet the following requirements:

1. The PR must pass all integration tests. The universal tests (tests specifically designed for custom integrations) must be updated to test the integration.

2. There is documentation and a tutorial on how to use the integration.

3. The author (or someone else) agrees to take ownership of all the files, and maintain those files going forward.

This example demonstrates how to enable MCP (Model Context Protocol) support in Feast, allowing AI agents and applications to interact with your features through standardized MCP interfaces.

Prerequisites

1. Python 3.8+

2. Feast installed

3. FastAPI MCP library

Installation

1. Install Feast with MCP support:

Alternatively, you can install the dependencies separately:

1. Navigate to this example directory within your cloned Feast repository:

1. Initialize a Feast repository in this directory. We'll use the existing feature_store.yaml that's already configured for MCP:

This will create a data subdirectory and a feature_repo subdirectory if they don't exist, and will use the feature_store.yaml present in the current directory (examples/mcp_feature_store).

1. Apply the feature store configuration:

Start the Feast feature server with MCP support:

If MCP is properly configured, you should see a log message indicating that MCP support has been enabled:

The fastapi_mcp integration automatically exposes your Feast feature server's FastAPI endpoints as MCP tools. This means AI assistants can:

• Call /get-online-features to retrieve features from the feature store

• Use /health to check server status

The key configuration that enables MCP support:

A feature repository is a directory that contains the configuration of the feature store and individual features. This configuration is written as code (Python/YAML) and it's highly recommended that teams track it centrally using git. See Feature Repository for a detailed explanation of feature repositories.

The easiest way to create a new feature repository to use feast init command:

feast init

Creating a new Feast repository in /<...>/tiny_pika.

feast init -t snowflake
Snowflake

feast init -t gcp

Creating a new Feast repository in /<...>/tiny_pika.

feast init -t aws
AWS Region (e.g. us-west-2): ...
Redshift Cluster ID: ...
Redshift Database Name: ...
Redshift User Name: ...
Redshift S3 Staging Location (s3://*): ...
Redshift IAM Role for S3 (arn:aws:iam::*:role/*): ...
Should I upload example data to Redshift (overwriting 'feast_driver_hourly_stats' table)? (Y/n):

Creating a new Feast repository in /<...>/tiny_pika.

The init command creates a Python file with feature definitions, sample data, and a Feast configuration file for local development:

$ tree
.
└── tiny_pika
    ├── data
    │   └── driver_stats.parquet
    ├── example.py
    └── feature_store.yaml

1 directory, 3 files

Enter the directory:

You can now use this feature repository for development. You can try the following:

• Run feast apply to apply these definitions to Feast.

• Edit the example feature definitions in example.py and run feast apply again to change feature definitions.

• Initialize a git repository in the same directory and checking the feature repository into version control.

The Feast Python SDK allows users to retrieve feature values from an online store. This API is used to look up feature values at low latency during model serving in order to make online predictions.

Please ensure that you have materialized (loaded) your feature values into the online store before starting

Create a list of features that you would like to retrieve. This list typically comes from the model training step and should accompany the model binary.

Next, we will create a feature store object and call get_online_features() which reads the relevant feature values directly from the online store.

Throughout this tutorial, we’ll walk through the creation of a production-ready fraud prediction system. A prediction is made in real-time as the user makes the transaction, so we need to be able to generate a prediction at low latency.

Our end-to-end example will perform the following workflows:

• Computing and backfilling feature data from raw data

• Building point-in-time correct training datasets from feature data and training a model

Please see for a conceptual explanation of data sources.

Feast frequently has to mediate data across platforms and systems, each with its own unique type system. To make this possible, Feast itself has a type system for all the types it is able to handle natively.

Feast's type system is built on top of protobuf. The messages that make up the type system can be found here, and the corresponding python classes that wrap them can be found here.

Feast supports the following categories of data types:

• Primitive types: numerical values (Int32, Int64, Float32, Float64), String, Bytes, Bool, and UnixTimestamp.

• Zoned timestamp type: ZonedTimestamp stores a timezone-aware datetime as both the UTC instant and its originating zone, so the original wall-clock zone round-trips losslessly. This differs from UnixTimestamp, which is always decoded as UTC and discards the source zone. Use ZonedTimestamp when local time-of-day or the offset/zone itself is meaningful. It must be explicitly declared in schema (it is not inferred by any backend), and is not supported as an entity key.

• Domain-specific primitives: PdfBytes (PDF binary data for RAG/document pipelines) and ImageBytes (image binary data for multimodal pipelines). These are semantic aliases over Bytes and must be explicitly declared in schema — no backend infers them.

• UUID types: Uuid and TimeUuid for universally unique identifiers. Stored as strings at the proto level but deserialized to uuid.UUID objects in Python.

• Array types: ordered lists of any primitive type, e.g. Array(Int64), Array(String), Array(Uuid).

• Set types: unordered collections of unique values for any primitive type, e.g. Set(String), Set(Int64). Set types are not inferred by any backend and must be explicitly declared. They are best suited for online serving use cases.

• Map types: dictionary-like structures. Map has string keys and values that can be any supported Feast type (including nested maps), e.g. Map, Array(Map). ScalarMap has non-string scalar keys (int, float, bool, UUID, Decimal, bytes, datetime) — Feast infers ScalarMap automatically when the first key is not a string. ScalarMap must be explicitly declared in schema and is not inferred by any backend.

• JSON type: opaque JSON data stored as a string at the proto level but semantically distinct from String — backends use native JSON types (jsonb, VARIANT, etc.), e.g. Json, Array(Json).

• Struct type: schema-aware structured type with named, typed fields. Unlike Map (which is schema-free), a Struct declares its field names and their types, enabling schema validation, e.g. Struct({"name": String, "age": Int32}).

For a complete reference with examples, see .

Each feature or schema field in Feast is associated with a data type, which is stored in Feast's . These types are also used to ensure that Feast operates on values correctly (e.g. making sure that timestamp columns used for actually have the timestamp type).

As a result, each system that Feast interacts with needs a way to translate data types from the native platform into a Feast type. E.g., Snowflake SQL types are converted to Feast types . The onus is therefore on authors of offline or online store connectors to make sure that this type mapping happens correctly.

Map, JSON, and Struct types are supported across all major Feast backends:

Note: When the backend native type is ambiguous (e.g., jsonb could be Map, Json, or Struct), the schema-declared Feast type takes precedence. The backend-to-Feast type mappings above are only used for schema inference when no explicit type is provided.

Note: Feast currently does not support a null type in its type system.

Feast allows users to build a training dataset from time-series feature data that already exists in an offline store. Users are expected to provide a list of features to retrieve (which may span multiple feature views), and a dataframe to join the resulting features onto. Feast will then execute a point-in-time join of multiple feature views onto the provided dataframe, and return the full resulting dataframe.

Retrieving historical features

1. Register your feature views

Please ensure that you have created a feature repository and that you have registered (applied) your feature views with Feast.

2. Define feature references

Start by defining the feature references (e.g., driver_trips:average_daily_rides) for the features that you would like to retrieve from the offline store. These features can come from multiple feature tables. The only requirement is that the feature tables that make up the feature references have the same entity (or composite entity), and that they aren't located in the same offline store.

feature_refs = [
    "driver_trips:average_daily_rides",
    "driver_trips:maximum_daily_rides",
    "driver_trips:rating",
    "driver_trips:rating:trip_completed",
    # Optionally, reference a specific version: "driver_trips@v2:average_daily_rides"
]

3. Create an entity dataframe

An entity dataframe is the target dataframe on which you would like to join feature values. The entity dataframe must contain a timestamp column called event_timestamp and all entities (primary keys) necessary to join feature tables onto. All entities found in feature views that are being joined onto the entity dataframe must be found as column on the entity dataframe.

It is possible to provide entity dataframes as either a Pandas dataframe or a SQL query.

Pandas:

In the example below we create a Pandas based entity dataframe that has a single row with an event_timestamp column and a driver_id entity column. Pandas based entity dataframes may need to be uploaded into an offline store, which may result in longer wait times compared to a SQL based entity dataframe.

SQL (Alternative):

Below is an example of an entity dataframe built from a BigQuery SQL query. It is only possible to use this query when all feature views being queried are available in the same offline store (BigQuery).

4. Launch historical retrieval

Once the feature references and an entity dataframe are defined, it is possible to call get_historical_features(). This method launches a job that executes a point-in-time join of features from the offline store onto the entity dataframe. Once completed, a job reference will be returned. This job reference can then be converted to a Pandas dataframe by calling to_df().

Feast allows users to load their feature data into an online store in order to serve the latest features to models for online prediction.

Materializing features

1. Register feature views

Before proceeding, please ensure that you have applied (registered) the feature views that should be materialized.

2.a Materialize

The materialize command allows users to materialize features over a specific historical time range into the online store.

feast materialize 2021-04-07T00:00:00 2021-04-08T00:00:00

The above command will query the batch sources for all feature views over the provided time range, and load the latest feature values into the configured online store.

It is also possible to materialize for specific feature views by using the -v / --views argument.

feast materialize 2021-04-07T00:00:00 2021-04-08T00:00:00 \
--views driver_hourly_stats

The materialize command is completely stateless. It requires the user to provide the time ranges that will be loaded into the online store. This command is best used from a scheduler that tracks state, like Airflow.

2.b Materialize Incremental (Alternative)

For simplicity, Feast also provides a materialize command that will only ingest new data that has arrived in the offline store. Unlike materialize, materialize-incremental will track the state of previous ingestion runs inside of the feature registry.

The example command below will load only new data that has arrived for each feature view up to the end date and time (2021-04-08T00:00:00).

The materialize-incremental command functions similarly to materialize in that it loads data over a specific time range for all feature views (or the selected feature views) into the online store.

Unlike materialize, materialize-incremental automatically determines the start time from which to load features from batch sources of each feature view. The first time materialize-incremental is executed it will set the start time to the oldest timestamp of each data source, and the end time as the one provided by the user. For each run of materialize-incremental, the end timestamp will be tracked.

Subsequent runs of materialize-incremental will then set the start time to the end time of the previous run, thus only loading new data that has arrived into the online store. Note that the end time that is tracked for each run is at the feature view level, not globally for all feature views, i.e, different feature views may have different periods that have been materialized into the online store.

The Feast CLI can be used to deploy a feature store to your infrastructure, spinning up any necessary persistent resources like buckets or tables in data stores. The deployment target and effects depend on the provider that has been configured in your feature_store.yaml file, as well as the feature definitions found in your feature repository.

Deploying

To have Feast deploy your infrastructure, run feast apply from your command line while inside a feature repository:

feast apply

# Processing example.py as example
# Done!

Depending on whether the feature repository is configured to use a local provider or one of the cloud providers like GCP or AWS, it may take from a couple of seconds to a minute to run to completion.

Cleaning up

If you need to clean up the infrastructure created by feast apply, use the teardown command.

****

Note: The materialization is now constructed via unified compute engine interface.

A Compute Engine in Feast is a component that handles materialization and historical retrieval tasks. It is responsible for executing the logic defined in feature views, such as aggregations, transformations, and custom user-defined functions (UDFs).

A materialization task abstracts over specific technologies or frameworks that are used to materialize data. It allows users to use a pure local serialized approach (which is the default LocalComputeEngine), or delegates the materialization to separate components (e.g. AWS Lambda, as implemented by the LambdaComputeEngine).

If the built-in engines are not sufficient, you can create your own custom materialization engine. Please see for more details.

Please see for configuring engines.

Batch Engine Config can be configured in the feature_store.yaml file, and it serves as the default configuration for all materialization and historical retrieval tasks. The batch_engine config in BatchFeatureView. E.g

The OpenTelemetry integration in Feast provides comprehensive monitoring and observability capabilities for your feature serving infrastructure. This component enables you to track key metrics, traces, and logs from your Feast deployment.

Motivation

Monitoring and observability are critical for production machine learning systems. The OpenTelemetry integration addresses these needs by:

1. Performance Monitoring: Track CPU and memory usage of feature servers

2. Operational Insights: Collect metrics to understand system behavior and performance

3. Troubleshooting: Enable effective debugging through distributed tracing

4. Resource Optimization: Monitor resource utilization to optimize deployments

5. Production Readiness: Provide enterprise-grade observability capabilities

The OpenTelemetry integration in Feast consists of several components working together:

• OpenTelemetry Collector: Receives, processes, and exports telemetry data

• Prometheus Integration: Enables metrics collection and monitoring

• Instrumentation: Automatic Python instrumentation for tracking metrics

• Exporters: Components that send telemetry data to monitoring systems

1. Automated Instrumentation: Python auto-instrumentation for comprehensive metric collection

2. Metric Collection: Track key performance indicators including:

   • Memory usage

   • CPU utilization

To add monitoring to the Feast Feature Server, follow these steps:

Follow the to install the operator.

Before installing the OpenTelemetry Operator:

1. Install cert-manager

2. Validate that the pods are running

3. Apply the OpenTelemetry operator:

For additional installation steps, refer to the .

Add the OpenTelemetry Collector configuration under the metrics section in your values.yaml file:

Add the following annotations and environment variables to your deployment.yaml:

Add metric checks to all manifests and deployment files:

Add the following components to your chart:

• Instrumentation

• OpenTelemetryCollector

• ServiceMonitors

• Prometheus Instance

Deploy Feast with metrics enabled:

To enable OpenTelemetry monitoring in your Feast deployment:

1. Set metrics.enabled=true in your Helm values

2. Configure the OpenTelemetry Collector endpoint

3. Deploy with proper annotations and environment variables

Example configuration:

Once configured, you can monitor various metrics including:

• feast_feature_server_memory_usage: Memory utilization of the feature server

• feast_feature_server_cpu_usage: CPU usage statistics

• feast_feature_server_request_latency_seconds: Request latency with feature count dimensions

For the full list of metrics, see the .

These metrics can be visualized using Prometheus and other compatible monitoring tools.

Introduction

This example notebook provides a step-by-step demonstration of building and using a RAG system with Feast and the custom FeastRagRetriever. The notebook walks through:

1. Data Preparation

   • Loads a subset of the Wikipedia DPR dataset (1% of training data)

   • Implements text chunking with configurable chunk size and overlap

   • Processes text into manageable passages with unique IDs

2. Embedding Generation

   • Uses all-MiniLM-L6-v2 sentence transformer model

   • Generates 384-dimensional embeddings for text passages

   • Demonstrates batch processing with GPU support

3. Feature Store Setup

   • Creates a Parquet file as the historical data source

   • Configures Feast with the feature repository

   • Demonstrates writing embeddings from data source to Milvus online store which can be used for model training later

4. RAG System Implementation

   • Embedding Model: all-MiniLM-L6-v2 (configurable)

   • Generator Model: granite-3.2-2b-instruct (configurable)

5. Query Demonstration

   • Perform inference with retrieved context

• A Kubernetes cluster with:

  • GPU nodes available (for model inference)

  • At least 200GB of storage

  • A standalone Milvus deployment. See example .

Clone this repository: https://github.com/feast-dev/feast.git Navigate to the examples/rag-retriever directory. Here you will find the following files:

• feature_repo/feature_store.yaml This is the core configuration file for the RAG project's feature store, configuring a Milvus online store on a local provider.

  • In order to configure Milvus you should:

    • Update feature_store.yaml with your Milvus connection details:

Open rag_feast.ipynb and follow the steps in the notebook to run the example.

As an alternative to the manual data preparation steps in the notebook above, Feast provides the DocEmbedder class that automates the entire document-to-embeddings pipeline: chunking, embedding generation, FeatureView creation, and writing to the online store.

1. Generates a FeatureView: Automatically creates a Python file with Entity and FeatureView definitions compatible with feast apply

2. Applies the repo: Registers the FeatureView in the Feast registry and deploys infrastructure (e.g., Milvus collection)

3. Chunks documents: Splits text into smaller passages using TextChunker (configurable chunk size, overlap, etc.)

• Custom Chunker: Subclass BaseChunker for your own chunking strategy

• Custom Embedder: Subclass BaseEmbedder to use a different embedding model

• Logical Layer Function: Provide a SchemaTransformFn to control how the output maps to your FeatureView schema

See rag_feast_docembedder.ipynb for a complete end-to-end example that uses DocEmbedder with the Wiki DPR dataset and then queries the results using FeastRAGRetriever.

• Ensure your Milvus instance is properly configured and running

• Vector dimensions and similarity metrics can be adjusted in the feature store configuration

• The example uses Wikipedia data, but the system can be adapted for other datasets

In the steps below, we will set up a sample Feast project that leverages Snowflake as an offline store + materialization engine + online store.

Starting with data in a Snowflake table, we will register that table to the feature store and define features associated with the columns in that table. From there, we will generate historical training data based on those feature definitions and then materialize the latest feature values into the online store. Lastly, we will retrieve the materialized feature values.

Our template will generate new data containing driver statistics. From there, we will show you code snippets that will call to the offline store for generating training datasets, and then the code for calling the online store to serve you the latest feature values to serve models in production.

Snowflake Offline Store Example

Install feast-snowflake

pip install 'feast[snowflake]'

Get a Snowflake Trial Account (Optional)

Snowflake Trial Account

Create a feature repository

feast init -t snowflake {feature_repo_name}
Snowflake Deployment URL (exclude .snowflakecomputing.com):
Snowflake User Name::
Snowflake Password::
Snowflake Role Name (Case Sensitive)::
Snowflake Warehouse Name (Case Sensitive)::
Snowflake Database Name (Case Sensitive)::
Should I upload example data to Snowflake (overwrite table)? [Y/n]: Y
cd {feature_repo_name}

The following files will automatically be created in your project folder:

• feature_store.yaml -- This is your main configuration file

• driver_repo.py -- This is your main feature definition file

• test.py -- This is a file to test your feature store configuration

Here you will see the information that you entered. This template will use Snowflake as the offline store, materialization engine, and the online store. The main thing to remember is by default, Snowflake objects have ALL CAPS names unless lower case was specified.

Feast can generate tailored Jupyter notebooks for any Feast project. The notebooks adapt to your feature_store.yaml configuration and provide a hands-on walkthrough of core Feast functionality.

What you get

For each project discovered, Feast creates a directory with notebooks covering:

The content adapts automatically based on:

• Online / offline store types — descriptions reflect the actual backends configured.

• Registry type — local registries include feast apply; remote registries use refresh_registry().

• Authentication — auth details from feature_store.yaml are surfaced when configured.

• Python 3.9+

• Feast installed (pip install feast)

• A feature repository with a valid feature_store.yaml

Run the command from (or pointing to) a directory containing feature_store.yaml:

This searches for feature_store.yaml in the current directory and every file inside the feast-config/ directory. Each file in feast-config/ is treated as a separate project config. For each project found, notebooks are written to ./feast-demo-notebooks/<project>/.

If your feast-config/ directory contains multiple files, each is treated as a separate project and a dedicated notebook directory is created:

Open any generated notebook in Jupyter, JupyterLab, or VS Code and run cells from top to bottom. Each notebook:

1. Configures the path to your feature_store.yaml automatically (no manual editing needed).

2. Connects to the feature store using the Feast Python SDK.

3. Walks through relevant operations with real data from your project.

Overview

Feast batch materialization operations (materialize and materialize-incremental), and get_historical_features are executed through a ComputeEngine.

Custom batch compute engines allow Feast users to extend Feast to customize the materialization and get_historical_features process. Examples include:

• Setting up custom materialization-specific infrastructure during feast apply (e.g. setting up Spark clusters or Lambda Functions)

• Launching custom batch ingestion (materialization) jobs (Spark, Beam, AWS Lambda)

• Tearing down custom materialization-specific infrastructure during feast teardown (e.g. tearing down Spark clusters, or deleting Lambda Functions)

Feast comes with built-in materialization engines, e.g, LocalComputeEngine, and an experimental LambdaComputeEngine. However, users can develop their own compute engines by creating a class that implements the contract in the .

The fastest way to add custom logic to Feast is to implement the ComputeEngine. The guide that follows will extend the LocalProvider with operations that print text to the console. It is up to you as a developer to add your custom code to the engine methods, but the guide below will provide the necessary scaffolding to get you started.

The first step is to define a custom compute engine class. We've created the MyCustomEngine below. This python file can be placed in your feature_repo directory if you're following the Quickstart guide.

Notice how in the above engine we have only overwritten two of the methods on the LocalComputeEngine, namely update and materialize. These two methods are convenient to replace if you are planning to launch custom batch jobs. If you want to use the compute to execute the get_historical_features method, you will need to implement the get_historical_features method as well.

Configure your file to point to your new engine class:

Notice how the batch_engine field above points to the module and class where your engine can be found.

Now you should be able to use your engine by running a Feast command:

It may also be necessary to add the module root path to your PYTHONPATH as follows:

That's it. You should now have a fully functional custom engine!

This page covers deploying Feast on Kubernetes, including the Feast Operator and feature servers.

Kubernetes is a common target environment for running Feast in production. You can use Kubernetes to:

1. Run Feast feature servers for online feature retrieval.

2. Run scheduled and ad-hoc jobs (e.g. materialization jobs) as Kubernetes Jobs.

3. Operate Feast components using Kubernetes-native primitives.

These guides cover the FeatureStore Custom Resource (CR) from an operator perspective — what to put in the CR, how the operator translates it into Kubernetes objects, and where to look for store-specific YAML options in the Feast SDK docs.

A common scenario when using Feast in production is to want to test changes to Feast object definitions. For this, we recommend setting up a staging environment for your offline and online stores, which mirrors production (with potentially a smaller data set). Having this separate environment allows users to test changes by first applying them to staging, and then promoting the changes to production after verifying the changes on staging.

There are three common ways teams approach having separate environments

1. Have separate git branches for each environment

2. Have separate feature_store.yaml