The SQLite online store provides support for materializing feature values into an SQLite database for serving online features.
All feature values are stored in an on-disk SQLite database
Only the latest feature values are persisted
Example
The full set of configuration options is available in .
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Sqlite online store.
Sqlite
To compare this set of functionality against other online stores, please see the full .
yes
readable by Java
no
readable by Go
yes
support for entityless feature views
yes
support for concurrent writing to the same key
no
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
yes
collocated by feature service
no
collocated by entity key
no
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
Here are the methods exposed by the OnlineStore interface, along with the core functionality supported by the method:
online_write_batch: write feature values to the online store
online_read: read feature values from the online store
update: update infrastructure (e.g. tables) in the online store
teardown: teardown infrastructure (e.g. tables) in the online store
plan: generate a plan of infrastructure changes based on feature repo changes
There is also additional functionality not properly captured by these interface methods:
support for on-demand transforms
readable by Python SDK
readable by Java
readable by Go
Finally, there are multiple data models for storing the features in the online store. For example, features could be:
collocated by feature view
collocated by feature service
collocated by entity key
See this for a discussion around the tradeoffs of each of these data models.
Functionality Matrix
There are currently five core online store implementations: SqliteOnlineStore, RedisOnlineStore, DynamoDBOnlineStore, SnowflakeOnlineStore, and DatastoreOnlineStore. There are several additional implementations contributed by the Feast community (PostgreSQLOnlineStore, HbaseOnlineStore, CassandraOnlineStore and IKVOnlineStore), which are not guaranteed to be stable or to match the functionality of the core implementations. Details for each specific online store, such as how to configure it in a feature_store.yaml, can be found .
Below is a matrix indicating which online stores support what functionality.
support for entityless feature views
support for concurrent writing to the same key
support for ttl (time to live) at retrieval
support for deleting expired data
yes
yes
yes
read feature values from the online store
yes
yes
yes
yes
yes
yes
yes
yes
yes
update infrastructure (e.g. tables) in the online store
yes
yes
yes
yes
yes
yes
yes
yes
yes
teardown infrastructure (e.g. tables) in the online store
The online store provides support for materializing feature values into Redis.
Both Redis and Redis Cluster are supported.
ScyllaDB (contrib)
Description
ScyllaDB is a low-latency and high-performance Cassandra-compatible (uses CQL) database. You can use the existing Cassandra connector to use ScyllaDB as an online store in Feast.
The online store provides support for materializing feature values into a ScyllaDB or cluster for serving online features real-time.
Snowflake
Description
The online store provides support for materializing feature values into a Snowflake Transient Table for serving online features.
Only the latest feature values are persisted
The data model used to store feature values in Redis is described in more detail here.
Getting started
In order to use this online store, you'll need to install the redis extra (along with the dependency needed for the offline store of choice). E.g.
pip install 'feast[gcp, redis]'
pip install 'feast[snowflake, redis]'
pip install 'feast[aws, redis]'
pip install 'feast[azure, redis]'
You can get started by using any of the other templates (e.g. feast init -t gcp or feast init -t snowflake or feast init -t aws), and then swapping in Redis as the online store as seen below in the examples.
Examples
Connecting to a single Redis instance:
Connecting to a Redis Cluster with SSL enabled and password authentication:
Connecting to a Redis Sentinel with SSL enabled and password authentication:
Additionally, the redis online store also supports automatically deleting data via a TTL mechanism. The TTL is applied at the entity level, so feature values from any associated feature views for an entity are removed together. This TTL can be set in the feature_store.yaml, using the key_ttl_seconds field in the online store. For example:
The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Redis online store.
Redis
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
no
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
The full set of configuration options is available in CassandraOnlineStoreConfig. For a full explanation of configuration options please look at file sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md.
Storage specifications can be found at docs/specs/online_store_format.md.
Functionality Matrix
The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Cassandra plugin.
Cassandra
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
yes
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
The data model for using a Snowflake Transient Table as an online store follows a tall format (one row per feature)):
"entity_feature_key" (BINARY) -- unique key used when reading specific feature_view x entity combination
"entity_key" (BINARY) -- repeated key currently unused for reading entity_combination
"feature_name" (VARCHAR)
"value" (BINARY)
"event_ts" (TIMESTAMP)
"created_ts" (TIMESTAMP)
(This model may be subject to change when Snowflake Hybrid Tables are released)
Getting started
In order to use this online store, you'll need to run pip install 'feast[snowflake]'. You can then get started with the command feast init REPO_NAME -t snowflake.
Example
Tags KWARGs Actions:
"snowflake-online-store/online_path": Adding the "snowflake-online-store/online_path" key to a FeatureView tags parameter allows you to choose the online table path for the online serving table (ex. "{database}"."{schema}").
The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Snowflake online store.
Snowflake
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
no
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
The PostgreSQL online store provides support for materializing feature values into a PostgreSQL database for serving online features.
Only the latest feature values are persisted
sslmode, sslkey_path, sslcert_path, and sslrootcert_path are optional
Getting started
In order to use this online store, you'll need to run pip install 'feast[postgres]'. You can get started by then running feast init -t postgres.
Example
The full set of configuration options is available in .
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Postgres online store.
Postgres
To compare this set of functionality against other online stores, please see the full .
PGVector
The Postgres online store supports the use of for storing feature values. To enable PGVector, set vector_enabled: true in the online store configuration.
The vector_len parameter can be used to specify the length of the vector. The default value is 512.
Please make sure to follow the instructions in the repository, which, as the time of this writing, requires you to run CREATE EXTENSION vector; in the database.
Then you can use retrieve_online_documents to retrieve the top k closest vectors to a query vector. For the Retrieval Augmented Generation (RAG) use-case, you have to embed the query prior to passing the query vector.
DynamoDB
Description
The DynamoDB online store provides support for materializing feature values into AWS DynamoDB.
Getting started
In order to use this online store, you'll need to run pip install 'feast[aws]'. You can then get started with the command feast init REPO_NAME -t aws.
Example
The full set of configuration options is available in .
Permissions
Feast requires the following permissions in order to execute commands for DynamoDB online store:
The following inline policy can be used to grant Feast the necessary permissions:
Lastly, this IAM role needs to be associated with the desired Redshift cluster. Please follow the official AWS guide for the necessary steps .
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the DynamoDB online store.
DynamoDB
To compare this set of functionality against other online stores, please see the full .
Dragonfly
Description
Dragonfly is a modern in-memory datastore that implements novel algorithms and data structures on top of a multi-threaded, share-nothing architecture. Thanks to its API compatibility, Dragonfly can act as a drop-in replacement for Redis. Due to Dragonfly's hardware efficiency, you can run a single node instance on a small 8GB instance or scale vertically to large 768GB machines with 64 cores. This greatly reduces infrastructure costs as well as architectural complexity.
Similar to Redis, Dragonfly can be used as an online feature store for Feast.
Using Dragonfly as a drop-in Feast online store instead of Redis
Make sure you have Python and pip installed.
Install the Feast SDK and CLI
pip install feast
In order to use Dragonfly as the online store, you'll need to install the redis extra:
pip install 'feast[redis]'
1. Create a feature repository
Bootstrap a new feature repository:
Update feature_repo/feature_store.yaml with the below contents:
2. Start Dragonfly
There are several options available to get Dragonfly up and running quickly. We will be using Docker for this tutorial.
docker run --network=host --ulimit memlock=-1 docker.dragonflydb.io/dragonflydb/dragonfly
3. Register feature definitions and deploy your feature store
feast apply
The apply command scans python files in the current directory (example_repo.py in this case) for feature view/entity definitions, registers the objects, and deploys infrastructure. You should see the following output:
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Redis online store.
Redis
To compare this set of functionality against other online stores, please see the full .
Remote
Description
This remote online store will let you interact with remote feature server. At this moment this only supports the read operation. You can use this online store and able retrieve online features store.get_online_features from remote feature server.
The registry is pointing to registry of remote feature store. If it is not accessible then should be configured to use remote registry.
ssl_cert_path is an optional configuration to the public certificate path when the online server starts in SSL mode. This may be needed if the online server is started with a self-signed certificate, typically this file ends with *.crt, *.cer, or *.pem.
How to configure Authentication and Authorization
Please refer the page for more details on how to configure authentication and authorization.
....
Created entity driver
Created feature view driver_hourly_stats_fresh
Created feature view driver_hourly_stats
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v1
Created feature service driver_activity_v3
Created feature service driver_activity_v2
Bigtable
Description
The Bigtable online store provides support for materializing feature values into Cloud Bigtable. The data model used to store feature values in Bigtable is described in more detail here.
Getting started
In order to use this online store, you'll need to run pip install 'feast[gcp]'. You can then get started with the command feast init REPO_NAME -t gcp.
Example
The full set of configuration options is available in .
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Bigtable online store.
Bigtable
To compare this set of functionality against other online stores, please see the full .
MySQL (contrib)
Description
The MySQL online store provides support for materializing feature values into a MySQL database for serving online features.
Only the latest feature values are persisted
Getting started
In order to use this online store, you'll need to run pip install 'feast[mysql]'. You can get started by then running feast init and then setting the feature_store.yaml as described below.
Example
The full set of configuration options is available in .
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Mys online store.
Mys
To compare this set of functionality against other online stores, please see the full .
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
yes
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
yes
collocated by feature service
no
collocated by entity key
yes
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
type: mysql
host: DB_HOST
port: DB_PORT
database: DB_NAME
user: DB_USERNAME
password: DB_PASSWORD
Cassandra + Astra DB (contrib)
Description
The [Cassandra / Astra DB] online store provides support for materializing feature values into an Apache Cassandra / Astra DB database for online features.
The whole project is contained within a Cassandra keyspace
Each feature view is mapped one-to-one to a specific Cassandra table
This implementation inherits all strengths of Cassandra such as high availability, fault-tolerance, and data distribution
Getting started
In order to use this online store, you'll need to run pip install 'feast[cassandra]'. You can then get started with the command feast init REPO_NAME -t cassandra.
Example (Cassandra)
Example (Astra DB)
The full set of configuration options is available in . For a full explanation of configuration options please look at file sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md.
Storage specifications can be found at docs/specs/online_store_format.md.
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Cassandra online store.
Cassandra
To compare this set of functionality against other online stores, please see the full .
IKV
Description
is a fully-managed embedded key-value store, primarily designed for storing ML features. Most key-value stores (think Redis or Cassandra) need a remote database cluster, whereas IKV allows you to utilize your existing application infrastructure to store data (cost efficient) and access it without any network calls (better performance). See detailed performance benchmarks and cost comparison with Redis on . IKV can be used as an online-store in Feast, the rest of this guide goes over the setup.
Datastore
Description
The online store provides support for materializing feature values into Cloud Datastore. The data model used to store feature values in Datastore is described in more detail .
Hazelcast (contrib)
Description
The online store provides support for materializing feature values into a Hazelcast cluster for serving online features in real-time. In order to use Hazelcast as an online store, you need to have a running Hazelcast cluster. See this page for more details.
Each feature view is mapped one-to-one to a specific Hazelcast IMap
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
no
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
yes
collocated by feature service
no
collocated by entity key
no
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
In order to use this online store, you'll need to install the IKV extra (along with the dependency needed for the offline store of choice). E.g.
pip install 'feast[gcp, ikv]'
pip install 'feast[snowflake, ikv]'
pip install 'feast[aws, ikv]'
pip install 'feast[azure, ikv]'
You can get started by using any of the other templates (e.g. feast init -t gcp or feast init -t snowflake or feast init -t aws), and then swapping in IKV as the online store as seen below in the examples.
Update my_feature_repo/feature_store.yaml with the below contents:
After provisioning an IKV account/store, you should have an account id, passkey and store-name. Additionally you must specify a mount-directory - where IKV will pull/update (maintain) a copy of the index for online reads (IKV is an embedded database). It can be skipped only if you don't plan to read any data from this container. The mount directory path usually points to a location on local/remote disk.
The full set of configuration options is available in IKVOnlineStoreConfig at sdk/python/feast/infra/online_stores/contrib/ikv_online_store/ikv.py
Functionality Matrix
The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the IKV online store.
IKV
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
no
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Datastore online store.
Datastore
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
no
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
This implementation inherits all strengths of Hazelcast such as high availability, fault-tolerance, and data distribution.
Secure TSL/SSL connection is supported by Hazelcast online store.
You can set TTL (Time-To-Live) setting for your features in Hazelcast cluster.
Each feature view corresponds to an IMap in Hazelcast cluster and the entries in that IMap correspond to features of entities. Each feature value stored separately and can be retrieved individually.
Getting started
In order to use Hazelcast online store, you'll need to run pip install 'feast[hazelcast]'. You can then get started with the command feast init REPO_NAME -t hazelcast.
Examples
Functionality Matrix
Hazelcast
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store
yes
generate a plan of infrastructure changes
no
support for on-demand transforms
yes
To compare this set of functionality against other online stores, please see the full functionality matrix.
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
type: hazelcast
cluster_name: dev
cluster_members: ["localhost:5701"]
key_ttl_seconds: 36000
readable by Python SDK
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
yes
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
no
collocated by feature service
no
collocated by entity key
yes
readable by Python SDK
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
no
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
yes
collocated by feature service
no
collocated by entity key
no
readable by Python SDK
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
yes
support for ttl (time to live) at retrieval
yes
support for deleting expired data
yes
collocated by feature view
no
collocated by feature service
no
collocated by entity key
yes
SingleStore (contrib)
Description
The SingleStore online store provides support for materializing feature values into a SingleStore database for serving online features.
Getting started
In order to use this online store, you'll need to run pip install 'feast[singlestore]'. You can get started by then running feast init and then setting the feature_store.yaml as described below.
Example
Functionality Matrix
The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the SingleStore online store.
SingleStore
To compare this set of functionality against other online stores, please see the full .
yes
readable by Java
no
readable by Go
no
support for entityless feature views
yes
support for concurrent writing to the same key
no
support for ttl (time to live) at retrieval
no
support for deleting expired data
no
collocated by feature view
yes
collocated by feature service
no
collocated by entity key
no
write feature values to the online store
yes
read feature values from the online store
yes
update infrastructure (e.g. tables) in the online store
yes
teardown infrastructure (e.g. tables) in the online store