arrow-left
All pages
gitbookPowered by GitBook
1 of 17

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Online stores

Please see Online Store for an explanation of online stores.

Overviewchevron-rightSQLitechevron-rightSnowflakechevron-rightRedischevron-rightDragonflychevron-rightIKVchevron-rightDatastorechevron-rightDynamoDBchevron-rightBigtablechevron-rightPostgreSQL (contrib)chevron-rightCassandra + Astra DB (contrib)chevron-rightMySQL (contrib)chevron-rightHazelcast (contrib)chevron-rightScyllaDB (contrib)chevron-rightRemotechevron-right

remote.md

SingleStore (contrib)chevron-right

SQLite

hashtag
Description

The SQLitearrow-up-right online store provides support for materializing feature values into an SQLite database for serving online features.

  • All feature values are stored in an on-disk SQLite database

  • Only the latest feature values are persisted

hashtag
Example

The full set of configuration options is available in .

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Sqlite online store.

Sqlite

To compare this set of functionality against other online stores, please see the full .

yes

readable by Java

no

readable by Go

yes

support for entityless feature views

yes

support for concurrent writing to the same key

no

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

no

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

yes

support for on-demand transforms

yes

SqliteOnlineStoreConfigarrow-up-right
here
functionality matrix

readable by Python SDK

feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: sqlite
  path: data/online_store.db

Remote

hashtag
Description

This remote online store will let you interact with remote feature server. At this moment this only supports the read operation. You can use this online store and able retrieve online features store.get_online_features from remote feature server.

hashtag
Examples

The registry is pointing to registry of remote feature store. If it is not accessible then should be configured to use remote registry.

ssl_cert_path is an optional configuration to the public certificate path when the online server starts in SSL mode. This may be needed if the online server is started with a self-signed certificate, typically this file ends with *.crt, *.cer, or *.pem.

hashtag
How to configure Authentication and Authorization

Please refer the for more details on how to configure authentication and authorization.

DynamoDB

hashtag
Description

The online store provides support for materializing feature values into AWS DynamoDB.

hashtag
Getting started

In order to use this online store, you'll need to run

IKV

hashtag
Description

is a fully-managed embedded key-value store, primarily designed for storing ML features. Most key-value stores (think Redis or Cassandra) need a remote database cluster, whereas IKV allows you to utilize your existing application infrastructure to store data (cost efficient) and access it without any network calls (better performance). See detailed performance benchmarks and cost comparison with Redis on . IKV can be used as an online-store in Feast, the rest of this guide goes over the setup.

hashtag

feature_store.yaml
project: my-local-project
registry: /remote/data/registry.db
provider: local
online_store:
  path: http://localhost:6566
  type: remote
  ssl_cert_path: /path/to/cert.pem
entity_key_serialization_version: 2
auth:
  type: no_auth
page
pip install 'feast[aws]'
. You can then get started with the command
feast init REPO_NAME -t aws
.

hashtag
Example

The full set of configuration options is available in DynamoDBOnlineStoreConfigarrow-up-right.

hashtag
Permissions

Feast requires the following permissions in order to execute commands for DynamoDB online store:

Command

Permissions

Resources

Apply

dynamodb:CreateTable

dynamodb:DescribeTable

dynamodb:DeleteTable

arn:aws:dynamodb:<region>:<account_id>:table/*

Materialize

dynamodb.BatchWriteItem

arn:aws:dynamodb:<region>:<account_id>:table/*

Get Online Features

dynamodb.BatchGetItem

arn:aws:dynamodb:<region>:<account_id>:table/*

The following inline policy can be used to grant Feast the necessary permissions:

Lastly, this IAM role needs to be associated with the desired Redshift cluster. Please follow the official AWS guide for the necessary steps herearrow-up-right.

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the DynamoDB online store.

DynamoDB

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

To compare this set of functionality against other online stores, please see the full functionality matrix.

DynamoDBarrow-up-right
Getting started

Make sure you have Python and pip installed.

Install the Feast SDK and CLI: pip install feast

In order to use this online store, you'll need to install the IKV extra (along with the dependency needed for the offline store of choice). E.g.

  • pip install 'feast[gcp, ikv]'

  • pip install 'feast[snowflake, ikv]'

  • pip install 'feast[aws, ikv]'

  • pip install 'feast[azure, ikv]'

You can get started by using any of the other templates (e.g. feast init -t gcp or feast init -t snowflake or feast init -t aws), and then swapping in IKV as the online store as seen below in the examples.

hashtag
1. Provision an IKV store

Go to https://inlined.ioarrow-up-right or email onboarding[at]inlined.io

hashtag
2. Configure

Update my_feature_repo/feature_store.yaml with the below contents:

After provisioning an IKV account/store, you should have an account id, passkey and store-name. Additionally you must specify a mount-directory - where IKV will pull/update (maintain) a copy of the index for online reads (IKV is an embedded database). It can be skipped only if you don't plan to read any data from this container. The mount directory path usually points to a location on local/remote disk.

The full set of configuration options is available in IKVOnlineStoreConfig at sdk/python/feast/infra/online_stores/contrib/ikv_online_store/ikv.py

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the IKV online store.

IKV

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

To compare this set of functionality against other online stores, please see the full functionality matrix.

IKVarrow-up-right
https://inlined.ioarrow-up-right

Datastore

hashtag
Description

The Datastorearrow-up-right online store provides support for materializing feature values into Cloud Datastore. The data model used to store feature values in Datastore is described in more detail herearrow-up-right.

hashtag
Getting started

In order to use this online store, you'll need to run pip install 'feast[gcp]'. You can then get started with the command feast init REPO_NAME -t gcp.

hashtag
Example

The full set of configuration options is available in .

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Datastore online store.

Datastore

To compare this set of functionality against other online stores, please see the full .

MySQL (contrib)

hashtag
Description

The MySQL online store provides support for materializing feature values into a MySQL database for serving online features.

  • Only the latest feature values are persisted

hashtag
Getting started

In order to use this online store, you'll need to run pip install 'feast[mysql]'. You can get started by then running feast init and then setting the feature_store.yaml as described below.

hashtag
Example

The full set of configuration options is available in .

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Mys online store.

Mys

To compare this set of functionality against other online stores, please see the full .

SingleStore (contrib)

hashtag
Description

The SingleStore online store provides support for materializing feature values into a SingleStore database for serving online features.

hashtag
Getting started

In order to use this online store, you'll need to run pip install 'feast[singlestore]'. You can get started by then running feast init and then setting the feature_store.yaml as described below.

hashtag
Example

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the SingleStore online store.

SingleStore

To compare this set of functionality against other online stores, please see the full .

Bigtable

hashtag
Description

The Bigtablearrow-up-right online store provides support for materializing feature values into Cloud Bigtable. The data model used to store feature values in Bigtable is described in more detail herearrow-up-right.

hashtag
Getting started

In order to use this online store, you'll need to run pip install 'feast[gcp]'. You can then get started with the command feast init REPO_NAME -t gcp.

hashtag
Example

The full set of configuration options is available in .

hashtag
Functionality Matrix

The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Bigtable online store.

Bigtable

To compare this set of functionality against other online stores, please see the full .

Hazelcast (contrib)

hashtag
Description

The online store provides support for materializing feature values into a Hazelcast cluster for serving online features in real-time. In order to use Hazelcast as an online store, you need to have a running Hazelcast cluster. See this page for more details.

  • Each feature view is mapped one-to-one to a specific Hazelcast IMap

feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: aws
online_store:
  type: dynamodb
  region: us-west-2
{
    "Statement": [
        {
            "Action": [
                "dynamodb:CreateTable",
                "dynamodb:DescribeTable",
                "dynamodb:DeleteTable",
                "dynamodb:BatchWriteItem",
                "dynamodb:BatchGetItem"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:dynamodb:<region>:<account_id>:table/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}
feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: ikv
    account_id: secret
    account_passkey: secret
    store_name: your-store-name
    mount_directory: /absolute/path/on/disk/for/ikv/embedded/index

readable by Python SDK

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

no

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

no

readable by Python SDK

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

yes

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

no

collocated by feature service

no

collocated by entity key

yes

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

no

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

no

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

DatastoreOnlineStoreConfigarrow-up-right
here
functionality matrix

readable by Python SDK

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

no

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

no

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

MySQLOnlineStoreConfigarrow-up-right
here
functionality matrix

readable by Python SDK

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

no

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

no

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

here
functionality matrix

readable by Python SDK

yes

readable by Java

no

readable by Go

no

support for entityless feature views

yes

support for concurrent writing to the same key

yes

support for ttl (time to live) at retrieval

no

support for deleting expired data

no

collocated by feature view

yes

collocated by feature service

no

collocated by entity key

yes

write feature values to the online store

yes

read feature values from the online store

yes

update infrastructure (e.g. tables) in the online store

yes

teardown infrastructure (e.g. tables) in the online store

yes

generate a plan of infrastructure changes

no

support for on-demand transforms

yes

BigtableOnlineStoreConfigarrow-up-right
here
functionality matrix

readable by Python SDK

feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: gcp
online_store:
  type: datastore
  project_id: my_gcp_project
  namespace: my_datastore_namespace
feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: mysql
    host: DB_HOST
    port: DB_PORT
    database: DB_NAME
    user: DB_USERNAME
    password: DB_PASSWORD
feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: singlestore
    host: DB_HOST
    port: DB_PORT
    database: DB_NAME
    user: DB_USERNAME
    password: DB_PASSWORD
feature_store.yaml
project: my_feature_repo
registry: data/registry.db
provider: gcp
online_store:
  type: bigtable
  project_id: my_gcp_project
  instance: my_bigtable_instance

This implementation inherits all strengths of Hazelcast such as high availability, fault-tolerance, and data distribution.

  • Secure TSL/SSL connection is supported by Hazelcast online store.

  • You can set TTL (Time-To-Live) setting for your features in Hazelcast cluster.

  • Each feature view corresponds to an IMap in Hazelcast cluster and the entries in that IMap correspond to features of entities. Each feature value stored separately and can be retrieved individually.

    hashtag
    Getting started

    In order to use Hazelcast online store, you'll need to run pip install 'feast[hazelcast]'. You can then get started with the command feast init REPO_NAME -t hazelcast.

    hashtag
    Examples

    hashtag
    Functionality Matrix

    Hazelcast

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    no

    support for on-demand transforms

    yes

    To compare this set of functionality against other online stores, please see the full functionality matrix.

    Hazelcastarrow-up-right
    getting startedarrow-up-right

    Redis

    hashtag
    Description

    The Redisarrow-up-right online store provides support for materializing feature values into Redis.

    • Both Redis and Redis Cluster are supported.

    • The data model used to store feature values in Redis is described in more detail .

    hashtag
    Getting started

    In order to use this online store, you'll need to install the redis extra (along with the dependency needed for the offline store of choice). E.g.

    • pip install 'feast[gcp, redis]'

    • pip install 'feast[snowflake, redis]'

    • pip install 'feast[aws, redis]'

    You can get started by using any of the other templates (e.g. feast init -t gcp or feast init -t snowflake or feast init -t aws), and then swapping in Redis as the online store as seen below in the examples.

    hashtag
    Examples

    Connecting to a single Redis instance:

    Connecting to a Redis Cluster with SSL enabled and password authentication:

    Connecting to a Redis Sentinel with SSL enabled and password authentication:

    Additionally, the redis online store also supports automatically deleting data via a TTL mechanism. The TTL is applied at the entity level, so feature values from any associated feature views for an entity are removed together. This TTL can be set in the feature_store.yaml, using the key_ttl_seconds field in the online store. For example:

    The full set of configuration options is available in .

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Redis online store.

    Redis

    To compare this set of functionality against other online stores, please see the full .

    ScyllaDB (contrib)

    hashtag
    Description

    ScyllaDB is a low-latency and high-performance Cassandra-compatible (uses CQL) database. You can use the existing Cassandra connector to use ScyllaDB as an online store in Feast.

    The ScyllaDBarrow-up-right online store provides support for materializing feature values into a ScyllaDB or ScyllaDB Cloudarrow-up-right cluster for serving online features real-time.

    hashtag
    Getting started

    Install Feast with Cassandra support:

    Create a new Feast project:

    hashtag
    Example (ScyllaDB)

    hashtag
    Example (ScyllaDB Cloud)

    The full set of configuration options is available in . For a full explanation of configuration options please look at file sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md.

    Storage specifications can be found at docs/specs/online_store_format.md.

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Cassandra plugin.

    Cassandra

    To compare this set of functionality against other online stores, please see the full .

    hashtag
    Resources

    Snowflake

    hashtag
    Description

    The Snowflakearrow-up-right online store provides support for materializing feature values into a Snowflake Transient Table for serving online features.

    • Only the latest feature values are persisted

    The data model for using a Snowflake Transient Table as an online store follows a tall format (one row per feature)):

    • "entity_feature_key" (BINARY) -- unique key used when reading specific feature_view x entity combination

    • "entity_key" (BINARY) -- repeated key currently unused for reading entity_combination

    • "feature_name" (VARCHAR)

    • "value" (BINARY)

    • "event_ts" (TIMESTAMP)

    • "created_ts" (TIMESTAMP)

    (This model may be subject to change when Snowflake Hybrid Tables are released)

    hashtag
    Getting started

    In order to use this online store, you'll need to run pip install 'feast[snowflake]'. You can then get started with the command feast init REPO_NAME -t snowflake.

    hashtag
    Example

    hashtag
    Tags KWARGs Actions:

    "snowflake-online-store/online_path": Adding the "snowflake-online-store/online_path" key to a FeatureView tags parameter allows you to choose the online table path for the online serving table (ex. "{database}"."{schema}").

    The full set of configuration options is available in .

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Snowflake online store.

    Snowflake

    To compare this set of functionality against other online stores, please see the full .

    Dragonfly

    hashtag
    Description

    Dragonflyarrow-up-right is a modern in-memory datastore that implements novel algorithms and data structures on top of a multi-threaded, share-nothing architecture. Thanks to its API compatibility, Dragonfly can act as a drop-in replacement for Redis. Due to Dragonfly's hardware efficiency, you can run a single node instance on a small 8GB instance or scale vertically to large 768GB machines with 64 cores. This greatly reduces infrastructure costs as well as architectural complexity.

    Similar to Redis, Dragonfly can be used as an online feature store for Feast.

    hashtag
    Using Dragonfly as a drop-in Feast online store instead of Redis

    Make sure you have Python and pip installed.

    Install the Feast SDK and CLI

    pip install feast

    In order to use Dragonfly as the online store, you'll need to install the redis extra:

    pip install 'feast[redis]'

    hashtag
    1. Create a feature repository

    Bootstrap a new feature repository:

    Update feature_repo/feature_store.yaml with the below contents:

    hashtag
    2. Start Dragonfly

    There are several options available to get Dragonfly up and running quickly. We will be using Docker for this tutorial.

    docker run --network=host --ulimit memlock=-1 docker.dragonflydb.io/dragonflydb/dragonfly

    hashtag
    3. Register feature definitions and deploy your feature store

    feast apply

    The apply command scans python files in the current directory (example_repo.py in this case) for feature view/entity definitions, registers the objects, and deploys infrastructure. You should see the following output:

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Redis online store.

    Redis

    To compare this set of functionality against other online stores, please see the full .

    Overview

    hashtag
    Functionality

    Here are the methods exposed by the OnlineStore interface, along with the core functionality supported by the method:

    • online_write_batch: write feature values to the online store

    PostgreSQL (contrib)

    hashtag
    Description

    The PostgreSQL online store provides support for materializing feature values into a PostgreSQL database for serving online features.

    • Only the latest feature values are persisted

    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
      type: hazelcast
      cluster_name: dev
      cluster_members: ["localhost:5701"]
      key_ttl_seconds: 36000

    readable by Python SDK

    yes

    readable by Java

    no

    readable by Go

    no

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    yes

    support for ttl (time to live) at retrieval

    yes

    support for deleting expired data

    yes

    collocated by feature view

    no

    collocated by feature service

    no

    collocated by entity key

    yes

    pip install 'feast[azure, redis]'

    yes

    readable by Java

    yes

    readable by Go

    yes

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    yes

    support for ttl (time to live) at retrieval

    yes

    support for deleting expired data

    yes

    collocated by feature view

    no

    collocated by feature service

    no

    collocated by entity key

    yes

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    no

    support for on-demand transforms

    yes

    herearrow-up-right
    RedisOnlineStoreConfigarrow-up-right
    here
    functionality matrix

    readable by Python SDK

    yes

    readable by Java

    no

    readable by Go

    no

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    no

    support for ttl (time to live) at retrieval

    no

    support for deleting expired data

    no

    collocated by feature view

    yes

    collocated by feature service

    no

    collocated by entity key

    no

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    yes

    support for on-demand transforms

    yes

    CassandraOnlineStoreConfigarrow-up-right
    here
    functionality matrix
    Sample application with ScyllaDBarrow-up-right
    ScyllaDB websitearrow-up-right
    ScyllaDB Cloud documentationarrow-up-right

    readable by Python SDK

    yes

    readable by Java

    no

    readable by Go

    no

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    no

    support for ttl (time to live) at retrieval

    no

    support for deleting expired data

    no

    collocated by feature view

    yes

    collocated by feature service

    no

    collocated by entity key

    no

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    no

    support for on-demand transforms

    yes

    SnowflakeOnlineStoreConfigarrow-up-right
    here
    functionality matrix

    readable by Python SDK

    yes

    readable by Java

    yes

    readable by Go

    yes

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    yes

    support for ttl (time to live) at retrieval

    yes

    support for deleting expired data

    yes

    collocated by feature view

    no

    collocated by feature service

    no

    collocated by entity key

    yes

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    no

    support for on-demand transforms

    yes

    here
    functionality matrix

    readable by Python SDK

    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
      type: redis
      connection_string: "localhost:6379"
    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
      type: redis
      redis_type: redis_cluster
      connection_string: "redis1:6379,redis2:6379,ssl=true,password=my_password"
    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
      type: redis
      redis_type: redis_sentinel
      sentinel_master: mymaster
      connection_string: "redis1:26379,ssl=true,password=my_password"
    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
      type: redis
      key_ttl_seconds: 604800
      connection_string: "localhost:6379"
    pip install "feast[cassandra]"
    feast init REPO_NAME -t cassandra
    feature_store.yaml
    project: scylla_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: cassandra
        hosts:
            - 172.17.0.2
        keyspace: feast
        username: scylla
        password: password
    feature_store.yaml
    project: scylla_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: cassandra
        hosts:
            - node-0.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
            - node-1.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
            - node-2.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
        keyspace: feast
        username: scylla
        password: password
    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: snowflake.online
        account: SNOWFLAKE_DEPLOYMENT_URL
        user: SNOWFLAKE_USER
        password: SNOWFLAKE_PASSWORD
        role: SNOWFLAKE_ROLE
        warehouse: SNOWFLAKE_WAREHOUSE
        database: SNOWFLAKE_DATABASE
    example_config.py
    driver_stats_fv = FeatureView(
        ...
        tags={"snowflake-online-store/online_path": '"FEAST"."ONLINE"'},
    )
    feast init feast_dragonfly
    cd feast_dragonfly/feature_repo
    project: feast_dragonfly
    registry: data/registry.db
    provider: local
    online_store:
    type: redis
    connection_string: "localhost:6379"
    ....
    Created entity driver
    Created feature view driver_hourly_stats_fresh
    Created feature view driver_hourly_stats
    Created on demand feature view transformed_conv_rate
    Created on demand feature view transformed_conv_rate_fresh
    Created feature service driver_activity_v1
    Created feature service driver_activity_v3
    Created feature service driver_activity_v2

    online_read: read feature values from the online store

  • update: update infrastructure (e.g. tables) in the online store

  • teardown: teardown infrastructure (e.g. tables) in the online store

  • plan: generate a plan of infrastructure changes based on feature repo changes

  • There is also additional functionality not properly captured by these interface methods:

    • support for on-demand transforms

    • readable by Python SDK

    • readable by Java

    • readable by Go

    • support for entityless feature views

    • support for concurrent writing to the same key

    • support for ttl (time to live) at retrieval

    • support for deleting expired data

    Finally, there are multiple data models for storing the features in the online store. For example, features could be:

    • collocated by feature view

    • collocated by feature service

    • collocated by entity key

    See this issuearrow-up-right for a discussion around the tradeoffs of each of these data models.

    hashtag
    Functionality Matrix

    There are currently five core online store implementations: SqliteOnlineStore, RedisOnlineStore, DynamoDBOnlineStore, SnowflakeOnlineStore, and DatastoreOnlineStore. There are several additional implementations contributed by the Feast community (PostgreSQLOnlineStore, HbaseOnlineStore, CassandraOnlineStore and IKVOnlineStore), which are not guaranteed to be stable or to match the functionality of the core implementations. Details for each specific online store, such as how to configure it in a feature_store.yaml, can be found here.

    Below is a matrix indicating which online stores support what functionality.

    Sqlite

    Redis

    DynamoDB

    Snowflake

    Datastore

    Postgres

    Hbase

    [ / ]

    write feature values to the online store

    yes

    yes

    yes

    yes

    yes

    sslmode, sslkey_path, sslcert_path, and sslrootcert_path are optional

    hashtag
    Getting started

    In order to use this online store, you'll need to run pip install 'feast[postgres]'. You can get started by then running feast init -t postgres.

    hashtag
    Example

    The full set of configuration options is available in PostgreSQLOnlineStoreConfigarrow-up-right.

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Postgres online store.

    Postgres

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    no

    support for on-demand transforms

    yes

    To compare this set of functionality against other online stores, please see the full functionality matrix.

    hashtag
    PGVector

    The Postgres online store supports the use of PGVectorarrow-up-right for storing feature values. To enable PGVector, set vector_enabled: true in the online store configuration.

    The vector_len parameter can be used to specify the length of the vector. The default value is 512.

    Please make sure to follow the instructions in the repository, which, as the time of this writing, requires you to run CREATE EXTENSION vector; in the database.

    Then you can use retrieve_online_documents to retrieve the top k closest vectors to a query vector. For the Retrieval Augmented Generation (RAG) use-case, you have to embed the query prior to passing the query vector.

    Cassandra + Astra DB (contrib)

    hashtag
    Description

    The [Cassandraarrow-up-right / Astra DBarrow-up-right] online store provides support for materializing feature values into an Apache Cassandra / Astra DB database for online features.

    • The whole project is contained within a Cassandra keyspace

    • Each feature view is mapped one-to-one to a specific Cassandra table

    • This implementation inherits all strengths of Cassandra such as high availability, fault-tolerance, and data distribution

    hashtag
    Getting started

    In order to use this online store, you'll need to run pip install 'feast[cassandra]'. You can then get started with the command feast init REPO_NAME -t cassandra.

    hashtag
    Example (Cassandra)

    hashtag
    Example (Astra DB)

    The full set of configuration options is available in . For a full explanation of configuration options please look at file sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md.

    Storage specifications can be found at docs/specs/online_store_format.md.

    hashtag
    Functionality Matrix

    The set of functionality supported by online stores is described in detail . Below is a matrix indicating which functionality is supported by the Cassandra online store.

    Cassandra

    To compare this set of functionality against other online stores, please see the full .

    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: postgres
        host: DB_HOST
        port: DB_PORT
        database: DB_NAME
        db_schema: DB_SCHEMA
        user: DB_USERNAME
        password: DB_PASSWORD
        sslmode: verify-ca
        sslkey_path: /path/to/client-key.pem
        sslcert_path: /path/to/client-cert.pem
        sslrootcert_path: /path/to/server-ca.pem
        vector_enabled: false
        vector_len: 512
    python
    from feast import FeatureStore
    from feast.infra.online_stores.postgres import retrieve_online_documents
    
    feature_store = FeatureStore(repo_path=".")
    
    query_vector = [0.1, 0.2, 0.3, 0.4, 0.5]
    top_k = 5
    
    feature_values = retrieve_online_documents(
        feature_store=feature_store,
        feature_view_name="document_fv:embedding_float",
        query_vector=query_vector,
        top_k=top_k,
    )

    readable by Python SDK

    yes

    readable by Java

    no

    readable by Go

    no

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    no

    support for ttl (time to live) at retrieval

    no

    support for deleting expired data

    no

    collocated by feature view

    yes

    collocated by feature service

    no

    collocated by entity key

    no

    yes

    yes

    yes

    yes

    read feature values from the online store

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    generate a plan of infrastructure changes

    yes

    no

    no

    no

    no

    no

    no

    yes

    no

    support for on-demand transforms

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    readable by Python SDK

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    readable by Java

    no

    yes

    no

    no

    no

    no

    no

    no

    no

    readable by Go

    yes

    yes

    no

    no

    no

    no

    no

    no

    no

    support for entityless feature views

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    yes

    support for concurrent writing to the same key

    no

    yes

    no

    no

    no

    no

    no

    no

    yes

    support for ttl (time to live) at retrieval

    no

    yes

    no

    no

    no

    no

    no

    no

    no

    support for deleting expired data

    no

    yes

    no

    no

    no

    no

    no

    no

    no

    collocated by feature view

    yes

    no

    yes

    yes

    yes

    yes

    yes

    yes

    no

    collocated by feature service

    no

    no

    no

    no

    no

    no

    no

    no

    no

    collocated by entity key

    no

    yes

    no

    no

    no

    no

    no

    no

    yes

    Cassandraarrow-up-right
    Astra DBarrow-up-right
    IKVarrow-up-right

    yes

    readable by Java

    no

    readable by Go

    no

    support for entityless feature views

    yes

    support for concurrent writing to the same key

    no

    support for ttl (time to live) at retrieval

    no

    support for deleting expired data

    no

    collocated by feature view

    yes

    collocated by feature service

    no

    collocated by entity key

    no

    write feature values to the online store

    yes

    read feature values from the online store

    yes

    update infrastructure (e.g. tables) in the online store

    yes

    teardown infrastructure (e.g. tables) in the online store

    yes

    generate a plan of infrastructure changes

    yes

    support for on-demand transforms

    yes

    CassandraOnlineStoreConfigarrow-up-right
    here
    functionality matrix

    readable by Python SDK

    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: cassandra
        hosts:
            - 192.168.1.1
            - 192.168.1.2
            - 192.168.1.3
        keyspace: KeyspaceName
        port: 9042                                                              # optional
        username: user                                                          # optional
        password: secret                                                        # optional
        protocol_version: 5                                                     # optional
        load_balancing:                                                         # optional
            local_dc: 'datacenter1'                                             # optional
            load_balancing_policy: 'TokenAwarePolicy(DCAwareRoundRobinPolicy)'  # optional
        read_concurrency: 100                                                   # optional
        write_concurrency: 100                                                  # optional
    feature_store.yaml
    project: my_feature_repo
    registry: data/registry.db
    provider: local
    online_store:
        type: cassandra
        secure_bundle_path: /path/to/secure/bundle.zip
        keyspace: KeyspaceName
        username: Client_ID
        password: Client_Secret
        protocol_version: 4                                                     # optional
        load_balancing:                                                         # optional
            local_dc: 'eu-central-1'                                            # optional
            load_balancing_policy: 'TokenAwarePolicy(DCAwareRoundRobinPolicy)'  # optional
        read_concurrency: 100                                                   # optional
        write_concurrency: 100                                                  # optional