Description

The Snowflake online store provides support for materializing feature values into a Snowflake Transient Table for serving online features.

• Only the latest feature values are persisted

The data model for using a Snowflake Transient Table as an online store follows a tall format (one row per feature)):

• "entity_feature_key" (BINARY) -- unique key used when reading specific feature_view x entity combination

• "entity_key" (BINARY) -- repeated key currently unused for reading entity_combination

• "feature_name" (VARCHAR)

• "value" (BINARY)

• "event_ts" (TIMESTAMP)

• "created_ts" (TIMESTAMP)

(This model may be subject to change when Snowflake Hybrid Tables are released)

Getting started

In order to use this online store, you'll need to run pip install 'feast[snowflake]'. You can then get started with the command feast init REPO_NAME -t snowflake.

Example

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: snowflake.online
    account: SNOWFLAKE_DEPLOYMENT_URL
    user: SNOWFLAKE_USER
    password: SNOWFLAKE_PASSWORD
    role: SNOWFLAKE_ROLE
    warehouse: SNOWFLAKE_WAREHOUSE
    database: SNOWFLAKE_DATABASE

Tags KWARGs Actions:

"snowflake-online-store/online_path": Adding the "snowflake-online-store/online_path" key to a FeatureView tags parameter allows you to choose the online table path for the online serving table (ex. "{database}"."{schema}").

driver_stats_fv = FeatureView(
    ...
    tags={"snowflake-online-store/online_path": '"FEAST"."ONLINE"'},
)

The full set of configuration options is available in SnowflakeOnlineStoreConfig.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Snowflake online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Description

The Datastore online store provides support for materializing feature values into Cloud Datastore. The data model used to store feature values in Datastore is described in more detail here.

Getting started

In order to use this online store, you'll need to run pip install 'feast[gcp]'. You can then get started with the command feast init REPO_NAME -t gcp.

Example

project: my_feature_repo
registry: data/registry.db
provider: gcp
online_store:
  type: datastore
  project_id: my_gcp_project
  namespace: my_datastore_namespace

The full set of configuration options is available in DatastoreOnlineStoreConfig.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Datastore online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Functionality

Here are the methods exposed by the OnlineStore interface, along with the core functionality supported by the method:

• online_write_batch: write feature values to the online store

• online_read: read feature values from the online store

• update: update infrastructure (e.g. tables) in the online store

• teardown: teardown infrastructure (e.g. tables) in the online store

• plan: generate a plan of infrastructure changes based on feature repo changes

There is also additional functionality not properly captured by these interface methods:

• support for on-demand transforms

• readable by Python SDK

• readable by Java

• readable by Go

• support for entityless feature views

• support for concurrent writing to the same key

• support for ttl (time to live) at retrieval

• support for deleting expired data

Finally, there are multiple data models for storing the features in the online store. For example, features could be:

• collocated by feature view

• collocated by feature service

• collocated by entity key

See this issue for a discussion around the tradeoffs of each of these data models.

Functionality Matrix

There are currently five core online store implementations: SqliteOnlineStore, RedisOnlineStore, DynamoDBOnlineStore, SnowflakeOnlineStore, and DatastoreOnlineStore. There are several additional implementations contributed by the Feast community (PostgreSQLOnlineStore, HbaseOnlineStore, and CassandraOnlineStore), which are not guaranteed to be stable or to match the functionality of the core implementations. Details for each specific online store, such as how to configure it in a feature_store.yaml, can be found here.

Below is a matrix indicating which online stores support what functionality.

Description

The Redis online store provides support for materializing feature values into Redis.

• Both Redis and Redis Cluster are supported.

• The data model used to store feature values in Redis is described in more detail here.

Getting started

In order to use this online store, you'll need to install the redis extra (along with the dependency needed for the offline store of choice). E.g.

• pip install 'feast[gcp, redis]'

• pip install 'feast[snowflake, redis]'

• pip install 'feast[aws, redis]'

• pip install 'feast[azure, redis]'

You can get started by using any of the other templates (e.g. feast init -t gcp or feast init -t snowflake or feast init -t aws), and then swapping in Redis as the online store as seen below in the examples.

Examples

Connecting to a single Redis instance:

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: redis
  connection_string: "localhost:6379"

Connecting to a Redis Cluster with SSL enabled and password authentication:

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: redis
  redis_type: redis_cluster
  connection_string: "redis1:6379,redis2:6379,ssl=true,password=my_password"

Additionally, the redis online store also supports automatically deleting data via a TTL mechanism. The TTL is applied at the entity level, so feature values from any associated feature views for an entity are removed together. This TTL can be set in the feature_store.yaml, using the key_ttl_seconds field in the online store. For example:

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: redis
  key_ttl_seconds: 604800
  connection_string: "localhost:6379"

The full set of configuration options is available in RedisOnlineStoreConfig.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Redis online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Description

The DynamoDB online store provides support for materializing feature values into AWS DynamoDB.

Getting started

In order to use this online store, you'll need to run pip install 'feast[aws]'. You can then get started with the command feast init REPO_NAME -t aws.

Example

project: my_feature_repo
registry: data/registry.db
provider: aws
online_store:
  type: dynamodb
  region: us-west-2

The full set of configuration options is available in DynamoDBOnlineStoreConfig.

Permissions

Feast requires the following permissions in order to execute commands for DynamoDB online store:

The following inline policy can be used to grant Feast the necessary permissions:

{
    "Statement": [
        {
            "Action": [
                "dynamodb:CreateTable",
                "dynamodb:DescribeTable",
                "dynamodb:DeleteTable",
                "dynamodb:BatchWriteItem",
                "dynamodb:BatchGetItem"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:dynamodb:<region>:<account_id>:table/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

Lastly, this IAM role needs to be associated with the desired Redshift cluster. Please follow the official AWS guide for the necessary steps here.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the DynamoDB online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Description

The [Cassandra / Astra DB] online store provides support for materializing feature values into an Apache Cassandra / Astra DB database for online features.

• The whole project is contained within a Cassandra keyspace

• Each feature view is mapped one-to-one to a specific Cassandra table

• This implementation inherits all strengths of Cassandra such as high availability, fault-tolerance, and data distribution

Getting started

In order to use this online store, you'll need to run pip install 'feast[cassandra]'. You can then get started with the command feast init REPO_NAME -t cassandra.

Example (Cassandra)

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: cassandra
    hosts:
        - 192.168.1.1
        - 192.168.1.2
        - 192.168.1.3
    keyspace: KeyspaceName
    port: 9042                                                              # optional
    username: user                                                          # optional
    password: secret                                                        # optional
    protocol_version: 5                                                     # optional
    load_balancing:                                                         # optional
        local_dc: 'datacenter1'                                             # optional
        load_balancing_policy: 'TokenAwarePolicy(DCAwareRoundRobinPolicy)'  # optional
    read_concurrency: 100                                                   # optional
    write_concurrency: 100                                                  # optional

Example (Astra DB)

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
    type: cassandra
    secure_bundle_path: /path/to/secure/bundle.zip
    keyspace: KeyspaceName
    username: Client_ID
    password: Client_Secret
    protocol_version: 4                                                     # optional
    load_balancing:                                                         # optional
        local_dc: 'eu-central-1'                                            # optional
        load_balancing_policy: 'TokenAwarePolicy(DCAwareRoundRobinPolicy)'  # optional
    read_concurrency: 100                                                   # optional
    write_concurrency: 100                                                  # optional

The full set of configuration options is available in CassandraOnlineStoreConfig. For a full explanation of configuration options please look at file sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md.

Storage specifications can be found at docs/specs/online_store_format.md.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Cassandra online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Description

The Bigtable online store provides support for materializing feature values into Cloud Bigtable. The data model used to store feature values in Bigtable is described in more detail here.

Getting started

In order to use this online store, you'll need to run pip install 'feast[gcp]'. You can then get started with the command feast init REPO_NAME -t gcp.

Example

project: my_feature_repo
registry: data/registry.db
provider: gcp
online_store:
  type: bigtable
  project_id: my_gcp_project
  instance: my_bigtable_instance

The full set of configuration options is available in BigtableOnlineStoreConfig.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Bigtable online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Description

The SQLite online store provides support for materializing feature values into an SQLite database for serving online features.

• All feature values are stored in an on-disk SQLite database

• Only the latest feature values are persisted

Example

project: my_feature_repo
registry: data/registry.db
provider: local
online_store:
  type: sqlite
  path: data/online_store.db

The full set of configuration options is available in SqliteOnlineStoreConfig.

Functionality Matrix

The set of functionality supported by online stores is described in detail here. Below is a matrix indicating which functionality is supported by the Sqlite online store.

To compare this set of functionality against other online stores, please see the full functionality matrix.

Please see Online Store for an explanation of online stores.

Description

The MySQL online store provides support for materializing feature values into a MySQL database for serving online features.

• Only the latest feature values are persisted

Getting started

In order to use this online store, you'll need to run pip install 'feast[mysql]'. You can get started by then running feast init and then setting the feature_store.yaml as described below.

Example

Functionality Matrix

Description

The PostgreSQL online store provides support for materializing feature values into a PostgreSQL database for serving online features.

• Only the latest feature values are persisted

• sslmode, sslkey_path, sslcert_path, and sslrootcert_path are optional

Getting started

In order to use this online store, you'll need to run pip install 'feast[postgres]'. You can get started by then running feast init -t postgres.

Example

Functionality Matrix