v0.26-branch
Search…
⌃K

Spark (contrib)

Description

The Spark batch materialization engine is considered alpha status. It relies on the offline store to output feature values to S3 via to_remote_storage, and then loads them into the online store.
See SparkMaterializationEngine for configuration options.

Example

feature_store.yaml
...
offline_store:
type: snowflake.offline
...
batch_engine:
type: spark.engine
partitions: [optional num partitions to use to write to online store]