In v0.7, Feast Core no longer accepts starting with number (0-9) and using dash in names for:
Project
Feature Set
Entities
Features
Migrate all project, feature sets, entities, feature names:
with ‘-’ by recreating them with '-' replace with '_'
recreate any names with a number (0-9) as the first letter to one without.
Feast now prevents feature sets from being applied if no store is subscribed to that Feature Set.
Ensure that a store is configured to subscribe to the Feature Set before applying the Feature Set.
In v0.7, Feast Core's Job Coordinator has been decoupled from Feast Core and runs as a separate Feast Job Controller application. See its Configuration reference for how to configure Feast Job Controller.
Ingestion Job API
In v0.7, the following changes are made to the Ingestion Job API:
Changed List Ingestion Job API to return list of FeatureSetReference instead of list of FeatureSet in response.
Moved ListIngestionJobs, StopIngestionJob, RestartIngestionJob calls from CoreService to JobControllerService.
Python SDK/CLI: Added new Job Controller client and jobcontroller_url config option.
Users of the Ingestion Job API via gRPC should migrate by:
Add new client to connect to Job Controller endpoint to call JobControllerService and call ListIngestionJobs, StopIngestionJob, RestartIngestionJob from new client.
Migrate code to accept feature references instead of feature sets returned in ListIngestionJobs response.
Users of Ingestion Job via Python SDK (ie feast ingest-jobs list or client.stop_ingest_job() etc.) should migrate by:
ingest_job()methods only: Create a new separate Job Controller client to connect to the job controller and call ingest_job() methods using the new client.
Configure the Feast Job Controller endpoint url via jobcontroller_url config option.
Rename feast.jobs.consolidate-jobs-per-source property to feast.jobs.controller.consolidate-jobs-per-sources
Renamefeast.security.authorization.options.subjectClaim to feast.security.authentication.options.subjectClaim
Rename feast.logging.audit.messageLoggingEnabled to feast.audit.messageLogging.enabled
In Release 0.6 we introduced Flyway to handle schema migrations in PostgreSQL. Flyway is integrated into core and for now on all migrations will be run automatically on core start. It uses table flyway_schema_history in the same database (also created automatically) to keep track of already applied migrations. So no specific maintenance should be needed.
If you already have existing deployment of feast 0.5 - Flyway will detect existing tables and omit first baseline migration.
After core started you should have flyway_schema_history look like this
In this release next major schema changes were done:
Source is not shared between FeatureSets anymore. It's changed to 1:1 relation
and source's primary key is now auto-incremented number.
Due to generalization of Source sources.topics & sources.bootstrap_servers columns were deprecated.
They will be replaced with sources.config. Data migration handled by code when respected Source is used.
topics and bootstrap_servers will be deleted in the next release.
Job (table jobs) is no longer connected to Source (table sources) since it uses consolidated source for optimization purposes.
All data required by Job would be embedded in its table.
New Models (tables):
feature_statistics
Minor changes:
FeatureSet has new column version (see proto for details)
Connecting table jobs_feature_sets in many-to-many relation between jobs & feature sets
has now version and delivery_status.
For all versions earlier than 0.5 seamless migration is not feasible due to earlier breaking changes and creation of new database will be required.
Since database will be empty - first (baseline) migration would be applied: