To monitor data quality, we check that the characteristics of the tested dataset (aka the tested dataset's profile) are "equivalent" to the characteristics of the reference dataset. How exactly profile equivalency should be measured is up to the user.
validation_reference
can be passed as a parameter to methods .to_df(validation_reference=...)
or .to_arrow(validation_reference=...)
of RetrievalJob. If parameter is provided Feast will run validation once dataset is materialized. In case if validation successful materialized dataset is returned. Otherwise, feast.dqm.errors.ValidationFailed
exception would be raised. It will consist of all details for expectations that didn't pass.