For the complete documentation index, see llms.txt. This page is also available as Markdown.

Data types

Feast frequently has to mediate data across platforms and systems, each with its own unique type system. To make this possible, Feast itself has a type system for all the types it is able to handle natively.

Feast's type system is built on top of protobuf. The messages that make up the type system can be found here, and the corresponding python classes that wrap them can be found here.

Feast supports the following categories of data types:

  • Primitive types: numerical values (Int32, Int64, Float32, Float64), String, Bytes, Bool, and UnixTimestamp.

  • Zoned timestamp type: ZonedTimestamp stores a timezone-aware datetime as both the UTC instant and its originating zone, so the original wall-clock zone round-trips losslessly. This differs from UnixTimestamp, which is always decoded as UTC and discards the source zone. Use ZonedTimestamp when local time-of-day or the offset/zone itself is meaningful. It must be explicitly declared in schema (it is not inferred by any backend), and is not supported as an entity key.

  • Domain-specific primitives: PdfBytes (PDF binary data for RAG/document pipelines) and ImageBytes (image binary data for multimodal pipelines). These are semantic aliases over Bytes and must be explicitly declared in schema — no backend infers them.

  • UUID types: Uuid and TimeUuid for universally unique identifiers. Stored as strings at the proto level but deserialized to uuid.UUID objects in Python.

  • Array types: ordered lists of any primitive type, e.g. Array(Int64), Array(String), Array(Uuid).

  • Set types: unordered collections of unique values for any primitive type, e.g. Set(String), Set(Int64). Set types are not inferred by any backend and must be explicitly declared. They are best suited for online serving use cases.

  • Map types: dictionary-like structures. Map has string keys and values that can be any supported Feast type (including nested maps), e.g. Map, Array(Map). ScalarMap has non-string scalar keys (int, float, bool, UUID, Decimal, bytes, datetime) — Feast infers ScalarMap automatically when the first key is not a string. ScalarMap must be explicitly declared in schema and is not inferred by any backend.

  • JSON type: opaque JSON data stored as a string at the proto level but semantically distinct from String — backends use native JSON types (jsonb, VARIANT, etc.), e.g. Json, Array(Json).

  • Struct type: schema-aware structured type with named, typed fields. Unlike Map (which is schema-free), a Struct declares its field names and their types, enabling schema validation, e.g. Struct({"name": String, "age": Int32}).

For a complete reference with examples, see Type System.

Each feature or schema field in Feast is associated with a data type, which is stored in Feast's registry. These types are also used to ensure that Feast operates on values correctly (e.g. making sure that timestamp columns used for point-in-time correct joins actually have the timestamp type).

As a result, each system that Feast interacts with needs a way to translate data types from the native platform into a Feast type. E.g., Snowflake SQL types are converted to Feast types here. The onus is therefore on authors of offline or online store connectors to make sure that this type mapping happens correctly.

Backend Type Mapping for Complex Types

Map, JSON, and Struct types are supported across all major Feast backends:

Backend
Native Type
Feast Type

PostgreSQL

jsonb

Map, Json, Struct

PostgreSQL

jsonb[]

Array(Map)

Snowflake

VARIANT, OBJECT

Map

Snowflake

JSON

Json

Redshift

SUPER

Map

Redshift

json

Json

BigQuery

JSON

Json

BigQuery

STRUCT, RECORD

Struct

Spark

map<string,string>

Map

Spark

array<map<string,string>>

Array(Map)

Spark

struct<...>

Struct

Spark

array<struct<...>>

Array(Struct(...))

MSSQL

nvarchar(max)

Map, Json, Struct

DynamoDB

Proto bytes

Map, Json, Struct, ScalarMap

Redis

Proto bytes

Map, Json, Struct, ScalarMap

Milvus

VARCHAR (serialized)

Map, Json, Struct

Note: When the backend native type is ambiguous (e.g., jsonb could be Map, Json, or Struct), the schema-declared Feast type takes precedence. The backend-to-Feast type mappings above are only used for schema inference when no explicit type is provided.

Note: Feast currently does not support a null type in its type system.

Last updated

Was this helpful?