Logical Types

Schema Registry supports Logical Types, which let you represent a schema using SQL types, not the syntax of a serialization format. For example, this Avro schema:

{
  "type": "record",
  "name": "Person",
  "fields": [
    {"name": "id", "type": "string"},
    {"name": "name", "type": "string"},
    {"name": "age", "type": ["null", "int"]},
    {"name": "active", "type": "boolean"}
  ]
}

becomes this Logical Type:

STRUCT Person (
  id STRING NOT NULL,
  name STRING NOT NULL,
  age INT,
  active BOOLEAN NOT NULL
)

Register the Logical Type the same way as any other schema:

curl -X POST -H "Content-Type: application/json" \
  --data '{
    "schemaType": "AVRO",
    "schema": "STRUCT Person ( id STRING NOT NULL, name STRING NOT NULL, age INT, active BOOLEAN NOT NULL )"
  }' \
  'http://localhost:8081/subjects/mytopic-value/versions'

schemaType still must be AVRO, PROTOBUF, or JSON, because this indicates which serialization format clients use. If you retrieve the preceding schema, it matches the Avro form shown earlier. Specifying PROTOBUF instead returns:

syntax = "proto3";

message Person {
  string id = 1;
  string name = 2;
  optional int32 age = 3;
  bool active = 4;
}

And specifying JSON returns:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "additionalProperties": false,
  "properties": {
    "id": {"type": "string"},
    "name": {"type": "string"},
    "age": {"oneOf": [{"type": "null"}, {"type": "number"}]},
    "active": {"type": "boolean"}
  },
  "required": ["id", "name", "active"],
  "title": "Person",
  "type": "object"
}

Retrieve a schema in Logical Type form

To retrieve the Logical Type form of a schema, pass ?format=LOGICAL on the request:

curl -H "Content-Type: application/json" \
  'http://localhost:8081/subjects/mytopic-value/versions/1?format=LOGICAL'

Every endpoint that returns a schema accepts the format parameter, not only version reads:

Registering a schema
Subject and version lookups
The by-ID endpoint under /schemas
The by-GUID endpoint under /schemas

Input works the other way: Schema Registry detects a Logical Type from the request body. It first tries to parse the content as the declared schemaType and falls back to Logical Type syntax only if that parse fails. Because detection is content-driven, you can send a Logical Type anywhere you can send a schema, including the compatibility endpoints for a single version (/compatibility/subjects/<subject>/versions/<version>) and for all versions (/compatibility/subjects/<subject>/versions). This flexibility lets you check whether Schema Registry would accept a Logical Type before you register it:

curl -X POST -H "Content-Type: application/json" \
  --data '{
    "schemaType": "AVRO",
    "schema": "STRUCT Person ( id STRING NOT NULL, name STRING NOT NULL, age BIGINT, active BOOLEAN NOT NULL )"
  }' \
  'http://localhost:8081/compatibility/subjects/mytopic-value/versions/latest'

Namespaces, ENUM, and nested STRUCT

Logical Types can represent most of what any of the serialization formats can express. This example declares a NAMESPACE, an ENUM, and two STRUCT types, one of which references the other:

NAMESPACE com.example.demo;

ENUM Status ('ACTIVE', 'PAUSED', 'CANCELLED');

STRUCT Address (
  street STRING NOT NULL,
  city STRING NOT NULL,
  postal_code VARCHAR(16)
);

STRUCT Order (
  id STRING NOT NULL,
  retries INT NOT NULL DEFAULT 0,
  status Status NOT NULL,
  shipping Address,
  billing Address
);

An ENUM symbol is a quoted string literal, so the grammar itself allows any character. Schema Registry doesn’t check symbol characters against the target format at parse time. It passes each symbol straight to that format’s own schema builder, which enforces its own naming rules. Avro and Protobuf both require symbols to be valid identifiers, so a symbol like 'in-progress' fails at registration for either one. JSON Schema has no such restriction.

For Protobuf, symbols number in declaration order starting at 0. To set a symbol’s number explicitly, add a WITH clause after that symbol, for example ENUM Status ('UNKNOWN' WITH ('protobuf.enum.number' = '0'), 'ACTIVE' WITH ('protobuf.enum.number' = '1')). proto3 requires the first declared symbol to be number 0, and every number must be unique. Number 0 is also proto3’s default value for an unset enum field. An unset field then silently reads as the symbol numbered 0, ACTIVE in the Order example’s Status, even though status is NOT NULL. Declare a sentinel symbol first, such as 'UNKNOWN', if that default matters to readers of the data.

Because numbering follows declaration order, inserting, removing, or reordering symbols renumbers every symbol after the change. Existing Protobuf-encoded data then decodes as different symbols. Add new symbols only at the end and keep existing ones, or number every symbol explicitly. This also applies to a sentinel symbol: declare it in the first version, because adding it first to an existing ENUM renumbers every other symbol.

When a script declares more than one named type, Schema Registry infers the root schema as a declared type that no other declared type references. In this example, that’s Order, because Status and Address are both referenced by it. When more than one declared type has no incoming references, the first one declared becomes the root. The rest remain as peer named types in Protobuf and JSON Schema, which both support more than one named type per schema. Avro allows only one top-level type, so the Avro output entirely drops a declared type that the root doesn’t reference. When every declared type has an incoming reference from another declared type, forming a cycle, Schema Registry can’t infer a root and rejects the script.

To set the root explicitly instead of relying on this inference, add a trailing TYPE Order NOT NULL; statement after the declarations. If the inferred root is an ENUM, registering it for Protobuf fails, because Protobuf has no message-less root. Use the ENUM as a field of a STRUCT and let that STRUCT be the root, by inference or with TYPE.

This Order example corresponds to the following Avro schema. The Avro form is more verbose, and nullable fields translate to unions, as Avro requires:

{
  "type": "record",
  "name": "Order",
  "namespace": "com.example.demo",
  "fields": [
    {"name": "id", "type": "string"},
    {"name": "retries", "type": "int", "default": 0},
    {"name": "status", "type": {
      "type": "enum", "name": "Status",
      "symbols": ["ACTIVE", "PAUSED", "CANCELLED"]
    }},
    {"name": "shipping", "type": ["null", {
      "type": "record", "name": "Address",
      "fields": [
        {"name": "street", "type": "string"},
        {"name": "city", "type": "string"},
        {"name": "postal_code", "type": ["null",
          {"type": "string", "flink.maxLength": 16}]}
      ]
    }]},
    {"name": "billing", "type": ["null", "Address"]}
  ]
}

Logical Types can be user-defined (Status, Order, and Address in the preceding example, using STRUCT or ENUM) or built-in, corresponding to SQL types such as INT. To register a built-in type on its own, such as when using a simple integer as a message key’s schema, use TYPE:

TYPE INT NOT NULL

This declaration corresponds to {"type": "int"} in Avro. Types are nullable by default, so specify NOT NULL to get a non-nullable type. Without it, TYPE INT corresponds to ["null", "int"].

Logical Types can also express unions:

STRUCT IdAndPayload (
  id STRING NOT NULL,
  payload UNION (
    text STRING NOT NULL,
    number BIGINT NOT NULL,
    blob BINARY(16) NOT NULL
  ) NOT NULL
);

A UNION translates to an Avro union, a Protobuf oneof, or a JSON Schema oneOf. Every branch in the preceding example is NOT NULL because a nullable branch would become a nested union, and Avro doesn’t support that. Avro also rejects a union with two unnamed branches of the same Avro type. Each branch must map to a distinct Avro type, not only use a distinct name or Logical Type. For example, INT, SMALLINT, and DATE all map to Avro int, and CHAR(n), VARCHAR(n), and STRING all map to string. The same rule covers collections, whatever their element types. An Avro union can hold at most one array and one map. ARRAY<T>, MULTISET<T>, and a MAP<K,V> with a non-string key all become Avro arrays, so any two of them conflict. For Avro, Schema Registry rejects such a schema at registration. It doesn’t silently collapse or rename a branch.

JSON Schema has no matching check. Two branches that map to the same JSON type, such as text STRING and blob BINARY(16) in the preceding example (both become string), register without error. oneOf passes only when exactly one branch matches a value, so a JSON Schema validator rejects every string value for that UNION, because a string always matches both branches. number values, such as the number BIGINT branch, still validate. The confluent:union branch names in the schema don’t prevent this. When you register a UNION as JSON, make sure each branch maps to a distinct JSON type.

The UNION itself can also be nullable, unlike its branches. For example, payload UNION (...) without NOT NULL doesn’t nest a ["null", [...]] union, which Avro forbids. Schema Registry splices null into the existing branch list instead, producing ["null", "string", "long", {"type": "fixed", "name": "blob", "size": 16}] for the preceding example.

A BINARY(n) branch, such as blob in the example, becomes an Avro fixed type named after the branch, here blob. This name isn’t qualified by the enclosing struct or union. If two different unions in the same schema each declare a same-named BINARY(n) branch with different sizes, Schema Registry rejects the schema at registration. The two conflict under the same Avro fixed-type name.

Schema references for Logical Types

Logical Types can refer to other named types. Schema Registry stores every Logical Type in its converted native form, the same way it stores any other schema, so the referenced schema must use the same schemaType as the referencing one. It can be registered either as a Logical Type or as a native schema of that type. If a type name isn’t built-in or locally defined as a STRUCT or ENUM, Schema Registry treats it as a reference to an external Logical Type. This also applies to a misspelled built-in or local type name, for example BIGIN for BIGINT. Schema Registry doesn’t reject it as an unknown type. It treats the misspelling as an external reference, so it fails with a missing- or unresolved-reference error. If you see that error on a name you expected Schema Registry to treat as built-in or local, check it for a typo before adding a schema reference. Register the Logical Type with schema references to the external types:

curl -X POST -H "Content-Type: application/json" \
  --data '{
    "schemaType": "AVRO",
    "schema": "STRUCT Order ( id STRING NOT NULL, quantity INT NOT NULL, ship_to com.example.demo.Address NOT NULL )",
    "references": [
      {"name": "com.example.demo.Address", "subject": "address", "version": 1}
    ]
  }' \
  'http://localhost:8081/subjects/orders-value/versions'

The simplest reference is a bare reference:

TYPE com.example.demo.Address NOT NULL

Use TYPE here, not STRUCT, because STRUCT only defines a new user-defined type. This TYPE reference points to a STRUCT already defined elsewhere. Types are nullable by default here too, so specify NOT NULL to get a non-nullable reference. Register it the same way as any other Logical Type, including the references array. For Avro, a bare reference resolves to the string "com.example.demo.Address". For Protobuf, it resolves to an import public statement, an otherwise-empty file that re-exports the referenced message. Schema Registry resolves through the import public when reading the schema back, the same way a Protobuf serializer or deserializer does, so the bare reference works as a value or key schema. For JSON Schema, it resolves to a $ref pointing into $defs.

Struct and field clauses

Structs and fields can carry the following clauses. NOT NULL also applies to any other type position: standalone TYPE declarations, a UNION and each of its branches, and collection element, key, and value types. DEFAULT applies only to fields. The rest apply to both structs and fields:

NOT NULL: marks the value as required, disallowing null
DEFAULT: field only, a default value for primitive types
CHECK: a struct-level or field-level constraint. Write a field-level CHECK after the field’s type. Write a struct-level CHECK as its own comma-separated entry inside the struct’s parentheses, not after them with COMMENT, TAGS, and WITH. Optionally follow either form with MESSAGE '<text>' to set the error message returned when the constraint fails
COMMENT: a human-readable description of a struct or field
TAGS: labels used to select structs or fields for rule targeting, such as PII
WITH: key-value parameters for struct- or field-level metadata

DEFAULT becomes the native default field for Avro and JSON Schema. proto3 has no field defaults, so for Protobuf, DEFAULT becomes a connect.default parameter under (confluent.field_meta), letting ?format=LOGICAL restore it. This parameter is metadata only. A Protobuf deserializer still reads an unset field as the type’s zero value, such as 0 for INT, not as the declared default.

For Avro, a nullable field with no DEFAULT clause gets no default at all, not an implicit null default, as the preceding examples show. Adding such a field in a later version breaks BACKWARD compatibility, because Avro schema resolution has nothing to fill in for old data. When you add a nullable field in a later version, give it DEFAULT NULL.

COMMENT becomes the native doc field or comment for the target format. For Avro and JSON Schema, TAGS becomes a confluent:tags property and WITH parameters become a confluent:params property. For Protobuf, both become .tags and .params sub-fields of the (confluent.field_meta) option on a field, or of the (confluent.message_meta) option on a struct. Format-native keys are the exception: any key prefixed avro. or protobuf., such as avro.aliases. For the format they name, they map to a native attribute, as in the ssn field’s alias in the following example. For every other format, Schema Registry drops them. They never appear as an ordinary parameter. Schema Registry also reserves any key prefixed connect., flink., or logical., such as connect.default, for the same reason but a different mechanism: it populates them itself to round-trip a value it already stores structurally, such as a field’s default. Setting one directly in WITH isn’t supported.

CHECK compiles to a CEL expression and becomes a domain rule in the schema’s ruleSet. Schema Registry doesn’t run this check at registration, and it never sees record data. The client serializer or deserializer enforces the rule when it runs data contract rules. A client that ignores ruleSet, such as an older serializer or a non-Confluent client, writes data that violates the constraint with no error.

Because CHECK produces a schema rule, it’s subject to the same requirements as any other rule. Schema Registry must run on Confluent Enterprise with resource.extension.class=io.confluent.kafka.schemaregistry.rulehandler.RuleSetResourceExtension set. Clients run rules only for the root schema, not for referenced schemas. For example, the CHECK constraints on Person in the following example don’t run when you serialize the Employee type shown in the next section, which references Person. Flink SQL doesn’t run rules.

A field-level CHECK’s compiled rule skips evaluation entirely when that field’s value is null or absent, so a null value always passes, the same outcome as a SQL CHECK on a NULL column. The ssn and age checks in the following example never run against a null value, so pair a field-level CHECK with a struct-level one, such as age IS NOT NULL OR ssn IS NOT NULL, to require at least one of them to be present.

NAMESPACE com.example.common;

STRUCT Person (
  id BIGINT NOT NULL
    COMMENT 'Primary key',
  ssn STRING
    CHECK (MATCHES(ssn, '^[0-9]{3}-[0-9]{2}-[0-9]{4}$'))
      MESSAGE 'ssn must look like 123-45-6789'
    COMMENT 'US social security number'
    TAGS ('PII')
    WITH ('avro.aliases' = 'social_security_number'),
  email STRING
    COMMENT 'Contact address'
    TAGS ('PII', 'CONTACT'),
  age INT
    CHECK (age BETWEEN 0 AND 150),
  CHECK (age IS NOT NULL OR ssn IS NOT NULL) MESSAGE 'need age or ssn'
)
COMMENT 'A person record'
TAGS ('INTERNAL')
WITH ('owner' = 'hr-team', 'retention.days' = '365');

Local aliases with USING

A USING directive declares a local alias for a type name. A short alias improves readability when a user-defined type isn’t in the current namespace:

NAMESPACE com.example.hr;

USING TYPE Person FOR com.example.common.Person;

STRUCT Employee (
  id BIGINT NOT NULL,
  person Person
);

Person isn’t built-in or locally defined as a STRUCT or ENUM in this script, so it’s still an external reference. USING only shortens the name used in the syntax. You still add com.example.common.Person to the references array when you register this schema.

Logical Type mappings

Avro also has its own, unrelated logicalType schema attribute, which the following table uses to encode some Logical Types. The table shows how built-in Logical Types map to Avro, Protobuf, and JSON Schema.

Logical Type	Avro	Protobuf	JSON Schema
`BOOLEAN`	`boolean`	`bool`	`boolean`
`TINYINT`, `SMALLINT`, and `INT`	`int`. `TINYINT` and `SMALLINT` also set `connect.type` to `int8` or `int16`, so `?format=LOGICAL` can restore the original type	`int32`	`number`
`BIGINT`	`long`	`int64`	`number`
`FLOAT`	`float`	`float`	`number`
`DOUBLE`	`double`	`double`	`number`
`DECIMAL(p,s)`	`bytes`, using `logicalType=decimal`, `precision`, and `scale`	`confluent.type.Decimal`	`number`
`CHAR(n)`, `VARCHAR(n)`, and `STRING`	`string`. `CHAR(n)` also sets `flink.minLength`, and both `CHAR(n)` and `VARCHAR(n)` set `flink.maxLength`, so `?format=LOGICAL` can restore the original length	`string`. Sets the same `flink.minLength` and `flink.maxLength` parameters as Avro, as sub-fields of `(confluent.field_meta)`	`string`. `CHAR(n)` sets native `minLength` and `maxLength`. `VARCHAR(n)` sets only `maxLength`
`BINARY(n)`	`fixed`, using `size=n`	`bytes`	`string`
`VARBINARY(n)` and `BYTES`	`bytes`	`bytes`	`string`
`DATE`	`int`, using `logicalType=date`	`google.type.Date`	`number`
`TIME(n)`	`int` using `logicalType=time-millis` for `n` 0-3, or `long` using `time-micros` for `n` 4-9. Avro has no nanosecond time type, so `n` 7-9 loses precision beyond microseconds, unlike `TIMESTAMP`	`google.type.TimeOfDay`	`number`
`TIMESTAMP(n)`	`long`, using `logicalType=local-timestamp-millis` for `n` 0-3, `local-timestamp-micros` for 4-6, or `local-timestamp-nanos` for 7-9	`google.protobuf.Timestamp`	`number`
`TIMESTAMP_LTZ(n)`	`long`, using `logicalType=timestamp-millis` for `n` 0-3, `timestamp-micros` for 4-6, or `timestamp-nanos` for 7-9	`google.protobuf.Timestamp`	`number`
`ARRAY<T>`	`array`, using `items`	`repeated <ElementWrapper> message` when `T` is nullable, a collection, or a `UNION`. Otherwise, `repeated T`. When the field itself is nullable, `<RepeatedWrapper> message` wraps the whole `repeated` field	`array`, using `items`
`MULTISET<T>`	Array of `Entry` record, using `logicalType=map` and `flink.type=multiset`. Entry fields are `key` and `value`, the value holding each element’s count	`repeated <Entry> message`	Array of key-value objects
`MAP<K,V>`	`map`, using `values`, when `K` is `CHAR`, `VARCHAR`, or `STRING` (`STRING` is `VARCHAR` with no length limit). Otherwise, an array of `Entry` records with `key` and `value` fields	`repeated <Entry> message`. When the field itself is nullable, `<RepeatedWrapper> message` wraps the whole `repeated` field	`object`, using `additionalProperties`, when `K` is `CHAR`, `VARCHAR`, or `STRING`. Otherwise, an array of key-value objects
`ROW (...)` and inline `STRUCT (...)`	`record` with a generated name, for example `T_f`, tagged `logical.anonymous`	Nested message, for example `.T.FRow`	`object`, using `properties` and `title`
`UNION (n T, ...)`	Avro union, for example `["int","string"]`	`oneof`	`oneOf` plus `confluent:union` branch names
`VARIANT`	`confluent.type.Variant` record, using `logicalType=variant`	`confluent.type.Variant`	`{}`
`STRUCT Name (...)`	Named record	`.Name` message	Top-level `object` with `title: Name` for the root struct. Otherwise, `$ref: #/$defs/Name`
`ENUM Name (...)`	`enum`, using `symbols`	`.Name` enum. Protobuf has no message-less root, so an `ENUM` can’t be the root. Register it as a field or branch of a `STRUCT` instead	Top-level schema using `enum` with `title: Name` for the root `ENUM`. Otherwise, `$ref: #/$defs/Name`

The JSON Schema column uses number for every integer and temporal Logical Type, including TINYINT through BIGINT, DATE, TIME(n), and the TIMESTAMP types. JSON Schema validation against number doesn’t reject a fractional value, unlike the matching Avro and Protobuf forms. None of the three formats rejects a TINYINT or SMALLINT value outside that type’s range, because Avro and Protobuf store both as a 32-bit integer.

The generated name for an inline ROW or STRUCT, shown in the preceding table as T_f for Avro and .T.FRow for Protobuf, combines its enclosing type’s name and its own field name in both formats. Renaming either one changes this generated name. Because Avro and Protobuf compatibility checks match nested records or messages by name, even a rename that’s normally safe for the field itself, such as an Avro field with avro.aliases or a Protobuf field that keeps its tag, can break compatibility for the nested type. To keep the nested name stable, declare the nested type as a named STRUCT instead of an inline ROW or STRUCT. JSON Schema isn’t affected, because its title isn’t used for compatibility matching.

For Protobuf, an ARRAY<T> element gets the ElementWrapper shown in the mapping table when T is nullable, which it is by default, or when T is a nested collection or UNION, whether or not it’s nullable. Write ARRAY<STRING NOT NULL> to keep a plain repeated string on the wire. Avro and JSON Schema need no wrapper either way: a nullable element becomes a union or oneOf inside items, the same as any other nullable field.

An ARRAY<T> field’s own nullability is different: proto3 forbids optional repeated, so a nullable ARRAY, MAP, or MULTISET field wraps the whole repeated value in a <RepeatedWrapper> message instead, shown in the mapping table. Presence on that wrapper message distinguishes a null array from an empty one, because a message field can be unset independent of the repeated value inside it. Write ARRAY<T> NOT NULL to keep a plain repeated field with no wrapper. Avro and JSON Schema need no wrapper either way: a null array is already a distinct value from an empty one in both formats.

MULTISET<T> and MAP<K,V> don’t use ElementWrapper for their element or value type, because their Entry message’s key and value fields are ordinary struct fields, not array elements. A nullable scalar V becomes an optional field on Entry. A collection or UNION as V triggers the same RepeatedWrapper or OneofWrapper a nullable field of that shape gets anywhere else, because Entry’s fields go through the same field-conversion rules as any other struct’s. For MULTISET<T>, the element T is the Entry message’s key field and follows the same rule, and value holds the count.

For the native map and additionalProperties forms, a MAP<K,V> key’s own NOT NULL clause has no effect on which form it takes. These forms require a plain CHAR, VARCHAR, or STRING key type, regardless of NOT NULL, so MAP<STRING, V> and MAP<STRING NOT NULL, V> produce the same native form. Keys in these forms can’t be null, so a record with a null key fails at serialization.