Kafka Connect Single Message Transformations Reference for Confluent Platform

A Single Message Transformation (SMT) modifies individual records as they flow through Connect, eliminating the need for a separate stream-processing application. SMTs process inbound records after a source connector produces them but before they are written to Apache Kafka®. Similarly, they process outbound records before they are delivered to a sink connector.

The following table outlines the SMTs available for Kafka Connect and their compatibility with specific connectors.

Tip

For a tutorial and a deep dive into this topic, see How to Use Single Message Transformations in Kafka Connect.

Transform	Description
Cast	Cast fields or the entire key or value to a specific type (for example, to force an integer field to a smaller width).
Drop	Drop either a key or a value from a record and set it to null.
DropHeaders	Drop one or more headers from each record.
EventRouter	Route Debezium outbox events using a regex configuration option.
ExtractField	Extract the specified field from a Struct when schema present, or a Map in the case of schemaless data. Any null values are passed through unmodified.
ExtractTopic	Replace the record topic with a new topic derived from its key or value.
Filter (Apache Kafka)	Drop all records. Designed to be used in conjunction with a Predicate.
Filter (Confluent)	Include or drop records that match a configurable filter.condition.
Flatten	Flatten a nested data structure. This generates names for each field by concatenating the field names at each level with a configurable delimiter character.
GzipDecompress	Not currently available for managed connectors. Gzip-decompress the entire byteArray key or value input.
HeaderFrom	Moves or copies fields in a record key or value into the record’s header.
HoistField	Wrap data using the specified field name in a Struct when schema present, or a Map in the case of schemaless data.
InsertField	Insert field using attributes from the record metadata or a configured static value.
InsertHeader	Insert a literal value as a record header.
MaskField	Mask specified fields with a valid null value for the field type.
MessageTimestampRouter	Update the record’s topic field as a function of the original topic value and the record’s timestamp field.
RegexRouter	Not currently available for managed connectors. Update the record topic using the configured regular expression and replacement string.
ReplaceField (Apache Kafka)	Filter or rename fields.
ReplaceField (Confluent)	Filter or rename nested fields.
SetSchemaMetadata	Set the schema name, version, or both on the record’s key or value schema.
TimestampConverter	Convert timestamps between different formats such as Unix epoch, strings, and Connect Date and Timestamp types.
TimestampRouter	Update the record’s topic field as a function of the original topic value and the record timestamp.
TimezoneConverter	Only available for Oracle XStream CDC source and Debezium connectors. Convert the timezone of timestamp fields in Debezium change event records to a specified target timezone.
TimescaleDB	Only available for Debezium PostgreSQL CDC V2 Source connectors. Consolidate change events from individual TimescaleDB chunk tables into logical hypertable topics and add TimescaleDB metadata to the record headers.
TombstoneHandler	Manage tombstone records. A tombstone record is defined as a record with the entire value field being null, whether or not it has ValueSchema.
ValueToKey	Replace the record key with a new key formed from a subset of fields in the record value.

If the available SMTs do not provide the necessary transformation, you can create your own SMT for Confluent Platform. For more information, see Create Custom Kafka Connect Single Message Transformations for Confluent Platform.