Kafka Connect TimestampRouter SMT for Confluent Platform

The following provides usage information for the Kafka SMT org.apache.kafka.connect.transforms.TimestampRouter.

Description

Update the record’s topic field as a function of the original topic value and the record timestamp.

This is useful for sink connectors, because the topic field often determines the equivalent entity name in the destination system (for example, a database table or search index name). See Kafka Connect MessageTimestampRouter SMT for Confluent Platform to extract the timestamp from a specified message field.

Example

This configuration snippet shows how to use TimestampRouter to route messages to topics and include a formatted timestamp.

"transforms": "TimestampRouter",
"transforms.TimestampRouter.type": "org.apache.kafka.connect.transforms.TimestampRouter",
"transforms.TimestampRouter.topic.format": "foo-${topic}-${timestamp}",
"transforms.TimestampRouter.timestamp.format": "YYYYMM"

Before: ordersTopic

After: foo-ordersTopic-201804

Tip

For additional examples, see Timestamp Router for managed connectors.

Properties

Name Description Type Default Valid Values Importance
timestamp.format Format string for the timestamp that is compatible with java.text.SimpleDateFormat. For additional details, see SimpleDateFormat. string yyyyMMdd   high
topic.format Format string which can contain ${topic} and ${timestamp} as placeholders for the topic and timestamp, respectively. string ${topic}-${timestamp}   high

Predicates

Transformations can be configured with predicates so that the transformation is applied only to records which satisfy a condition. You can use predicates in a transformation chain and, when combined with the Kafka Connect Filter (Kafka) SMT for Confluent Platform, predicates can conditionally filter out specific records. For details and examples, see Predicates.