ExtractTopic

Description

Extract data from a message, and use it as the topic name. Can either use the entire key/value (which should be a string), or use a field from a map or struct. Use the concrete transformation type designed for the record key (io.confluent.connect.transforms.ExtractTopic$Key) or value (io.confluent.connect.transforms.ExtractTopic$Value).

Installation

This transformation is developed by Confluent and does not ship by default with Apache Kafka or Confluent Platform. You can install this transformation via the Confluent Hub Client:

confluent-hub install confluentinc/connect-transforms:latest

Properties

Name Description Type Default Valid Values Importance
field Field name to use as the topic name. If left blank, the entire key or value is used (and assumed to be a string). string “”   medium
skip.missing.or.null How to handle missing fields and null fields, keys, and values. By default, this transformation will throw an exception if a field specified via the field configuration is missing or null, or if no field is specified but the message’s key or value is null. If this configuration is set to true, the transformation will instead silently ignore these conditions and allow the record to pass through unaltered. boolean false   low

Example

Below is a snippet of a configuration file for a standalone connector that demonstrates how to use and configure the ExtractTopic SMT.

transforms=requiredKeyExample, requiredValueFieldExample, optionalKeyFieldExample

# Use the key of the message as the topic name
transforms.requiredKeyExample.type=io.confluent.connect.transforms.ExtractTopic$Key

# Extract a required field named "f2" from the value, and use it as the topic name.
transforms.requiredValueFieldExample.type=io.confluent.connect.transforms.ExtractTopic$Value
transforms.requiredValueFieldExample.field=f2

# Extract a field named "f3" from the key, and use it as the topic name.
# If the field is null or missing, leave the topic name as-is.
transforms.optionalKeyFieldExample.type=io.confluent.connect.transforms.ExtractTopic$Value
transforms.optionalKeyFieldExample.field=f3
transforms.optionalKeyFieldExample.skip.missing.or.null=true