Kafka Connect TombstoneHandler SMT Usage Reference for Confluent Platform

The TombstoneHandler SMT (io.confluent.connect.transforms.TombstoneHandler) manages tombstone records within a Connect data pipeline.

Description

A tombstone record is any Apache Kafka® record whose entire value field is null, regardless of whether a value schema is present.

Many downstream systems and sink connectors cannot natively process empty payloads without throwing exceptions. This SMT provides a mechanism to intercept them. Depending on your configuration, the TombstoneHandler can be set to silently ignore, log a warning, or fail the execution when a tombstone record is processed.

Installation

This transformation is developed by Confluent and does not ship by default with Kafka or Confluent Platform. You can install this transformation using the confluent connect plugin install command:

confluent connect plugin install confluentinc/connect-transforms:latest

Properties

Name	Description	Type	Default	Valid values	Importance
`behavior`	If set to `ignore`, the tombstone record is ignored silently, with a DEBUG message written to log. If set to `warn`, the tombstone record is ignored silently, with a WARN message written to log. If set to `fail`, the execution fails, a `DataException` is thrown, and a DEBUG message is written to log.	string	warn	[ignore, warn, fail]	medium

Examples

The following snippets show how to configure the TombstoneHandler SMT for each supported behavior value:

Behavior	Description
`warn` (default)	Drops the tombstone record and logs a WARN message.
`ignore`	Drops the tombstone record and logs a DEBUG message.
`fail`	Fails execution, throws a `DataException`, and logs a DEBUG message.

Handle tombstones with the default (warn) behavior

"transforms": "tombstoneHandlerExample",
"transforms.tombstoneHandlerExample.type": "io.confluent.connect.transforms.TombstoneHandler"

Ignore tombstone records

"transforms.tombstoneHandlerExample.type": "io.confluent.connect.transforms.TombstoneHandler",
"transforms.tombstoneHandlerExample.behavior": "ignore"

Fail on tombstone records

"transforms.tombstoneHandlerExample.type": "io.confluent.connect.transforms.TombstoneHandler",
"transforms.tombstoneHandlerExample.behavior": "fail"

Predicates

Configure transformations with predicates to ensure they process only records that satisfy a particular condition. You can also use predicates in a transformation chain with the Kafka Connect Filter (Kafka) SMT Usage Reference for Confluent Platform to conditionally filter specific records. For more information, see Predicates.