Kafka Connect TimescaleDB (Debezium) SMT Usage Reference for Confluent Cloud

The TimescaleDb (io.debezium.connector.v2.postgresql.transforms.timescaledb.TimescaleDb) open-source Debezium SMT (Single Message Transform) processes raw change event records captured from TimescaleDB databases. Use this SMT to perform logical routing, add relevant metadata, and streamline the handling of time-series data captured by the Debezium PostgreSQL connector.

For complete details, see the official Debezium TimescaleDB SMT documentation.

Limitations

  • The TimescaleDb SMT is only available for fully-managed Debezium PostgreSQL CDC V2 Source connectors.

  • The SMT requires a dedicated database connection to the TimescaleDB database for metadata lookups. It cannot reuse the connector-level database configuration.

Example

The following example shows how to configure and use the TimescaleDb SMT with a Debezium PostgreSQL CDC V2 Source connector.

"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "192.168.99.100",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname": "postgres",
"topic.prefix": "dbserver1",
"plugin.name": "pgoutput",
"schema.include.list": "_timescaledb_internal",
"transforms": "timescaledb",
"transforms.timescaledb.type": "io.debezium.connector.v2.postgresql.transforms.timescaledb.TimescaleDb",
"transforms.timescaledb.database.hostname": "timescaledb",
"transforms.timescaledb.database.port": "5432",
"transforms.timescaledb.database.user": "postgres",
"transforms.timescaledb.database.password": "postgres",
"transforms.timescaledb.database.dbname": "postgres",
"transforms.timescaledb.target.topic.prefix": "timescaledb",
"transforms.timescaledb.schema.list": "_timescaledb_internal"

Properties

The following table lists the configuration options that you can set for the SMT.

Name

Description

Type

Default

Importance

database.hostname

The hostname or IP address of the TimescaleDB metadata server used by the SMT for metadata lookups.

String

High

database.port

The port number of the TimescaleDB metadata server.

Int

5432

High

database.user

The database user for the TimescaleDB metadata connection.

String

High

database.password

The password for the TimescaleDB metadata connection.

String

High

database.dbname

The name of the TimescaleDB database containing the change data.

String

High

target.topic.prefix

The namespace prefix for the routed target topics.

String

timescaledb

Medium

schema.list

Comma-separated list of schema names that contain raw internal TimescaleDB data tables.

List

_timescaledb_internal

High

Predicates

Transformations can be configured with predicates so that the transformation is applied only to records which satisfy a condition. You can use predicates in a transformation chain and, when combined with the Kafka Connect Filter (Kafka) SMT Usage Reference for Confluent Cloud, predicates can conditionally filter out specific records. For details and examples, see Predicates.