Monitoring using Java Management Extensions (JMX)

The Oracle CDC connector can be monitored using metrics to gain insight into the connector and troubleshoot issues. These metrics can be queried using Java Management Extensions (JMX).

At the moment, metrics are only exposed for the connector task that captures redo records from the source database during the change data capture phase.

Note

Support for monitoring using JMX is available in version 2.6.0 and later.

Enabling Monitoring

You can enable monitoring using JMX by following the instructions in the Kafka documentation. The Kafka Monitoring documentation outlines the steps to enable monitoring using JMX in a containerized environment (Docker).

To enable the connector to emit JMX metrics, set enable.metrics.collection to true.

There are several open source tools available to monitor JMX, including the console based Jmxterm, JConsole that is part of the Java Development Kit (JDK), or load the metrics into a monitoring platform, such as Prometheus.

Available Metrics

The Oracle CDC connector publishes MBeans under the kafka.connect.oracle.cdc domain. The connector produces metrics using the following MBean naming format:

kafka.connect.oracle.cdc:type=source-task-metrics, context=<context-type>, connector=<connector-name>, task=<task-id>, name=<metric-name>

Where:

  • context-type groups the metrics into categories, such as “streaming” when processing change events
  • connector-name is the name of the connector
  • task-id is the numeric identifier of the connector task
  • metric-name is the name of the metric

The following table lists the metrics that are available.

Name Type Description
streaming-lag-from-source-in-milliseconds long The latency between when the database change was made and when the redo task completed processing this change. It is measured as the difference (in milliseconds) between the timestamp associated with the redo producer record and the timestamp from the redo record for the database change. Clock differences between the database and connect servers are accounted for in the calculation of this metric.
current-scn long The most recent system change number that has been processed.
offset-scn long The most recent system change number stored in the connector offsets topic.
last-processed-transaction-id string The identifier of the last processed (committed or rolled back) transaction. This is in the format of (XIDUSN.XIDSLT.XIDSQN). (Available only when record.buffer.mode is set to connector)
count-active-transactions-in-buffer long The count of active transactions in the transaction buffer. (Available only when record.buffer.mode is set to connector)
count-processed-transactions-in-cache long The count of processed transactions in the recently processed transactions cache. (Available only when record.buffer.mode is set to connector)
age-of-oldest-transaction-in-buffer-in-milliseconds long The age (milliseconds) of the oldest transaction in the transaction buffer. This is the difference between the timestamp of the most recent redo record that the connector has identified and the start timestamp of the oldest transaction. A high value that keeps increasing can be due to a long running transaction that is ongoing. Check the oldest-transaction-id-in-buffer metric to get the transaction ID and query the GV$TRANSACTION view to get more information about this transaction. (Available only when record.buffer.mode is set to connector)
oldest-scn-in-buffer long The system change number of oldest transaction in the transaction buffer. (Available only when record.buffer.mode is set to connector)
oldest-transaction-id-in-buffer string The identifier of the oldest transaction in the transaction buffer. This is in the format of (XIDUSN.XIDSLT.XIDSQN). (Available only when record.buffer.mode is set to connector)
logmining-session-start-scn long The start system change number of the most recent LogMiner session.
logmining-session-end-scn long The end system change number of the most recent LogMiner session (end SCN is -1 in case of continuous mining).
logmining-session-redo-log-files string The set of redo log files included in the most recent LogMiner session.
logmining-session-redo-log-files-count long The count of redo log files included in the most recent LogMiner session.