Monitoring using Java Management Extensions (JMX)¶
The Oracle CDC connector can be monitored using metrics to gain insight into the connector and troubleshoot issues. These metrics can be queried using Java Management Extensions (JMX).
At the moment, metrics are only exposed for the connector task that captures redo records from the source database during the change data capture phase.
Note
Support for monitoring using JMX is available in version 2.6.0 and later.
Enabling Monitoring¶
You can enable monitoring using JMX by following the instructions in the Kafka documentation. The Kafka Monitoring documentation outlines the steps to enable monitoring using JMX in a containerized environment (Docker).
To enable the connector to emit JMX metrics, set enable.metrics.collection
to
true
.
There are several open source tools available to monitor JMX, including the console based Jmxterm, JConsole that is part of the Java Development Kit (JDK), or load the metrics into a monitoring platform, such as Prometheus.
Available Metrics¶
The Oracle CDC connector publishes MBeans under the kafka.connect.oracle.cdc
domain. The connector produces metrics using the following MBean naming format:
kafka.connect.oracle.cdc:type=source-task-metrics, context=<context-type>, connector=<connector-name>, task=<task-id>, name=<metric-name>
Where:
context-type
groups the metrics into categories, such as “streaming” when processing change eventsconnector-name
is the name of the connectortask-id
is the numeric identifier of the connector taskmetric-name
is the name of the metric
The following table lists the metrics that are available.
Name | Type | Description |
---|---|---|
streaming-lag-from-source-in-milliseconds |
long | The latency between when the database change was made and when the redo task completed processing this change. It is measured as the difference (in milliseconds) between the timestamp associated with the redo producer record and the timestamp from the redo record for the database change. Clock differences between the database and connect servers are accounted for in the calculation of this metric. |
current-scn |
long | The most recent system change number that has been processed. |
offset-scn |
long | The most recent system change number stored in the connector offsets topic. |
last-processed-transaction-id |
string | The identifier of the last processed (committed or rolled back)
transaction. (Available only when record.buffer.mode ) is set to
connector) |
count-active-transactions-in-buffer |
long | The count of active transactions in the transaction buffer. (Available
only when record.buffer.mode is set to connector) |
count-processed-transactions-in-cache |
long | The count of processed transactions in the recently processed
transactions cache. (Available only when record.buffer.mode is set to
connector) |
age-of-oldest-transaction-in-buffer-in-milliseconds |
long | The age (milliseconds) of the oldest transaction in the transaction
buffer. This is the difference between the current database timestamp and
the start timestamp of the oldest transaction in the transaction buffer.
A high value that keeps increasing can be due to a long running
transaction that is ongoing or due to an increased streaming lag from the
source. (Available only when record.buffer.mode is set to connector) |
oldest-scn-in-buffer |
long | The system change number of oldest transaction in the transaction buffer.
(Available only when record.buffer.mode is set to connector) |
oldest-transaction-id-in-buffer |
string | The identifier of the oldest transaction in the transaction buffer.
(Available only when record.buffer.mode is set to connector) |
logmining-session-start-scn |
long | The start system change number of the most recent LogMiner session. |
logmining-session-end-scn |
long | The end system change number of the most recent LogMiner session (end SCN is -1 in case of continuous mining). |
logmining-session-redo-log-files |
string | The set of redo log files included in the most recent LogMiner session. |