Monitoring Oracle XStream CDC Source Connector for Confluent Platform¶
The Oracle XStream CDC Source connector exposes several metrics through JMX (Java Management Extensions) that are useful for monitoring connector and troubleshooting issues.
Enable monitoring¶
You can enable monitoring using JMX by following the instructions in the Kafka documentation. The Kafka Monitoring documentation outlines the steps to enable monitoring using JMX in a containerized environment (Docker).
You can use open-source tools to monitor JMX, including the console-based Jmxterm, JConsole that is part of the Java Development Kit (JDK), or load the metrics into a monitoring platform, such as Prometheus.
Connector metrics¶
The Oracle XStream CDC Source connector publishes MBeans under the debezium.confluent.oracle domain. The
connector provides three metric types:
Snapshot metrics¶
Snapshot metrics help you monitor the connector during snapshot operations. These metrics are only available if a snapshot is currently active or if a snapshot has been performed since the connector was last started.
MBean: debezium.confluent.oracle:type=connector-metrics,context=snapshot,server=<topic.prefix>
| Attribute | Type | Description |
|---|---|---|
| CapturedTables | string[] | The list of tables captured by the connector. |
| RemainingTableCount | int | The number of tables remaining to be snapshotted. |
| TotalTableCount | int | The total number of tables included in the snapshot. |
| RowsScanned | tabular | Map containing the number of rows scanned for each table in the snapshot. Tables are incrementally added to the map during processing. Updates every 10 seconds and once a table’s snapshot is complete. |
| LastEvent | string | The most recent snapshot event that the connector has processed and dispatched. |
| MilliSecondsSinceLastEvent | long | The number of milliseconds since the connector last read and processed the most recent event, including those filtered out. |
| TotalNumberOfEventsSeen | long | The total number events the connector has encountered since its last start or reset, including those filtered out. |
| NumberOfEventsFiltered | long | The number of events filtered out based on the include/exclude list rules configured in the connector. |
| QueueRemainingCapacity | int | The remaining capacity of the queue that passes events from the snapshot source to the main Kafka Connect loop. |
| QueueTotalCapacity | int | The total capacity of the queue that passes events from the snapshot source to the main Kafka Connect loop. |
| SnapshotRunning | boolean | Whether the snapshot has started and is actively running. |
| SnapshotCompleted | boolean | Whether the snapshot has been completed. |
| SnapshotPaused | boolean | Whether the snapshot has been paused. |
| SnapshotAborted | boolean | Whether the snapshot has been aborted. |
| SnapshotDurationInSeconds | long | The total running time for the snapshot, in seconds, including when it was paused. |
| SnapshotPausedDurationInSeconds | long | The total paused time for the snapshot, in seconds. The pause time adds up from all the times the snapshot was paused. |
Streaming metrics¶
Streaming metrics help you monitor the connector when streaming changes from the database.
MBean: debezium.confluent.oracle:type=connector-metrics,context=streaming,server=<topic.prefix>
| Attribute | Type | Description |
|---|---|---|
| CapturedTables | string[] | The list of tables captured by the connector. |
| Connected | boolean | Whether the connector is currently streaming changes from the database server. |
| LastEvent | string | The most recent streaming event that the connector has processed and dispatched. |
| MilliSecondsSinceLastEvent | long | The number of milliseconds since the connector last read and processed the most recent event, including those filtered out. |
| TotalNumberOfEventsSeen | long | The total number events the connector has seen since its last start or reset, including those filtered out. |
| TotalNumberOfCreateEventsSeen | long | The total number create events the connector has processed since its last start or reset, including those filtered out. |
| TotalNumberOfUpdateEventsSeen | long | The total number update events the connector has processed since its last start or reset, including those filtered out. |
| TotalNumberOfDeleteEventsSeen | long | The total number delete events the connector has processed since its last start or reset, including those filtered out. |
| NumberOfEventsFiltered | long | The number of events filtered out based on the include/exclude list rules configured in the connector. |
| QueueRemainingCapacity | int | The remaining capacity of the queue that passes events from the streaming source to the main Kafka Connect loop. |
| QueueTotalCapacity | int | The total capacity of the queue that passes events from the streaming source to the main Kafka Connect loop. |
| SourceEventPosition | tabular | The coordinates of the most recent streaming event that the connector has processed and dispatched. |
| LastTransactionId | string | Transaction identifier of the last transaction that the connector has processed and dispatched. |
| NumberOfCommittedTransactions | long | The number of committed transactions that the connector has processed and dispatched. |
| MilliSecondsBehindSource | long | The number of milliseconds between the timestamp of the last change event and the time the connector processed it. This metric is computed based on the time when the change was generated in the source database’s redo log. Long-running transactions can cause this metric to show high values, since the connector only receives the transaction changes after the transaction is committed in the source database. |
| CommitMilliSecondsBehindSource | long | The number of milliseconds between the timestamp of the last COMMIT event received and the time the connector processed it.This metric is only available on Confluent Platform. |
| TotalSchemaChangeParseErrorCount | int | The number of DDL statements that could not be parsed by the DDL parser. This number should always be 0. However, if the connector allows skipping unparseable DDL statements, this metric helps determine if the connector has logged any warnings. |
Note
The source event timestamps for the MilliSecondsBehindSource and CommitMilliSecondsBehindSource metrics are in the database server’s operating system timezone. Please ensure that the database.os.timezone connector configuration is set to the database server’s operating system timezone.
Schema history metrics¶
Schema history metrics help you monitor the status of the connector’s schema history.
MBean: debezium.confluent.oracle:type=connector-metrics,context=schema-history,server=<topic.prefix>
| Attribute | Type | Description |
|---|---|---|
| Status | string | The state of the database schema history, which can be one of the following: STOPPED, RECOVERING (recovering history from storage), or RUNNING. |
| RecoveryStartTime | long | The time, in epoch seconds, when the recovery process started. |
| ChangesRecovered | long | The number of changes read during the recovery phase. |
| LastRecoveredChange | string | The last change recovered from the history store. |
| MilliSecondsSinceLastRecoveredChange | long | The number of milliseconds that have elapsed since the last change was recovered from the history store. |
| ChangesApplied | long | The number of schema changes applied during both the recovery phase and runtime. |
| LastAppliedChange | string | The last change applied from the history store. |
| MilliSecondsSinceLastAppliedChange | long | The number of milliseconds that have elapsed since the last change was applied from the history store. |