The Oracle CDC Source connector scales horizontally using the existing Kafka Connect framework.
The connector is configured with three tasks in the following graphic.
- Task 1: Reads records from the Oracle Database Redo Log, then writes these records to a redo log topic in Apache Kafka®.
- Task 2: Creates a snapshot for the DEPARTMENTS and EMPLOYEES tables.
- Task 3: Creates a snapshot for the JOBS table.
Once Task 2 and Task 3 are done with snapshotting, they read records from the
Redo Log Topic in Kafka rather than from the Oracle database Redo Log Topic. The
three tasks populate Kafka with table-specific topics from the snapshot forward.
If you need to create snapshots for many tables, you can add more tasks to
get better snapshot performance.
Horizontal Scaling Example