# Downstream Capture for Oracle XStream CDC Source Connector for Confluent Cloud Downstream capture is a deployment topology in which the XStream capture process and outbound server run on a separate Oracle database (the downstream database) rather than on the database where DML and DDL originate (the source database). The connector connects to both databases: the source database for the initial snapshot and metadata queries, and the downstream database for streaming changes from the XStream outbound server. This topology offloads the XStream capture workload from the source database so production OLTP performance is not affected by capture overhead or long-running mining sessions. It is the foundation for the [Cascading Downstream Capture for Oracle XStream CDC Source Connector for Confluent Cloud](cascading-downstream-capture.md#xstream-cascading-downstream-capture) topology that uses Oracle Data Guard. For more information, see [Oracle Database XStream Guide 19c, Downstream Capture](https://docs.oracle.com/en/database/oracle/oracle-database/19/xstrm/configuring-xstream-out.html#GUID-89EEFA3B-CED7-4682-97D6-A5BEA8E9C4D2__CEGGGFCG). ## Architecture The following architecture diagram shows how the connector reads from each database during the snapshot and streaming phases. The connector interacts with both databases throughout its lifecycle: * **During the snapshot phase**, the connector reads from the source database. This includes locking captured tables in `ROW SHARE MODE`, reading the snapshot SCN from `V$DATABASE`, capturing schemas, and running Flashback queries (`SELECT ... AS OF SCN`). * **During the streaming phase**, the connector attaches to the XStream outbound server on the downstream database and reads Logical Change Records (LCRs) starting from the `snapshot_scn` on first startup, or the persisted `lcr_position` on resume. * **Schema queries during streaming** (for example, when adding a table to `table.include.list`) are issued against the source database, because XStream LCRs do not carry full DDL history for tables not yet processed. ![Downstream capture](images/xstream-downstream-capture.png) Both the source and the downstream must be `PRIMARY` databases. Downstream capture cannot run on a standby database. For the topology in which redo arrives at the downstream through a Data Guard standby, see [Cascading Downstream Capture for Oracle XStream CDC Source Connector for Confluent Cloud](cascading-downstream-capture.md#xstream-cascading-downstream-capture). ## Capture modes XStream downstream capture supports two modes, distinguished by how redo arrives on the downstream database: | Aspect | Real-time downstream capture | Archived-log downstream capture | |------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------| | Source of redo on downstream | Standby redo logs | Foreign archived redo logs | | Capture parameter | `downstream_real_time_mine => 'Y'` | Not set (defaults to `'N'`) | | `LOG_ARCHIVE_DEST_n` redo log type on source | `ONLINE_LOGFILES` (no `TEMPLATE`) | `ONLINE_LOGFILES` with `TEMPLATE` attribute | | Standby redo logs required on downstream | Yes | No | | Latency | Low. Capture mines redo as RFS writes to standby logs. | Higher. Waits for source log switch and archive shipment. | | Monitoring view on downstream | `V$STANDBY_LOG` | `V$FOREIGN_ARCHIVED_LOG` | | Capture auto-restart after downstream recovery | Automatic if the capture status was `ENABLED` when shutdown. Otherwise requires
`DBMS_CAPTURE_ADM.START_CAPTURE`. | No. `DBMS_CAPTURE_ADM.START_CAPTURE` must be called manually. | The following diagram shows how redo flows in real-time downstream capture mode. ![Real-time downstream capture](images/xstream-real-time-downstream-capture.png) The following diagram shows how redo flows in archived-log downstream capture mode. ![Archived-log downstream capture](images/xstream-archived-log-downstream-capture.png) Both modes use a single XStream outbound server on the downstream and the same connector configuration. The difference is only in how redo arrives at the downstream. ## Connector configuration Downstream capture is enabled by setting one or more `downstream.database.*` configuration properties. When any of these properties is set, the connector enters downstream capture mode. When none are set, the connector runs in local capture mode. #### NOTE The connector’s existing `database.user` and `database.password` credentials are reused for both the source and the downstream connections. The same user with the same password must exist on both databases. Distinct credentials for each database are not supported. `database.pdb.name` continues to refer to the source PDB. The initial snapshot and all schema queries run against the source. The connector does not include a separate `downstream.database.pdb.name` configuration. Because the XStream queue, capture process, and outbound server run in the downstream CDB root, the connector attaches to the CDB service name specified in `downstream.database.service.name`. | Configuration property | Type | Default | Description | |------------------------------------|----------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `downstream.database.hostname` | `string` | (none) | **Required\*.** Hostname or IP address of the downstream Oracle database
server. Setting any `downstream.database.*` property activates
downstream capture mode. | | `downstream.database.port` | `int` | `1521` | Listener port for the downstream database. Use `2484`, or your
operator’s port, when `downstream.database.tls.mode` is not
`disable`. | | `downstream.database.dbname` | `string` | (none) | **Required\*.** Name of the downstream database. In a multitenant
architecture, this is the CDB name. | | `downstream.database.service.name` | `string` | (none) | **Required\*.** Service name for connecting to the downstream database.
In a CDB, use the service for the CDB root. For an Oracle RAC
downstream, use the service auto-created for the XStream outbound
server (`use_rac_service=Y`). | | `downstream.database.tls.mode` | `enum` | `disable` | TLS mode for the connector-to-downstream connection. One of
`disable`, `one-way`, `two-way`. Independent of
`database.tls.mode`. | | `downstream.database.wallet.file` | `string` | (empty) | The Oracle SSO auto-login wallet (`cwallet.sso`)
for TLS connection between the connector and downstream database. Required when
`downstream.database.tls.mode` is set to `two-way`. This configuration is
independent of `database.wallet.file`, but it shares the same requirements
as the `database.wallet.file` [configuration property](cc-oracle-xstream-cdc-source.md#cc-oracle-xstream-cdc-source-config-properties). | \* Required when downstream capture mode is activated (that is, when at least one `downstream.database.*` property is set). ### Configuration examples ### CDB / PDB (minimal) Minimum configuration for a CDB source and CDB downstream: ```json { "name": "OracleCDC-Downstream", "config": { "connector.class": "io.confluent.connect.oracle.xstream.cdc.OracleXStreamSourceConnector", "tasks.max": "1", "database.hostname": "src-db.example.com", "database.port": "1521", "database.user": "C##CFLTUSER", "database.password": "", "database.dbname": "SRCCDB", "database.service.name": "SRCCDB_DB.EXAMPLE.COM", "database.pdb.name": "SRCPDB1", "downstream.database.hostname": "cap-db.example.com", "downstream.database.port": "1521", "downstream.database.dbname": "CAPCDB", "downstream.database.service.name": "CAPCDB.EXAMPLE.COM", "database.out.server.name": "XOUT", "topic.prefix": "cflt", "table.include.list": "HR.EMPLOYEES", "schema.history.internal.kafka.bootstrap.servers": "broker:9092", "confluent.topic.bootstrap.servers": "broker:9092" } } ``` ### Non-CDB For non-CDB databases, omit `database.pdb.name`, use user names without the `c##` prefix, and qualify schemas without a PDB name: ```json { "config": { "database.hostname": "src-db.example.com", "database.user": "CFLTUSER", "database.dbname": "SRCDB", "database.service.name": "SRCDB_DB.EXAMPLE.COM", "downstream.database.hostname": "cap-db.example.com", "downstream.database.dbname": "CAPDB", "downstream.database.service.name": "CAPDB.EXAMPLE.COM", "database.out.server.name": "XOUT", "table.include.list": "DBUSER.EMPLOYEES" } } ``` ### TLS to downstream only Source over plaintext TCP/1521 and downstream over mutual TLS on TCPS/2484 with its own wallet: ```json { "config": { "database.hostname": "src-db.example.com", "database.port": "1521", "downstream.database.hostname": "cap-db.example.com", "downstream.database.port": "2484", "downstream.database.tls.mode": "two-way", "downstream.database.wallet.file": "data:text/plain;base64,/u3+7QAAAAIAAAACAAAAAQAGY2xpZ...omitted...==" } } ``` ## Database prerequisites Before configuring the connector for downstream capture, your Oracle database administrator must complete the prerequisite setup on both the source and downstream databases. See [Downstream capture database prerequisites](prereqs-validation.md#connect-oracle-xstream-cdc-source-prereqs-downstream). ## Connector startup The snapshot and streaming flow are unchanged from local capture, with one difference: the streaming phase attaches to the downstream’s outbound server rather than the source’s. Schema capture, the initial snapshot, and out-of-band schema queries continue to read from the source. The following additional validations run at startup when downstream capture mode is active: * The downstream database is reachable, on a supported version, and the connect user has read access to the required catalog views at `CDB$ROOT` (or at the database level for non-CDB), through `SELECT_CATALOG_ROLE` or the equivalent `SELECT` grants. * The XStream outbound server is looked up on the downstream rather than on the source. * `ARCHIVELOG` mode is verified on the downstream in addition to the source. Startup logs indicate when downstream capture mode is active and list both database hostnames. ## Oracle Real Application Clusters (RAC) Both the source and the downstream may be single-instance or RAC databases. All four source and downstream combinations are supported. When the downstream is RAC, configure the capture and the connector as documented in the [local capture RAC](prereqs-validation.md#xstream-downstream-capture-prereqs-rac-downstream) section, substituting `downstream.database.hostname` and `downstream.database.service.name` (not `database.hostname` and `database.service.name`) for the SCAN address and the auto-created XStream service name. The `use_rac_service=Y` mechanism, the failover behavior, and SCAN-based reconnect are unchanged from local capture. When the [source is RAC](prereqs-validation.md#xstream-downstream-capture-prereqs-rac-source) shipping to a downstream: * `LOG_ARCHIVE_DEST_n` on every source instance must point at the downstream. * The downstream needs `m+1` standby redo log groups per source thread (`n*(m+1)` total for an `n`-thread RAC source with `m` groups per thread), each assigned to the matching `THREAD` number. * For real-time downstream capture, both source instances ship to the same downstream RFS pool. For archived-log downstream capture, both instances archive independently to the foreign archive directory. ## Transport Layer Security (TLS) The connector connects to the source and the downstream independently. Each connection has its own TLS configuration: * Source: `database.tls.mode` and `database.wallet.file` * Downstream: `downstream.database.tls.mode` and `downstream.database.wallet.file` The two settings are independent. Any combination of `disable`, `one-way`, or `two-way` for each side is valid. | Topology | Source to connector | Downstream to connector | Source to downstream (redo transport) | |-----------------------------------------------------------|---------------------------------|---------------------------------|----------------------------------------------| | Both plaintext | TCP/1521 | TCP/1521 | TCP/1521 | | One-way TLS on both | TCPS/2484, server cert verified | TCPS/2484, server cert verified | Configured independently at Oracle Net layer | | Mutual TLS on both | TCPS/2484, mutual TLS | TCPS/2484, mutual TLS | Configured independently at Oracle Net layer | | Mixed (for example, one-way source, plaintext downstream) | TCPS/2484 | TCP/1521 | Configured on the source side independently | The redo-transport channel between the source and the downstream is a database-to-database connection that the connector is not in the path of. Configuring TLS on that channel uses the Oracle Net layer (TCPS listener on the receiver, sender wallet trusting the receiver’s CA, and a TCPS service alias referenced by `LOG_ARCHIVE_DEST_n`). For securing redo transport, see [Redo Transport Security](https://docs.oracle.com/en/database/oracle/oracle-database/26/sbydb/oracle-data-guard-redo-transport-services.html#GUID-FA3B203C-D28E-4A27-8585-9EBC19836F4C) in the Oracle documentation. For TLS support, see [Database Security Guide](https://docs.oracle.com/pls/topic/lookup?ctx=en/database/oracle/oracle-database/21/netag&id=DBSEG-GUID-6AD89576-526F-4D6B-A539-ADF4B840819F) in the Oracle documentation. ## Private networking Downstream capture topologies deploy across two databases: - Source database: Handles snapshots and table metadata. - Downstream database: Handles the event streaming workload. ### Networking rules Confluent Cloud initiates egress connections into your private network. A downstream capture connector requires verified private network paths to both the source and downstream databases. The connector task fails to start if either network path is missing or unreachable. #### NOTE This connection model applies identically to Amazon Web Services (AWS), Microsoft Azure (Azure), or Google Cloud deployments. For a complete list of supported networking features for fully managed connectors, see [Manage Networking for Confluent Cloud Connectors](../networking/internet-resource.md#clusters-connect-cloud). ### Connection architecture One Confluent access point maps to exactly one AWS endpoint service. Depending on your database locations, choose one of the following network architectures: - **Shared network setup**: Use this configuration if both databases reside in the same network or within peered networks. In this topology, the databases can share a single load balancer, which requires only one AWS endpoint service and one Confluent access point. To route traffic, use a single DNS name and configure a unique listener port for each database on the load balancer. The network configuration preserves these specific ports to route traffic to the correct destination database automatically. - **Isolated network setup**: Use this configuration if the databases reside in completely unconnected networks, such as different cloud regions or unpeered environments. Because the environments cannot communicate directly, they cannot share a load balancer. You must duplicate the network configuration infrastructure for each environment. Each database requires its own dedicated load balancer, AWS endpoint service, Confluent access point, and DNS name. ## Restrictions In addition to the existing [local capture restrictions](https://docs.confluent.io/kafka-connectors/oracle-xstream-cdc-source/current/overview.html#requirements-and-current-limitations): * The source and the downstream must both be Oracle Enterprise Edition at the same release, with matching operating system family. * The downstream database must be in the `PRIMARY` role. Capture cannot run on a physical or logical standby. For the topology where redo arrives through a Data Guard standby, see [Cascading Downstream Capture for Oracle XStream CDC Source Connector for Confluent Cloud](cascading-downstream-capture.md#xstream-cascading-downstream-capture). * The source and the downstream must have different `GLOBAL_NAME` values. Use a distinct `DB_DOMAIN` if `DB_NAME` would otherwise collide. * The capture process and the outbound server must both run on the same downstream database. Splitting them across two databases using XStream Propagation Send/Receive is not supported. * The connect user must exist on both databases with identical credentials. ## Downstream-specific failure scenarios Connector restart, snapshot resume, capture process restart, and outbound server failure behavior are the same as for local capture. Downstream-specific behavior: * If the downstream is offline long enough for the source to archive multiple redo log files, the source retains unshipped logs subject to its own log-retention policy. The gap resolves automatically when the downstream comes back online. As an operator, configure the source archive retention to ensure it covers expected downstream downtime. * A capture process created without the `_SEND_STREAMS_DICTIONARY` parameter set to `INFINITE` silently skips DML and DDL changes for tables that were created before the capture process’s first SCN. Any capture process created against a source database that already contains the tables to capture is affected, including the first capture process you create. The connector keeps running while a subset of its tables produces no streaming change events. Set the parameter after `CREATE_CAPTURE` and before the capture process is first started, as described in the [downstream capture prerequisites](prereqs-validation.md#xstream-downstream-capture-send-streams-dictionary). For detection and recovery, see [No change events for pre-existing tables](cc-oracle-xstream-cdc-source-troubleshoot.md#xstream-downstream-missing-events-pre-existing-tables). ## Monitoring and troubleshooting For health check queries and troubleshooting procedures, see [Downstream capture monitoring and troubleshooting](cc-oracle-xstream-cdc-source-troubleshoot.md#xstream-downstream-capture-monitoring).