Connect External Systems to Confluent Cloud¶
Confluent Cloud offers pre-built, fully managed, Kafka Connectors that make it easy to instantly connect to popular data sources and sinks. With a simple GUI-based configuration and elastic scaling with no infrastructure to manage, Confluent Cloud Connectors make moving data in and out of Kafka an effortless task, giving you more time to focus on app development.
- Source Connector
- A source connector, such as the Microsoft SQL Server Source Connector, ingests entire databases and streams table updates to Kafka topics. It can also collect metrics from all of your application servers and store these in Kafka topics, making the data available for stream processing with low latency.
- Sink Connector
- A sink connector delivers data from Kafka topics into secondary indexes, such as Google BigQuery or batch systems like Amazon S3, for offline analysis.
Supported Connectors¶
The following Confluent Cloud connectors are supported by Confluent:
- Amazon Kinesis Source Connector
- Amazon Redshift Sink Connector
- Amazon S3 Sink Connector
- Azure Blob Storage Sink Connector
- Azure Data Lake Storage Gen2 Sink Connector
- Azure Event Hubs Source Connector
- Datagen Source Connector (development and testing)
- Elasticsearch Service Sink Connector
- Google BigQuery Sink Connector
- Google Cloud Spanner Sink Connector
- Google Cloud Storage Sink Connector
- Google Pub/Sub Source Connector
- Microsoft SQL Server Source Connector
- MySQL Source Connector
- Oracle Database Source Connector
- PostgreSQL Source Connector
- Snowflake Sink Connector
Preview Connectors¶
Caution
Preview connectors are not currently supported and are not recommended for production use.
The following Confluent Cloud connectors are available for preview:
- AWS Lambda Sink Connector
- Azure Functions Sink Connector
- Google Cloud Dataproc Sink Connector
- Google Cloud Functions Sink Connector
- Microsoft SQL Server Sink Connector
- Microsoft SQL Server Source CDC Connector (Debezium)
- MongoDB Atlas Sink Connector
- MongoDB Atlas Source Connector
- MySQL Sink Connector
- MySQL Source CDC Connector (Debezium)
- Salesforce CDC Source Connector
- PostgreSQL CDC Source Connector (Debezium)
- PostgreSQL Sink Connector
Cloud Platforms¶
The following table shows the cloud platforms supported by each Confluent Cloud connector.
| Cloud Connector | AWS | Azure | GCP |
|---|---|---|---|
| Amazon Kinesis Source Connector | Yes | Yes | Yes |
| Amazon Redshift Sink Connector | Yes | No | No |
| Amazon S3 Sink Connector | Yes | No | No |
| AWS Lambda Sink Connector | Yes | No | No |
| Azure Blob Storage Sink Connector | No | Yes | No |
| Azure Data Lake Storage Gen2 Sink Connector | No | Yes | No |
| Azure Event Hubs Source Connector | Yes | Yes | Yes |
| Azure Functions Sink Connector | Yes | Yes | Yes |
| Datagen Source Connector | Yes | Yes | Yes |
| Elasticsearch Service Sink Connector | Yes | Yes | Yes |
| Google BigQuery Sink Connector | No | No | Yes |
| Google Cloud Dataproc Sink Connector | No | No | Yes |
| Google Cloud Functions Sink Connector | No | No | Yes |
| Google Cloud Spanner Sink Connector | No | No | Yes |
| Google Cloud Storage Sink Connector | No | No | Yes |
| Google Pub/Sub Source Connector | Yes | Yes | Yes |
| Microsoft SQL Server Sink Connector | Yes | Yes | Yes |
| Microsoft SQL Server Source CDC Connector (Debezium) | Yes | Yes | Yes |
| Microsoft SQL Server Source Connector | Yes | Yes | Yes |
| MongoDB Atlas Sink Connector | Yes | Yes | Yes |
| MongoDB Atlas Source Connector | Yes | Yes | Yes |
| MySQL Source CDC Connector (Debezium) | Yes | Yes | Yes |
| MySQL Sink Connector | Yes | Yes | Yes |
| MySQL Source Connector | Yes | Yes | Yes |
| Oracle Database Source Connector | Yes | Yes | Yes |
| PostgreSQL CDC Source Connector (Debezium) | Yes | Yes | Yes |
| PostgreSQL Sink Connector | Yes | Yes | Yes |
| PostgreSQL Source Connector | Yes | Yes | Yes |
| Salesforce CDC Source Connector | Yes | Yes | Yes |
| Snowflake Sink Connector | Yes | Yes | Yes |
Internet Access to Resources¶
Consider the following scenarios when determining the public Internet access configuration for resources that managed connectors must access. For full Confluent Cloud networking details, see Networking in Confluent Cloud.
- VPC peering + AWS Transit Gateway
- Dedicated clusters with VPC peering or AWS Transit Gateway access have a limited range of IP addresses (for example, a
/16CIDR block specified at cluster creation). Connectors launched in these clusters originate connections from the limited IP range configured. A customer could allow connections to their database from the/16IP range instead of0.0.0.0/0. - AWS PrivateLink
- Connectors launched in clusters using AWS PrivateLink must originate connections over the Internet. These connections can come from anywhere in the cloud provider’s IP range. This is because AWS PrivateLink is a one-way connection. That is, the customer can connect to Confluent Cloud using AWS PrivateLink, but Confluent Cloud cannot access the customer network (over AWS PrivateLink). A customer could allow connections to their database from the set of cloud provider IP ranges. This may be too open for many customers, but is much less open than
0.0.0.0/0. - Internet-facing clusters
- Connectors launched in clusters with Internet endpoints originate connections over the Internet from anywhere in the cloud provider’s IP range. A customer could allow connections to their database from the set of cloud provider IP ranges. This may be too open for many customers, but is much less open than
0.0.0.0/0.