Manage Networking for Confluent Cloud Connectors

This topic provides an overview of the networking features supported for fully managed connectors in Confluent Cloud.

Networking

Consider the following diagram and the table when determining the public or private networking for fully managed connectors.

For Confluent Cloud networking details, see the Cloud Networking docs.

../../_images/connector-networking.png

The following table summarizes networking features supported for fully managed connectors.

Cluster Networking Type (Cluster Type) Target Service’s Endpoint Cloud Service Provider Status
Public Endpoint (Dedicated, Standard, Basic) Public Endpoint AWS, Azure, Google Cloud Supported
  Private Endpoint AWS, Azure, Google Cloud Not supported
VPC/VNet Peering and Transit Gateway with /16 CIDRs (Dedicated) Public Endpoint AWS, Azure, Google Cloud Supported
 

Private Endpoint

  • Direct connection to Private IP address
  • Hostname with Public DNS
 AWS, Azure, Google Cloud Supported
 

Private Endpoint

  • Hostname with Private DNS
 AWS, Azure

Supported

Requires use of DNS Forwarding feature: AWS, Azure
    Google Cloud Currently not supported
VPC Peering and Transit Gateway with zonal /27 CIDRs in Limited Availability (Dedicated) Public Endpoint AWS Supported
 

Private Endpoint

  • Direct connection to Private IP address
  • Hostname with Public DNS
  • Hostname with Private DNS
 AWS Currently not supported
Private Link (Dedicated) Public Endpoint AWS, Azure, Google Cloud Supported
 

Private Endpoint

  • Direct connection to Private IP address
  • Hostname with Public DNS
  • Hostname with Private DNS
 AWS, Azure

Supported

Requires use of Egress Private Link Endpoints: AWS, Azure
    Google Cloud Currently not supported
Private Link (Enterprise) Public Endpoint AWS, Azure, Google Cloud Supported
 

Private Endpoint

  • Direct connection to Private IP address
  • Hostname with Public DNS
  • Hostname with Private DNS
 AWS, Azure

Supported

Requires use of Egress Private Link Endpoints: AWS, Azure
    Google Cloud Currently not supported

Target service networking supportability

The following table lists the networking supportability of the connectors with links to associated setup guides.

External Target Service Confluent Cloud Private Link (AWS Dedicated & Enterprise) Confluent Cloud Private Link (Azure Dedicated & Enterprise) Confluent Cloud Private Link (Google Cloud Dedicated) Confluent Cloud Peering / Transit Gateway Confluent Cloud Public
Self-managed services Yes Yes Yes only if the endpoint is public Yes with DNS Forwarding: AWS, Azure Yes only if the endpoint is public

AWS first-party services

  • S3
  • Kinesis
  • Lambda
  • DynamoDB
  • Cloudwatch
  • SQS
 Yes Yes only if the endpoint is public Yes only if the endpoint is public Yes Yes only if the endpoint is public
RDS Yes Yes only if the endpoint is public Yes only if the endpoint is public Yes Yes only if the endpoint is public
OpenSearch If private connectivity is required, use OpenSearch Ingestion. Yes only if the endpoint is public Yes only if the endpoint is public Yes Yes

Azure first-party services

  • Blob
  • Event Hubs
  • Service Bus
  • SQL Server
  • CosmoDB
 Yes only if the endpoint is public Yes Yes only if the endpoint is public Yes with DNS Forwarding Yes only if the endpoint is public

Google Cloud first-party services

  • Google Cloud Storage
  • BigQuery
  • Google Pub/Sub
  • Google Cloud Functions
 Yes only if the endpoint is public Yes only if the endpoint is public Yes only if the endpoint is public Yes only if the endpoint is public Yes only if the endpoint is public
Snowflake Yes Yes Yes only if the endpoint is public Yes Yes
MongoDB Atlas Yes Yes Yes only if the endpoint is public Yes Yes
ElasticSearch Yes Yes Yes only if the endpoint is public Yes Yes
Salesforce Yes Yes Yes if Salesforce is using a private link Yes Yes
Splunk Yes Yes Yes only if the endpoint is public Yes Yes
AWS S3 Sink Yes Yes only if the endpoint is public Yes only if the endpoint is public Yes Yes
Blob Storage Yes only if the endpoint is public Yes Yes only if the endpoint is public Yes Yes

The following pages describe how to configure Egress Private Link Endpoints for connectors;

Egress IP address ranges

The following tabs provide network connectivity IP address details. Note that a Connect node runs in the same VPC/VNet as the cluster the Connect node was provisioned with. This is true for all cluster types (Basic, Standard, Enterprise and Dedicated). For Confluent Cloud networking details, see the Cloud Networking docs.

Public egress IP addresses are available on all the major cloud platforms. For details, see Public Egress IP Addresses for Confluent Cloud Connectors.

Public egress IP addresses are not supported with Custom Connectors.

The following information applies to a fully managed Sink or Source connector connecting to an external system using a public IP address.

Cluster network type Public IP address connectivity IP range used by the connector
Public Endpoint Yes A set of public egress IP addresses (see Public Egress IP Addresses for Confluent Cloud Connectors)
VPC Peering and Transit Gateway Yes Dynamic public IP/CIDR range from the cloud provider region where the Confluent Cloud cluster is located
Private Link Yes Dynamic public IP/CIDR range from the cloud provider region where the Confluent Cloud cluster is located

See the following cloud provider documentation for additional information:

DNS zones

The Domain Name System (DNS) is the system used to translate URLs/Hostnames to IP addresses, for example, www.confluent.io to 54.177.145.149.

A public DNS server contains DNS records that can be resolved using the public internet. A private DNS server contains DNS records that can only be resolved in a private network, such as a VPC or an on-prem environment.

One way to check if a given hostname uses public DNS is running the dig command with a public DNS resolver:

dig [DNS-server] <hostname>

DNS-server can be any public DNS server, such as Google DNS server (8.8.8.8) and Cloudflare DNS server (1.1.1.1).

For example:

dig 8.8.8.8 www.confluent.io

Fully managed connectors in Confluent Cloud support the following types of DNS zones/servers for resolving and accessing required endpoints.

  AWS Azure Google Cloud
Public DNS Supported Supported Supported
Private DNS Supported with DNS Forwarding Supported with DNS Forwarding Not supported

Troubleshoot networking issues for fully managed connectors

This page describes common networking-related errors you may encounter when creating connectors, and it provides checklists that can help you to troubleshoot the issues.

Issues with Peering or Transit Gateway

Errors trying to connect via FQDN (fully qualified domain name) with publicly resolvable DNS

  • If able to directly connect to the private IP address, there is an issue when resolving DNS.
  • If not able to connect to the private IP address:
    • Check the peering/Transit Gateway setup, routes, associated firewalls, security groups, and network access control lists.
    • Check ports and protocol settings.

Errors trying to connect via FQDN with DNS that is not publicly resolvable

  • Check if DNS forwarding is correctly set up with the right IP address for the DNS server and is forwarding the needed domain name. For details, see DNS Forwarding for AWS or DNS Forwarding for Azure.
  • Check your DNS setup, peering/Transit Gateway setup, routes, associated firewalls, security groups, and network access control lists.
  • Check ports and protocol settings.