AWS PrivateLink allows for one-way
secure connection access from your VPC to Confluent Cloud with an added protection
against data exfiltration. This networking option is popular for its unique
combination of security and simplicity.
The following diagram summarizes the AWS PrivateLink architecture with the
customer VPC/account and the Confluent Cloud VPC/account.
To set up to use AWS PrivateLink with Confluent Cloud:
Your VPC must allow outbound internet connections for Confluent Cloud Schema Registry,
ksqlDB, and Confluent CLI to work.
Confluent Cloud Schema Registry is accessible over the internet.
Provisioning new ksqlDB clusters requires Internet access. After
ksqlDB clusters are up and running, they are fully accessible over
AWS PrivateLink connections.
Confluent CLI requires internet access to authenticate with the Confluent Cloud
Confluent Cloud Console components, such as topic management, require additional
configuration to function as they use cluster endpoints. To use all features
of the Confluent Cloud Console with AWS PrivateLink, see
Use Confluent Cloud with Private Networking.
Existing Confluent Cloud networks cannot be converted to use AWS PrivateLink.
After provisioning of a Confluent Cloud network, you cannot change the DNS resolution
option for the public or private DNS resolution.
Cross-region AWS PrivateLink connections are not supported.
Fully-managed Confluent Cloud connectors can connect to sources or sinks using public
IP addresses. Sources or sinks in the customer network with private IP addresses
are not supported. An exception to this is the Amazon S3 Sink connector
which can connect to an Amazon S3 bucket from a private network.
Japan (Osaka) region ap-northeast-3 and Jakarta region ap-southeast-3
are not supported for AWS PrivateLink clusters in Confluent Cloud. For these regions,
you can use VPC peering for clusters, or
use AWS PrivateLink with clusters in different regions.
All AWS availability zones, except use1-az3 in the us-east-1 region,
Each Confluent Cloud single zone cluster that uses AWS PrivateLink access
is provisioned with service endpoints in one availability zone. The
availability zone is selected based on Confluent Cloud placement policies.
To ensure connectivity over AWS PrivateLink connections, provision
subnets in your VPC that minimally include the single availability zone
in which the AWS PrivateLink access is provisioned.
To make an AWS PrivateLink connection to a cluster in Confluent Cloud you must register the
AWS Account ID you wish to use. This is a security measure so Confluent can ensure
only your organization can initiate AWS PrivateLink connections to the cluster.
AWS PrivateLink connections from a VPC not contained in a registered AWS account
will not be accepted by Confluent Cloud.
You can register multiple AWS accounts to the same Confluent Cloud cluster, and
AWS PrivateLink connections can be made from multiple VPCs in each registered
If a VPC exists in a different AWS account, you need to create a separate PrivateLink
Access on your Confluent Cloud network.
Open the Confluent Cloud Console, in the Network Management tab, click the
Confluent Cloud network you want to add the connection to.
Click + PrivateLink Access.
Enter the 12-digit AWS Account Number for the account containing
the VPCs you want to make the AWS PrivateLink connection from.
Note the VPC Endpoint service name to create an AWS PrivateLink
connection from your VPC to the Confluent Cloud cluster. This URL will also be
HTTP POST request
Your AWS PrivateLink connection status will transition from “Pending” to
“Active” in the Confluent Cloud Console. You still need to configure the Private
Endpoints in your VPC before you can connect to the cluster.
Create an AWS PrivateLink connection to Confluent Cloud¶
Follow this procedure to create an AWS PrivateLink connection to a Confluent Cloud
cluster on AWS using the Confluent Cloud Console or REST APIs.
Set up the VPC endpoint for AWS PrivateLink in your AWS account¶
After the connection status is “Active” in the Confluent Cloud Console, configure
your VPC private endpoints using the AWS VPC dashboard
to make the AWS PrivateLink connection to your Confluent Cloud cluster.
Confluent recommends using a
for setting up Private Link endpoints. This configuration automates the manual steps described below.
The Confluent Cloud VPC and cluster is created in specific zones that, for optimal
usage, should match the zones of the VPC you want to make the AWS PrivateLink
connections from. You must have subnets in your VPC for these zones so that
IP addresses can be allocated from them. It is allowed to also have subnets
in zones outside of these. AWS Zone IDs should be used for this. You can find
the specific Availability Zones for your Confluent Cloud cluster in the Confluent Cloud Console.
Because Availability Zone names (for example, us-west-2a) are
inconsistent across AWS accounts, Availability Zone IDs (like
usw2-az1) are used.
Verify that DNS hostnames and DNS resolution are enabled.
In the navigation menu under VIRTUAL PRIVATE CLOUD, click Endpoints.
The Endpoints page appears.
Click Create endpoint. The Create endpoint page appears.
Under Service category, select Other endpoint services.
Under Service settings, enter the Service name for your
Confluent Cloud VPC Service Endpoint Name. You can find this in
the Confluent Cloud Console.
Click Verify service. If you get an error, ensure that your account
is allowed to create PrivateLink connections.
Under VPC, select the VPC in which to create your endpoint.
Uncheck the Enable DNS name setting under Additional settings
(only appearing after the VPC is selected).
This step is required for all types of DNS resolution for Confluent Cloud.
Click Create endpoint.
Your VPC endpoint is created and displayed. Copy the VPC Endpoint ID for
Note the availability zones for your Confluent Cloud cluster from the
Networking tab in the Confluent Cloud Console. Select the service in these
zones. Ensure the desired subnet is selected for each zone. Failure to
add all zones as displayed in the Confluent Cloud Console can cause connectivity
issues to brokers in the omitted zones, which can result in an unusable cluster.
Confluent Cloud single availability zone clusters need service and subnet
selection in one zone whereas Confluent Cloud multi-availability zone clusters
need service and subnet selection in three zones.
Select or create a security group for the VPC Endpoints.
Add three inbound rules for each of ports 80, 443, and 9092
from your desired source (your VPC CIDR). The Protocol should be
TCP for all three rules.
Port 80 is not required, but is available as a redirect only to
https/443, if desired.
Wait for acceptance by Confluent Cloud. This should happen almost immediately
(less than a minute). After it is accepted, the endpoint will transition
from “Pending” to “Active”.
You must update your DNS records to ensure connectivity through AWS PrivateLink
in the supported pattern. Any DNS web service that can ensure that DNS requests are
routed as follows can be used, but for the example, AWS Route53 is used.
The public (also known as chased private in Confluent Cloud) DNS resolution is used for
the bootstrap server and broker hostnames of a Confluent Cloud cluster that is using
AWS Private Link. When the public resolution is used, the clusters in this
network requires both public and private DNS to resolve cluster endpoints.
Only the the Confluent Global DNS Resolver (GLB) endpoints are advertised.
The public DNS resolution performs the following two-step process:
The Confluent Cloud Global DNS Resolver removes the glb subdomain and returns a
CNAME for your bootstrap and broker hostnames.
From an instance within the VPC (or anywhere the previous step’s DNS is
set up), run the following to validate Kafka connectivity through AWS
PrivateLink is working correctly.
Set an environment variable with the cluster bootstrap URL.
The Bootstrap URL displayed in Confluent Cloud Console includes the port (9092).
The BOOTSTRAP value should include the full hostname, but do not include
the port. This is so that you can run the
openssls_client-connect<host>:<port> command with the required values.
# Public DNS resolutionexportBOOTSTRAP=lkc-2v531-lg1y3.us-west-1.aws.glb.confluent.cloud
# Private DNS resolutionexportBOOTSTRAP=lkc-2v531.domz6wj0p.us-west-1.aws.confluent.cloud
Test connectivity to your cluster by running the openssls_client-connect<host>:<port>
command, specifying the $BOOTSTRAP environment variable for the <host> value and
9092 for the <port> value.
To run the openssls_client-connect command, the -connect option requires
that you specify the host and the port number. For details, see the
If the return output is -----BEGINCERTIFICATE-----Verifyreturncode:0(ok),
connectivity to the bootstrap is confirmed.
You might need to update the network security tools and firewalls to allow
connectivity. If you have issues connecting after following these
steps, confirm which network security systems your organization
uses and whether their configurations need to be changed. If you still have issues,
run the debug connectivity script
and provide the output to Confluent Support for assistance with your PrivateLink setup.