Configure an Azure Private Link connection to Confluent Cloud (Preview)
- Azure Private Link Service is currently in preview. Please contact your
Confluent representative to get access to the preview.
- Azure Private Link preview is not covered by SLA and is not recommended
for production workloads.
- Azure Private Link preview is supported only with multi-zone dedicated
clusters. Support for single zone dedicated clusters is on the roadmap.
- A Dedicated Kafka cluster in Azure with Azure
Private Link enabled. For more information about how to create a Dedicated
cluster, see Create a Cluster in Confluent Cloud.
Follow this procedure to configure Azure Private Link for a Dedicated cluster in
- Register your Azure subscription with Confluent Cloud using the Confluent Cloud UI.
- Set up the Private Endpoint(s) to Confluent Cloud Private Link Service Alias(es) in
your Azure subscription using the Azure portal.
- Set up Availability Zone mapped DNS records to use Azure Private Endpoints
using the Azure portal.
- Validate connectivity to Confluent Cloud.
- To use Azure Private Link with Confluent Cloud, your VNET must allow outbound
internet connections for DNS resolution, Schema Registry and Confluent Cloud
CLI to work.
- DNS requests to public authority traversing to private DNS zone is required.
- Confluent Cloud Schema Registry is only accessible over the internet.
- Confluent Cloud CLI requires internet access to authenticate with the
Confluent Cloud control plane.
- Confluent Cloud web UI components like topic management and ksqlDB need
additional configuration to function as they use cluster endpoints. To use
all features of the Confluent Cloud web UI with Azure Private Link, follow
For limitations of the Azure Private Link feature, see Limitations.
Register your Azure subscription with Confluent Cloud
To make an Azure Private Link connection to a cluster in Confluent Cloud you must register
the Azure subscription ID you wish to use. This is a security measure that enables
Confluent to ensure that only your organization can initiate Azure Private Link
connections to the cluster. Azure Private Link connections from a VNET not
contained in a registered Azure subscription will not be accepted by Confluent Cloud.
- Navigate to the Cluster Settings page, click the Networking tab, and
click Add Connection.
- Provide the Azure subscription Number for the subscription containing the VNETs
you want to make the Private Link connection from and click Save. The Azure
subscription number can be found on your Azure subscription page
on the Azure portal. Your Azure Private Link connection status will transition
from “Pending” to “Active” in the Confluent Cloud web UI. You still need to configure
the Private Endpoints in your VNET before you can connect to the cluster.
Set up the Private Endpoint(s) for Azure Private Link in your Azure subscription
After the connection status is “Active” in the Confluent Cloud UI,
you must configure Private Endpoint(s) in your VNET from Azure portal to make the
Private Link connection to your Confluent Cloud cluster.
In the Confluent Cloud UI you will find the following information for your Confluent Cloud
cluster under the Cluster Settings section. This information is needed
to configure Azure Private Link for a Dedicated cluster in Azure.
- Kafka Bootstrap (in the General tab)
- DNS domain for cluster (in the Networking tab)
- DNS domain per zone (in the Networking tab)
- Service Alias (in the Networking tab)
Create the Private Endpoint(s)
In the Azure Private Link Center:
- Create a Private Endpoint for Confluent Cloud Availability Zone 1 by clicking
Create Private Endpoint.
- Fill in subscription, resource group, name, and region for the virtual
endpoint and click next. The selected subscription must be the same as
the one registered with Confluent Cloud.
- Select the Connect to an Azure resource by resource ID or alias
option, paste in the Confluent Cloud Service Alias for Availability Zone 1 and
click next. You can find the Confluent Cloud Service Alias for Availability Zone 1
in the Networking tab under Cluster settings in the Confluent Cloud UI .
- Fill in virtual network and subnet where the Private Endpoint is to be
- Click Review + create. Review the details and click Create to
create the Private Endpoint.
- Wait for the Azure deployment to complete, go to the Private Endpoint
resource and verify Private Endpoint connection status is Approved.
- Repeat the above steps, creating two more Private Endpoints for
the remaining two Confluent Cloud Availability Zones.
Set up DNS records to use Azure Private Endpoints
DNS changes must be made to ensure connectivity passes through Azure Private
Link in the supported pattern. Any DNS provider that can
ensure DNS is routed as follows is acceptable. Azure Private
DNS Zone (used in this example) is one option.
Update DNS using Azure Private DNS Zone in the Azure console:
Create the Private DNS Zone.
Search for the Private DNS Zone resource in Azure portal.
Copy the DNS Domain name from the Networking tab under
Cluster Settings in the Confluent Cloud UI and use it as the
name for the Private DNS Zone.
Notice there is no glb in the DNS Domain name
Fill in subscription, resource group and name and click Review +
Wait for the Azure deployment to complete.
Create DNS records.
- Go to the Private DNS Zone resource as created above.
- Click + Record Set.
- Create the following record sets. The IP address of the Private Endpoint
can be found under its associated network interface.
- Select name as “*”, type as “A”, TTL as “1 Minute” and add IP
addresses of all three virtual endpoints as created above.
- Select name as “*.az1”, type as “A”, TTL as “1 Minute” and add IP
address of the az1 virtual endpoint as created above.
- Select name as “*.az2”, type as “A”, TTL as “1 Minute” and add IP
address of the az2 virtual endpoint as created above.
- Select name as “*.az3”, type as “A”, TTL as “1 Minute” and add IP
address of the az3 virtual endpoint as created above.
Attach the Private DNS Zone to the VNET(s) where clients/applications are
- Go to the Private DNS Zone resource and click Virtual network links
- Click Add.
- Fill in link name, subscription and virtual network.
Validate Connectivity to Confluent Cloud
From an instance within the VNET (or anywhere the previous step’s DNS is
set up), run the following to validate Kafka connectivity through Azure
Private Link is working correctly.
Set a variable with the cluster bootstrap URL.
Test connectivity to the cluster.
openssl s_client -connect $BOOTSTRAP:9092 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
If the return output is
-----BEGIN CERTIFICATE----- Verify return code: 0 (ok),
connectivity to the bootstrap is confirmed.
You might need to update the network security tools and firewalls
to allow connectivity. If you have issues connecting after following
these steps, confirm which network security systems your organization
uses and whether their configurations need to be changed.
Next, verify connectivity with the Confluent Cloud CLI.
Log in to the Confluent Cloud CLI with your Confluent Cloud credentials.
List the clusters in your organization.
ccloud kafka cluster list
Select the cluster with Azure Private Link you wish to test.
ccloud kafka cluster use ...
ccloud kafka cluster use lkc-222v1o
Create a cluster API key to authenticate with the cluster.
ccloud api-key create --resource ... --description ...
ccloud api-key create --resource lkc-222v1o --description "connectivity test"
Select the API key you just created.
ccloud api-key use ... --resource ...
ccloud api-key use R4XPKKUPLYZSHOAT --resource lkc-222v1o
Create a test topic.
ccloud kafka topic create test
Start consuming events from the test topic.
ccloud kafka topic consume test
Open another terminal tab or window.
Start a producer.
ccloud kafka topic produce test
Type anything into the produce tab and hit
Ctrl+C to stop the producer.
The tab running consume will print what was typed in the tab running produce.
You’re done! The cluster is ready for use.
- For limitations of the Azure Private Link feature preview,
see preview note.
- Cross-region Azure Private Link connections are not supported.
- Azure Private Link is only available for use with Dedicated clusters.
- Existing Confluent Cloud clusters cannot be converted to use Azure Private
- Fully-managed ksqlDB is not available for use with Azure Private Link
- Each Confluent Cloud cluster using Azure Private Link will be provisioned
with Private Link Service in three availability zones. Three private
endpoint connections are required for both single zone and multi zone
- Fully-managed Confluent Cloud connectors can connect to source(s) or sink(s)
using a public IP. Source(s) or sink(s) in the customer network with private
IP are not supported.
- Azure Private Link connections cannot be shared across multiple Confluent Cloud
clusters. Separate Azure Private Link connections must be made to
each Confluent Cloud cluster.
- Availability zone selection for placement of Confluent Cloud cluster and
Azure Private Link service is not supported.
- For requirements of the Azure Private Link feature,