Egress PrivateLink Endpoints Setup Guide: Self-Managed Services on Azure for Confluent Cloud

This topic presents the steps for setting up Egress PrivateLink Endpoints for self-managed services on Azure and Confluent Cloud to enable fully managed connectors in Confluent Cloud using Azure Private Link.

Prerequisites

In Confluent Cloud, one of the following cluster types was set up with the specified network resource:

• A Dedicated cluster with a Confluent Cloud network

  For the steps to create a Confluent Cloud network, see Create a Confluent Cloud network. The Connection type of the network needs to be “Private Link Access”.

• A Enterprise cluster with a network gateway

  For the steps to create a gateway, see Create a gateway for outbound connectivity in Confluent Cloud.

Step 1: Create a load balancer

1. In the Azure portal, browse to Virtual machines (VM), and select the applicable VM.
2. Browse to Networking → Load balancing.
3. If there’s an existing load balancer available that you’d like to use, skip to the Step 2. Create a Private Link service section.
4. To create a new load balancer, click Add load balancing → Create New → Load Balancer.
5. Specify the settings for the new load balancer.
   • Load balancer name
   • Type: Internal
   • Protocol: TCP
   • Load balancer rule
     • Port: The port for the external endpoint
     • Backend port: The port used to route traffic to VM
6. Click Create.
7. Wait for the load balancer to be successfully created and added. You can check your Azure notifications for status.

Step 2. Create a Private Link service

1. In the Azure portal, browse to Private Link Center → Private link services.
2. Click Create.
3. Specify the Basic settings for the new private link service.
   • Project details: Select the applicable subscription and resource group.
   • Instance details: Provide the name and associated region of the load balancer and the VM.
4. Click Next: Outbound settings.
5. Specify the Outbound settings for the new private link service.
   • Load balancer: Select the previously created load balancer.
   • Load balancer fronted IP address: Select the associated IP address of the load balancer.
   • Source NAT subnet: Select the subnet that can route to the load balancer.
   • All other settings can be left as default.
6. Click Next: Access security.
7. You can leave the Access security settings as default, or you can select the proper level of security required.
8. Click Next: Tags
9. Add any required tags to the private link service.
10. Click Next: Review + create
11. Review that all details are correct
12. Click Create.
13. Wait for the deployment to be completed.

Step 3. Retrieve the Resource ID

1. Once deployment is complete, click Go to Resource.

2. Click JSON View.

   

3. Copy the Resource ID which is required when you create the Egress Private Link Endpoint in Confluent Cloud.

   

Step 4. Create an Egress PrivateLink Endpoint

Confluent Cloud Egress Private Link Endpoints are Azure Private Endpoints used to connect to Azure Private Link Services.

Dedicated clusterEnterprise cluster

1. In the Network management page or tab of the desired Confluent Cloud environment, click the Confluent Cloud network you want to add the Private Link Endpoint to. The Connection Type of the network needs to be “Private Link Access”.

2. Click Create endpoint in the Egress connections tab.

3. Click the service you want to connect to. Select Other if you do not see the specific service.

4. Follow the guided steps to specify the field values, including:

   • Name: The name of the Private Link Endpoint.

   • Resource ID: The resource ID of the Private Link service.

     The Resource ID is the one noted down above, in the last step of the Step 3 section.

     Note that the resource alias is not supported.

   • Sub-resource name: Leave blank.

5. Click Create to create the Private Link Endpoint.

6. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and then click Finish.

Step 5. Accept the endpoint connection request

1. In the Azure portal, When the Private Link Endpoint status becomes “Pending Accept”, go to the Private Link service you previously created → Settings → Private endpoint connections.

2. Select the pending connection and click Approve.

   You can check that this is the connection request by Confluent by looking at the description column which will provide the associated Confluent Cloud gateway and environment ID.

   

3. Go back to Confluent Cloud and wait until the Private Link Endpoint status transitions from “Pending accept” to “Ready”.

Step 6. (Optional) Create the DNS record

Dedicated clusterEnterprise cluster

1. When the Private Link Endpoint status transitions to “Ready”, click Create DNS record in the DNS tab, or click Create Record on the associated Private Link Endpoint tile.
2. Specify the following:
   • Access point: The Egress PrivateLink Endpoint you created in the Step 4 section.
   • Domain: The associated service endpoint.

Step 7. Create the Connector

1. When the DNS Record status transitions to “Ready”, you can create the connector.

2. For the connector’s Endpoint/Host/Hostname, specify the IP address of the Access Point if no associated DNS records have been created.

   

3. For the steps to create the connector, refer to the connector-specific documentation that is listed for your specific connector in Supported connectors.