Egress Private Link Endpoint Setup: Azure Blob Storage for Confluent Cloud

This topic presents the steps for configuring the Azure Blob Storage Sink connector in Confluent Cloud with Azure Private Link and Egress Private Link Endpoint.

Prerequisites

The following is a list of prerequisites for configuring the Azure Blob Storage Sink connector with an Egress Private Link Endpoint:

• A Confluent Cloud Dedicated cluster was set up and is running within an Azure Private Link network.
• A Storage Container was created to sink data into and is running within the same region and cloud as the Confluent Cloud cluster.
• A source topic was created to sink data into Azure Blob Storage.

Step 1. Create a gateway in Confluent Cloud (for Enterprise cluster only)

If using an Enterprise cluster in Confluent Cloud, you must set up a gateway in Confluent Cloud as described in Create a gateway for outbound connectivity in Confluent Cloud.

Step 2. Retrieve your Azure Resource ID

In Azure, go to your storage accounts overview section, and click JSON View.

Step 3. Create an Egress Private Link Endpoint

1. In the Network Management tab of the desired Confluent Cloud environment, click the Confluent Cloud network you want to add the Private Link Endpoint to. The Connection Type of the network needs to be “Private Link Access”.

2. Click Create endpoint in the Egress connections tab.

3. Click the service you want to connect to, specifically, Azure Blob.

4. Follow the guided steps to specify the field values, including:

   • Name: The name of the Private Link Endpoint.

   • Resource ID: The resource ID of the Private Link service you retrieved in Obtain Azure Private Link Resource ID.

     Note that the resource alias is not supported.

   • Sub-resource name: The sub-resource name for the specific Azure service you retrieved in Obtain Azure Private Link Resource ID.

5. Click Create to create the Private Link Endpoint.

6. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and then click Finish.

Step 4. Accept the connection request

In the Azure portal, in the Pending Connections page, approve the associated connection request.

Step 5. Create a DNS record

1. In the Confluent Cloud Console, when the status of the Egress Private Link Endpoint becomes “Ready”, click Create record.

2. In Domain, specify the domain name record in the format: <storage-account-name>.blob.core.windows.net

   

3. Click Save, and wait until the status of the DNS record becomes “Ready”.

Step 6. Create the Azure Blob Storage Sink connector

1. Specify the authentication details for Azure Storage Account.
2. Follow the steps to create the source connector or the sink connector in Confluent Cloud.