Azure Blob Storage Sink Connector for Confluent Cloud with Azure Egress Access Point

This topic presents the steps for configuring the Azure Blob Storage Sink connector in Confluent Cloud with Azure Private Link and Egress Access Point.

Prerequisites

The following is a list of prerequisites for configuring the Azure Blob Storage Sink connector with an Egress Access Point:

  • A Confluent Cloud Dedicated cluster was set up and is running within an Azure Private Link network.
  • A Storage Container was created to sink data into and is running within the same region and cloud as the Confluent Cloud cluster.
  • A source topic was created to sink data into Azure Blob Storage.

Step 1. Retrieve your Azure Resource ID

In Azure, go to your storage accounts overview section, and click JSON View.

../../_images/azure-resource-id.png
../../_images/azure-resource-JSON.png

Step 2. Create an Egress Access Point

  1. In the Confluent Cloud Console, in the Network management tab, click the associated Private Link network.

  2. In the Egress access points tab, click Create access point.

  3. Specify the following:

    • Name: A name for this access point.
    • Azure Resource ID: Azure Resource ID that you retrieved in the first step.
    • Sub-resource name: Select blob.
    ../../_images/ccloud-access-point.png

  4. Click Save, and wait for the status to become “Pending Accept”.

Step 3. Accept the connection request

In the Azure portal, in the Pending Connections page, approve the associated connection request.

../../_images/azure-accept-connection.png

Step 4. Create a DNS record

  1. In the Confluent Cloud Console, when the status of the Egress Access Point becomes “Ready”, click Create record.

  2. In Domain, specify the domain name record in the format: <storage-account-name>.blob.core.windows.net

    ../../_images/dns-record.png

  3. Click Save, and wait until the status of the DNS record becomes “Ready”.

Step 5. Create the Azure Blob Storage Sink connector

  1. In the Confluent Cloud Console, go to your associated Dedicated cluster. In Connectors, click Azure Blob Storage Sink.
  2. Select the source topic, and click Continue.
  3. Specify the Kafka authentication mechanism, and click Continue.
  4. Specify the authentication details for Azure Storage Account.
  5. Specify configuration details for the connector.
  6. Specify sizing (number of tasks) for the connector.
  7. Review and launch the connector.

When the connector is successfully launched, the connector status becomes “Running”.