Egress Private Link Endpoint Setup: Snowflake on Azure for Confluent Cloud

This topic presents the steps for configuring the Snowflake Sink connector in Confluent Cloud with Azure Private Link and Egress Private Link Endpoint.

Prerequisites

The following is a list of prerequisites for configuring the Snowflake Sink connector with an Egress Private Link Endpoint:

• A Confluent Cloud Dedicated cluster was set up and is running within an Azure Private Link network.
• A source topic was created to sink data into the Snowflake database.
• Snowflake instance was created to sink data into and is running within the same region and cloud as the Confluent Cloud cluster.
• A database and a schema were created in Snowflake to sink data into.
• Snowflake imposes restrictions on which DNS hostnames can be used to connect. Be sure to use the hostnames mentioned in Azure Private Link and Snowflake.

Note

For added security, you can set up a Network rule within Snowflake to restrict incoming traffic to the specific Private Endpoint setup as part of Egress Private Link Endpoints.

Step 1. Create a gateway in Confluent Cloud (for Enterprise cluster only)

If using an Enterprise cluster in Confluent Cloud, you must set up a gateway in Confluent Cloud as described in Create a gateway for outbound connectivity in Confluent Cloud.

Step 2. Request the Private Link Service ID from Snowflake

Open a support case with Snowflake and request the Private Link Service ID associated with your Snowflake account. The ID should be in the following format:

/subscriptions/<subscription_ID>/resourceGroups/azure<region>-privatelink/providers/Microsoft.Network/privateLinkServices/sf-pvlinksvc-azure<region>

Note that you cannot use the Resource Alias for setting up an Egress Private Link Endpoint.

Leave the support ticket open as it will be required in a later step for Snowflake to allowlist Confluent’s Private Link Endpoint Resource ID.

Step 3. Create an Egress Private Link Endpoint

1. In the Network Management tab of the desired Confluent Cloud environment, click the Confluent Cloud network you want to add the Private Link Endpoint to. The Connection Type of the network needs to be “Private Link Access”.

2. Click Create endpoint in the Egress connections tab.

3. Click the service you want to connect to, specifically, Snowflake.

4. Follow the guided steps to specify the field values, including:

   • Name: The name of the Private Link Endpoint.

   • Resource ID: The resource ID of the Private Link service you retrieved in Obtain Azure Private Link Resource ID.

     Note that the resource alias is not supported.

   • Sub-resource name: The sub-resource name for the specific Azure service you retrieved in Obtain Azure Private Link Resource ID.

5. Click Create to create the Private Link Endpoint.

6. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and then click Finish.

Step 4. Request Snowflake support to allow Confluent’s Private Link Endpoint Resource ID

1. When the PrivateLink Endpoint you created in the previous step transitions from “Provisioning” to “Pending Accept”, request that Snowflake allowlist Confluent’s Resource ID using the earlier support ticket opened with Snowflake.
2. When Snowflake support has approved this request, the status of the Access Point with transition from “Pending Accept” to “Ready”.

Step 5. Create a DNS record

1. Obtain the required Domain by running the following within query within Snowflake.

   USE ROLE ACCOUNTADMIN;
   SELECT KEY, VALUE::VARCHAR HOST
   FROM TABLE(FLATTEN(INPUT=>PARSE_JSON(SYSTEM$GET_PRIVATELINK_CONFIG())));
   

2. Note down the HOST values for the privatelink-account-url and the regionless-privatelink-account-url KEY values. These are required based on how you want to connect:

   • Connect using Snowflake’s Account Locator (Legacy) URL (<account_name>.<region_id>.privatelink.snowflakecomputing.com)
   • Connect using Account Name URL (<org_name>-<account_name>.privatelink.snowflakecomputing.com)

3. In the Confluent Cloud Console, in the DNS tab, click Create record on the associated PrivateLink Endpoint.

   

4. Specify the following:

   • Ensure that the correct PrivateLink Endpoint is selected and put in the applicable Private Link Snowflake Domain.
   • Endpoint: Select the PrivateLink Endpoint you created in Step #2.
   • Domain: Specify the Domain you retrieved in Snowflake, using one of the below formats.
     • Account Locator (Legacy): <account_name>.<region_id>.privatelink.snowflakecomputing.com
     • Account Name: <org_name>-<account_name>.privatelink.snowflakecomputing.com
   

5. Click Save to create the record.

Step 6. Create the Snowflake Sink connector

1. Specify the authentication details for Snowflake.

   For Connection URL, specify the Snowflake’s private endpoint URL in one of the two possible formats:

   • Account Locator URL (Legacy): https://<account_name>.<region_id>.privatelink.snowflakecomputing.com
   • Account Name URL : https://<org_name>-<account_name>.privatelink.snowflakecomputing.com
   

2. Follow the steps to create the sink connector in Confluent Cloud.