Snowflake Sink Connector for Confluent Cloud with AWS Egress Access Point

This topic presents the steps for configuring the Snowflake Sink connector in Confluent Cloud with AWS PrivateLink and Egress Access Point.

Prerequisites

The following is a list of prerequisites for configuring the Snowflake Sink connector with an Egress Access Point:

• A Confluent Cloud Dedicated cluster was set up and is running within an AWS PrivateLink network.
• A source topic was created to sink data into the Snowflake database.
• Snowflake instance was created to sink data into and is running within the same region and cloud as the Confluent Cloud cluster.
• A database and a schema were created in Snowflake to sink data into.
• Snowflake imposes restrictions on which DNS hostnames can be used to connect. Be sure to use the hostnames mentioned in AWS PrivateLink and Snowflake.

Note

For added security, you can set up a Network rule within Snowflake to restrict incoming traffic to the specific VPC Endpoint setup as part of Egress Access Points.

Step 1. Request Snowflake to allowlist Confluent’s AWS account

1. In the Confluent Cloud Console, go to Environment → Network, and select the associated Privatelink network you want to use.

2. In the Egress Access Points tab, make note of Confluent’s AWS Account ID associated with the Access Point.

   

3. Open a support case with Snowflake with the following information to request that Confluent’s AWS Account ID be allowlisted for private connectivity.

   • Your Snowflake Account URL.
   • Confluent’s AWS Account ID associated with the Egress Access Point. You retrieved this in the previous step.

4. Wait to receive confirmation from Snowflake that Confluent’s AWS Account ID has been allowlisted before proceeding.

Step 2. Obtain the Snowflake Private Link Service ID

To obtain the Service ID associated with your Snowflake instance, execute the following statement from within your Snowflake account and make note of the value of privatelink-vpce-id:

USE ROLE ACCOUNTADMIN;

SELECT KEY, VALUE::VARCHAR VPCE_SERVICE_ID

FROM TABLE(FLATTEN(INPUT=>PARSE_JSON(SYSTEM$GET_PRIVATELINK_CONFIG())))
WHERE KEY = 'privatelink-vpce-id';

Step 3. Create an Egress Access Point

1. In the Confluent Cloud Console, go to Environment → Network, and select the associated Privatelink network you want to use.

2. In the Egress Access Points tab, click Create access point.

3. Specify the following, and click Save.

   • Name: The name for the Egress Access point.
   • PrivateLink service name: Your Snowflake service name (privatelink-vpce-id), you retrieved from the previous step.
   • Create an access point with high availability: Select if required.
   

Step 4. Create a DNS record

1. When the Access Point status transitions to “Ready”, in the DNS tab, click Create record on the associated Access Point.

   

2. Specify the following, and click Save.

   • Access point: Select the Access Point you created above.
   • Domain: privatelink.snowflakecomputing.com
   

Step 5. Create the Snowflake Sink connector

1. In the Confluent Cloud Console, when the DNS record status becomes “Ready”, go to your associated Dedicated cluster.

2. In the Connectors tab, click Snowflake Sink.

3. Select the source topic.

4. Specify the Kafka authentication mechanism.

5. Specify the authentication details for Snowflake.

   1. For the Connection URL, specify Snowflake’s private endpoint URL (https://<org_name>-<account_name>.privatelink.snowflakecomputing.com).

      

6. Specify configuration details for the connector.

7. Specify sizing (number of tasks) for the connector.

8. Review and launch the connector.

When the connector is successfully launched, the connector status becomes “Running”.