Egress PrivateLink Endpoint Setup: MongoDB CDC Source on AWS for Confluent Cloud

This topic presents the steps for configuring the MongoDB CDC Source connector in Confluent Cloud with AWS PrivateLink and Egress PrivateLink Endpoint.

Prerequisites

The following is a list of prerequisites for configuring the MongoDB CDC Source connector with an Egress PrivateLink Endpoint:

• In Confluent Cloud, one of the following cluster types was set up with the specified network resource:

  • A Dedicated cluster with a Confluent Cloud network

    For the steps to create a Confluent Cloud network, see Create a Confluent Cloud network. The Connection type of the network needs to be “PrivateLink Access”.

  • A Enterprise cluster with a network gateway

    For the steps to create a gateway, see Create a gateway for outbound connectivity in Confluent Cloud.

• A Dedicated MongoDB Atlas database was created to move data from, and it is running within the same region and cloud as the Confluent Cloud cluster.

• A Kafka topic was created to receive data from MongoDB Atlas.

Step 1. Create a MongoDB private endpoint

In MongoDB, create a private endpoint for your Dedicated MongoDB Atlas database:

1. Go to Network Access → Private Endpoint → Dedicated Cluster.

2. Click Add Private Endpoint.

3. Select the applicable cloud provider and region of your Dedicated MongoDB Atlas database.

4. Make a note of the Atlas Endpoint Service name which is required for creating a Confluent Cloud Egress PrivateLink Endpoint.

   

5. You can close the workflow for now. Edit the rest of the configuration after you create a VPC endpoint ID in Confluent Cloud.

Step 2. Create an Egress PrivateLink Endpoint

Dedicated cluster

1. In the Confluent Cloud Console, in the Network management page or tab of the desired Confluent Cloud environment, click the Confluent Cloud network you want to add the PrivateLink Endpoint to. The Connection Type of the network needs to be “PrivateLink Access”.

2. Click Create endpoint in the Egress connections tab.

3. Click the service you want to connect to, specifically, MongoDB.

4. Follow the guided steps to specify the field values, including:

   • Name: Name of the PrivateLink Endpoint.

   • PrivateLink service name: The name of the PrivateLink service you retrieved in Step 1.

   • Create an endpoint with high availability: Check the box if you wish to deploy an endpoint with High Availability.

     Endpoints deployed with high availability have network interfaces deployed in multiple availability zones.

5. Click Create to create the PrivateLink Endpoint.

6. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and then click Finish.

7. After the state of the Egress PrivateLink Endpoint changes from Provisioning to Pending, make a note of the Endpoint ID. The value is required to finish setting up a private endpoint for MongoDB Atlas.

   

Enterprise cluster

1. In the Network Management page or tab of the desired Confluent Cloud environment, click the For serverless products tab.

2. Click the gateway to which you want to add the PrivateLink Endpoint.

3. In the Access points tab, click Add access point.

4. Click the service you want to connect to. Select Other if you do not see the specific service.

   Specific services are listed based on the cloud provider for the gateway.

5. Follow the guided steps to specify the field values, including:

   • Access point name: Name of the PrivateLink Endpoint.

   • Create an endpoint with high availability: Check the box if you wish to deploy an endpoint with High Availability.

     Endpoints deployed with high availability have network interfaces deployed in multiple availability zones.

   • PrivateLink service name: The name of the PrivateLink service you retrieved in Step 1.

6. Click Create access point to create the PrivateLink Endpoint.

7. If there are additional steps for the specific target service, follow the prompt to complete the tasks, and click Finish.

Step 3. Configure the MongoDB private endpoint

Note

You must create a DNS record for self-managed MongoDB clusters. This is not required for MongoDB Atlas clusters.

1. In the MongoDB console, click Edit for the previously created MongoDB private endpoint.

2. Click Next.

3. In Your VPC Endpoint ID field, specify the Endpoint ID associated with your Confluent Cloud Egress PrivateLink Endpoint, and click Save.

   You can enter the Endpoint ID in the MongoDB console as shown below:

   

4. After the Endpoint Status changes to “Available”, you are ready to create the Confluent Cloud MongoDB CDC Source connector.

Step 4. Create the MongoDB CDC Source connector

1. Specify the authentication details for MongoDB.

   1. In the MongoDB Atlas console, click Database under Deployment.

   2. Click Connect for the associated database.

   3. Select the Private Endpoint connection type, and select the endpoint you created in Step 3.

   4. Click Choose a connection method.

   5. Select Shell in the Access your data through tools section.

   6. Get the endpoint.

      

   7. In Confluent Cloud, specify the MongoDB credentials during the connector creation steps.

      In the MongoDB Connection String field, specify the endpoint you retrieved in the previous step.

      Note

      When you configure a private endpoint, ensure that the standard hostname matches the correct PrivateLink endpoint, for example “-pl-0” as shown in the screen below.

      

2. Follow the steps to create the MongoDB CDC Source connector in Confluent Cloud.