Egress Private Service Connect Endpoints Setup Guide: First-Party Services on Google Cloud for Confluent Cloud

This topic presents the steps for setting up Egress Private Service Connect Endpoints for the first-party services on Google Cloud to enable the use of fully managed connectors in Confluent Cloud using Google Cloud Private Service Connect.

First-party services are the services that Google Cloud directly offers and supports.

Requirements

A Confluent Cloud Dedicated cluster must be set up, and the cluster is running within a Private Services Connect network.

Step 1: Create an Egress Private Service Connect Endpoint

  1. In the Confluent Cloud Console, navigate to the Network Management tab of the desired Confluent Cloud environment.

  2. Click the Confluent Cloud network to which you want to add the Private Service Connect Endpoint.

    Verify that the Connection Type of the network is “Private Service Connect”.

  3. Click Create endpoint in the Egress connections tab.

  4. Click Global Google APIs as the service you want to connect to.

  5. Follow the guided steps to specify the field values, including:

    • Service: The name of the Private Service Connect service you retrieved in Obtain Google Cloud Private Service Connect Endpoint target.

    • Endpoint name: Name of the Private Service Connect Endpoint.

    • Private Service Connect Endpoint Target:

      When Global Google APIs was selected in the previous step, the value is preset to all-google-apis.

  6. Click Create endpoint to create the Private Service Connect Endpoint.

Step 2: Create the DNS record

  1. When the Egress Private Service Connect Endpoint status transitions to “Ready”, click Create record under the DNS records section.

    ../../_images/gcp-1st-party-create-dns-record.png

  2. On the DNS record page, ensure that the Domain field is set accordingly.

    • If not using OAuth authentication, set the Domain field to googleapis.com, which should be the pre-populated value.

      ../../_images/gcp-1st-party-dns-record.png

    • If OAuth is used for connector authentication against the external system, update the Domain field specific to the Google Cloud service that the connector will be connecting to. For example:

      • For BigQuery: bigquery.googleapis.com
      • For Pub/Sub: pubsub.googlesapis.com

      Note

      The connector will fail with the following error message if OAuth is in use and there is an existing DNS record of googleapis.com:

      io.confluent.connect.oauth.ConnectOAuthException: An error occurred
when attempting to refresh access_token.

      To resolve the error, remove the googlesapis.com DNS record and replace with service-specific DNS record.

  3. Click Save.

Step 3: Create the connector

  1. When the Endpoint and DNS Record status transitions to “Ready”, proceed to create the connector.
  2. For the steps to create the connector, refer to the connector-specific documentation that is listed for your specific connector in Supported connectors.