# Use Google Cloud VPC Network Peering with Confluent Cloud Google Cloud VPC Network Peering enables you to route traffic using private IPv4 addresses between your Virtual Private Cloud (VPC) and Confluent Cloud. VPC peers can communicate with each other as if they are within the same network. For more information about, see [VPC Network Peering](https://cloud.google.com/vpc/docs/vpc-peering). [Managed connectors](../../connectors/overview.md#kafka-connect-cloud) created in a VPC-peered cluster can access data sources and sinks hosted in all peered VPCs, if the firewall rules allow connector traffic to and from the peered VPCs. ## Requirements and considerations * A [Confluent Cloud network](../overview.md#ccloud-network-overview) of the “VPC Peering” type and the “Google Cloud” provider. If a network does not exist, create a PEERING type network as described in [Create Confluent Cloud Network on Google Cloud](../ccloud-network/google-cloud.md#create-ccloud-network-gcp). * Your VPC subnets and Confluent Cloud must be in the same same region. Cross-region access to Confluent Cloud is not supported when VPC Network Peering is enabled with Google Cloud. * Transitive VPC peering is not supported. If you peer Network A to Network B, and peer Network B to Confluent Cloud, applications running in Network A will not be able to access Confluent Cloud. Although they don’t provide transitive routing, shared Google Cloud VPCs can be leveraged to enable Confluent Cloud connectivity. For more information, see [Google Cloud Shared VPC overview](https://cloud.google.com/vpc/docs/shared-vpc). * You can colocate multiple Confluent Cloud Dedicated clusters in the same Confluent Cloud network, but this is limited by the expected number and size of these clusters. The applicable limits are specified in [Networks](../../quotas/service-quotas.md#ccloud-resource-limits-network). * If using Google Cloud [Organization policy constraints](https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints) to restrict VPC Network Peering usage, contact Confluent Support to obtain the Confluent Cloud Organization ID in Google Cloud. Then whitelist the Organization ID in the `constraints/compute.restrictVpcPeering` policy in Google Cloud. ## Create a VPC Network Peering connection To establish a VPC Network Peering connection between Confluent Cloud and Google Cloud: 1. Create [a VPC Network Peering connection in Confluent Cloud](#gcp-peering-connection-ccloud). 2. Create [a VPC Network Peering connection in Google Cloud](#gcp-peering-connection-gcloud). ### Step 1: Create a VPC Network Peering connection in Confluent Cloud You need to gather the following information from the [Google Cloud console](https://console.cloud.google.com): * The Google Cloud project ID associated with the VPC that you are peering to Confluent Cloud. * The network name of the VPC that you are peering with Confluent Cloud. * You might need to increase your route quota when you use VPC Network Peering because the Confluent Cloud and Google Cloud routes are shared. Follow the steps to create a VPC Network Peering connection in Confluent Cloud. ### Confluent Cloud Console 1. In the **Network Management** tab of the desired Confluent Cloud environment, click the **For dedicated cluster** tab. 2. Click the Confluent Cloud network to which you want to add the peering connection. 3. In the **Ingress connections** tab, click **+ VPC Peering**. 4. Specify the following field values. * **Name**: : The name of this connection. * **GCP Project ID**: The project ID associated with the VPC that you are peering to Confluent Cloud. * **GCP Network Name**: The network name of the VPC that you are peering with Confluent Cloud. * **Import custom routes**: Select to import static and dynamic custom routes over this VPC peering connection. 5. Click **Add** to create the peering connection. Peering connection provisioning will take a few minutes to complete. Your peering connection status will transition from “Provisioning” to “Waiting for connection” in the Confluent Cloud Console. ### Confluent REST API A peering connection has to be created from your VPC to the Confluent Cloud network in order to access Confluent Cloud clusters and services in a Confluent Cloud network. **HTTP POST request** ```text POST https://api.confluent.cloud/networking/v1/peerings ``` **Authentication** See [Authentication](https://docs.confluent.io/cloud/current/api.html/#authentication). **Request specification** ```json { "spec":{ "cloud":{ "kind":"GcpPeering", "project":"my-project", "vpc_network":"gcp-vpc-peering", "import_custom_routes":false }, "display_name":"My-GCP-Peering-1", "environment":{ "id":"env-y0000w" }, "network":{ "id":"n-000000" } } } ``` ### Confluent CLI Use the [confluent network peering create](https://docs.confluent.io/confluent-cli/current/command-reference/overview.html) Confluent CLI command to create a peering connection: ```bash confluent network peering create gcp-peering ``` The following command-specific flags are supported: * `--network`: Required. Confluent Cloud network ID. * `--cloud`: Required. The cloud provider. Set to `gcp`. * `--cloud-account`: Required. Google Cloud project ID associated with the VPC that you are peering with Confluent Cloud network. * `--virtual-network`: Required. Name of the Google Cloud VPC that you are peering with Confluent Cloud network. * `--gcp-routes`: Enable customer route import for Google Cloud VPC Network Peering. You can specify additional optional CLI flags described in the [Confluent CLI command reference](https://docs.confluent.io/confluent-cli/current/command-reference/overview.html), such as `--environment`. The following is an example Confluent CLI command to create a VNet peering: ```bash confluent network peering create gcp-peering \ --network n-123456 \ --cloud gcp \ --cloud-account temp-123456 \ --virtual-network customer-test-vpc-network \ --gcp-routes ``` ### Terraform Use the [confluent_peering](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_peering) resource to create a peering connection. An example snippet of Terraform configuration: ```terraform resource "confluent_environment" "development" { display_name = "Development" lifecycle { prevent_destroy = true } } resource "confluent_network" "gcp-peering" { display_name = "GCP Peering Network" cloud = "GCP" region = "us-west4" cidr = "10.10.0.0/16" connection_types = ["PEERING"] environment { id = confluent_environment.development.id } lifecycle { prevent_destroy = true } } resource "confluent_peering" "gcp" { display_name = "GCP Peering" gcp { project = "temp-gear-123456" vpc_network = "customer-test-vpc-network" } environment { id = confluent_environment.development.id } network { id = confluent_network.gcp-peering.id } lifecycle { prevent_destroy = true } } ``` See more Terraform configuration examples for creating a Google Cloud peering connection using Terraform: * [Peering with ACLS](https://github.com/confluentinc/terraform-provider-confluent/tree/master/examples/configurations/dedicated-vpc-peering-gcp-kafka-acls) * [Peering with RBAC](https://github.com/confluentinc/terraform-provider-confluent/tree/master/examples/configurations/dedicated-vpc-peering-gcp-kafka-rbac) ### Step 2: Create a VPC Network Peering connection to Confluent Cloud in Google Cloud In order to complete setup, you must initiate a peering request to Confluent Cloud from Google Cloud VPC. You need to gather the following information from the Confluent Cloud network and the peering connection you created in the previous section: * Confluent Cloud GCP Project ID: Your Confluent Cloud Project ID. You can find the value in the Confluent Cloud Console under **Network overview** of your Confluent Cloud network as the **Confluent Cloud GCP Project ID**. * VPC network name: Your Confluent Cloud VPC name. You can find the value in the Confluent Cloud Console under **Network overview** of your Confluent Cloud network as the **Confluent Cloud VPC Network Name**. Follow the steps to create a VPC network peering connection in Google Cloud. 1. In [VPC network in the Google Cloud Console](https://console.cloud.google.com/networking/networks/list), select **VPC network peering**. 2. Click **CREATE CONNECTION** to create a peering connection to Confluent Cloud. 3. Specify the following field values: * **Name**: The name for this peering connection. * **Your VPC network**: The name of your Google Cloud VPC network. * **Peered VPC network**: Select **In another project**. * **Project ID**: The Confluent Cloud GCP Project ID you retrieved in Confluent Cloud. * **VPC network name**: The Confluent Cloud VPC name you retrieved in Confluent Cloud. 4. Click **CREATE** to initiate a peering connection to Confluent Cloud. 5. Verify that the new connection turns **Active** in Google Cloud. 6. Verify that the **Status** under **VPC Peering connections** is “Ready” in Confluent Cloud. ## Import custom routes The Import Custom Routes option in Confluent Cloud enables connectivity to a Confluent Cloud cluster from customer premise or other clouds, such as AWS and Azure. This connectivity is enabled by importing static and dynamic custom routes from a customer VPC into a Confluent Cloud VPC over the VPC Network Peering connection. Review the considerations mentioned by Google Cloud in their [VPC Network Peering documentation](https://cloud.google.com/vpc/docs/vpc-peering#considerations) before enabling Import Custom Routes option. The requirements and considerations for the Import Custom Routes option are: * The customer side VPC Network Peering has to be configured to [export custom routes](https://cloud.google.com/vpc/docs/using-vpc-peering#creating_a_peering_configuration). * Transitive routing to your VPCs in the same or different regions is not supported. * The only exception is when cross-regional VPCs are interconnected using [Google Cloud VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview). However, your VPC, which is peered with Confluent Cloud cluster, must be in the same region as Confluent Cloud cluster. * In Confluent Cloud, the global access option for the [Google Cloud Internal LoadBalancer](https://cloud.google.com/load-balancing/docs/internal/internal-tcp-udp-lb-and-other-networks) is not supported. * You cannot export Custom Routes from Confluent Cloud. * Privately used public IP (PUPI) addresses are not supported with the Import Custom Routes option. The Import Custom Routes option must be enabled when you set up the VPC Network Peering connection. * To enable Import Custom Routes option on an existing VPC Network Peering connection: Tear down the VPC peering connection and reestablish it with the Import Custom Routes option enabled. Allow 15 minutes between tearing down the VPC connection and reestablishing it to avoid getting an error message during recreation. * To disable the Import Custom Routes option on an existing VPC Network Peering connection: Tear down the VPC peering connection and reestablish it with the **Import Custom Routes** option disabled. Allow 15 minutes between tearing down the VPC connection and reestablishing it to avoid getting an error message during recreation. As an alternative, disable the **Export Custom Route** option in the customer VPC. ## DNS resolution Confluent Cloud hosts and manages the DNS records for endpoints which use Google Cloud VPC Network Peering connectivity. In order for your Kafka clients to resolve these DNS names, they must be able to resolve records from public DNS servers. Confluent does not provide a supported private DNS solution for the clusters that use Google Cloud VPC Network Peering. ## Configure DNS forwarding To resolve hostnames that reside within private DNS zones or a self-hosted DNS server and access your own VPC or on-prem from Confluent Cloud, set up DNS forwarding in Confluent Cloud. For example, you can use DNS forwarding for Confluent Cloud fully managed connectors that need to access data in your VPC. Confluent will forward DNS requests for a specified list of domains to Cloud DNS APIs of Google Cloud for resolution. You must grant Confluent’s IAM principal permission to do these lookups.  ### Step 1: Obtain Confluent’s IAM Principal Confluent Cloud uses a unique IAM principal to make calls to the Cloud DNS API of Google Cloud for each environment you configure DNS forwarding for. To obtain the IAM principal: ### Confluent Cloud Console 1. In Confluent Cloud, in the **Network management** tab, click **For dedicated clusters**. 2. Select the Confluent Cloud cluster you want to use. 3. Navigate to **Cluster Overview** -> **Networking**. 4. In the **Network details** section of the Peering type network you want to use, copy the value in the **IAM principal** field. An example value: `service-account@example.iam.gserviceaccount.com` ### Confluent REST API 1. Issue the following API request to get the gateway id. **HTTP GET request** ```rest GET https://api.confluent.cloud/networking/v1/networks/{Confluent Cloud network ID} ``` **Authentication** See [Authentication](https://docs.confluent.io/cloud/current/api.html/#authentication). You can find the gateway id in the response under `spec.gateway.id`. 2. Send a request to get the IAM principal id associated with the gateway. Use the gateway id you retrieved in the previous step. **HTTP GET request** ```text POST https://api.confluent.cloud/networking/v1/gateways/{gateway-id} ``` ### Step 2: Grant Confluent’s IAM Principal read permissions in Cloud DNS Next, you must grant Confluent’s IAM principal the **DNS Reader** role in each project that you have DNS queries to send to. ### Google Cloud console To grant permissions using the Google Cloud Console: 1. Navigate to the IAM & Admin product page. 2. Select the IAM sub-category. 3. In the **Allow** tab, click **Grant Access**. 4. In the **New Principals** field, paste the Confluent IAM principal that you retrieved in [Step 1: Obtain Confluent’s IAM Principal](#dns-forwarding-gcp-obtain-iam). 5. In the **Select a role** field, select “DNS Reader”. 6. Click **Save**. ### gcloud CLI To grant permissions using the gcloud CLI, using the Confluent IAM principal that you retrieved in [Step 1: Obtain Confluent’s IAM Principal](#dns-forwarding-gcp-obtain-iam): ```bash gcloud projects add-iam-policy-binding [PROJECT_ID] \ --member "user: [Confluent IAM principal]" \ --role="roles/dns.reader" ``` ### Step 3: Create a DNS Forwarder in Confluent Cloud Set up DNS forwarding in Confluent Cloud. ### Confluent Cloud Console Repeat the following steps for each domain you wish to forward. 1. In Confluent Cloud, navigate to the Confluent Cloud network and click **For dedicated clusters**. 2. In the **DNS forwarding** tab, click **Add configuration**. 3. Input the following information: * **Project ID:** The project where the relevant private DNS zone is located. * **Zone name:** The name of the DNS private zone. * **Domain:** The domain for which you wish to forward requests. 4. Click **Add** and wait until provisioning is complete and DNS is propagated. ### Confluent REST API 1. Get the gateway id using the following API request: ```rest GET https://api.confluent.cloud/networking/v1/networks/{Confluent Cloud network ID} ``` You can find the gateway id in the response under `spec.gateway.id`. 2. Send a request to create a DNS Forwarder resource: **REST request** ```rest POST https://api.confluent.cloud/networking/v1/dns-forwarders ``` **REST request body** ```json { "spec": { "display_name": "", "environment": { "id": "" }, "config": { "kind": "ForwardViaIp", "dns_server_ips": "" }, "domains": "", "gateway": { "id": "", "environment": "" } } } ``` ### Confluent CLI Use the [confluent network dns forwarder create](https://docs.confluent.io/confluent-cli/current/command-reference/overview.html) Confluent CLI command to set up a DNS forwarder: ```bash confluent network dns forwarder create ``` The following command-specific flags are supported: * `--dns-server-ip`: Required. A comma-separated list of IP addresses for the DNS server. * `--gateway`: Required. Gateway ID. To get the gateway id, run the following CLI command: ```bash confluent network describe ``` * `--domains`: A comma-separated list of domains for the DNS forwarder to use. You can specify additional optional CLI flags described in the [Confluent CLI command reference](https://docs.confluent.io/confluent-cli/current/command-reference/overview.html), such as `--environment` and `--output`. The following is an example Confluent CLI command to create a DNS forwarder: ```bash confluent network dns forwarder create \ --domains abc.com,def.com \ --dns-server-ips 10.200.0.0,10.201.0.0 \ --gateway gw-123456 ``` The following is an example Confluent CLI command to create a named DNS forwarder: ```bash confluent network dns forwarder create my-dns-forwarder \ --domains abc.com,def.com \ --dns-server-ips 10.200.0.0,10.201.0.0 \ --gateway gw-123456 ``` ### Terraform Use the [confluent_dns_forwarder](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_dns_forwarder) Confluent Terraform Provider resource to set up a DNS forwarder. An example snippet of Terraform configuration: ```terraform resource "confluent_environment" "development" { display_name = "Development" } resource "confluent_dns_forwarder" "main" { display_name = "dns_forwarder" environment { id = confluent_environment.development.id } domains = ["example.com", "domainname.com"] gateway { id = confluent_network.main.gateway[0].id } forward_via_ip { dns_server_ips = ["10.200.0.0", "10.200.0.1"] } } ``` See [Terraform configuration examples](https://github.com/confluentinc/terraform-provider-confluent/blob/master/examples/configurations/dns-forwarder/main.tf) for a full example of setting up DNS forwarding using Terraform.
A DNS forwarder can be in one of the following states. A DNS forwarder is successfully created when it is in the `CREATED` or `READY` state. * `PROVISIONING`: DNS forwarder provisioning is in progress. * `CREATED`: The DNS forwarder is created. It will automatically become ready once a Kafka cluster is provisioned. * `READY`: The DNS forwarder is ready. * `FAILED`: The DNS forwarder is in a failed state. * `DEGRADED`: The DNS forwarder is in a degraded state, transitioning from `READY` due to unreachable DNS resolvers. * `DEPROVISIONING`: The DNS forwarder is being de-provisioned.