.. _cloud-vpc: VPC Peering in |ccloud| ======================= You can use `virtual private clouds (VPC) `__ with |ccloud-ent| and |ccloud| Dedicated Clusters to maximize the security of your cloud infrastructure. All communication in :ccloud-cta:`Confluent Cloud|` is encrypted, but a VPC can decrease the available surface area for potential attackers. You can create VPCs with private IP `CIDR (Classless Inter-Domain Routing) `__ blocks and run your instances inside the VPCs on these private networks. The VPC can include applications and all of your cloud services. You can then peer your VPCs with Confluent VPCs so that you can access |ccloud| within the linked private networks. .. important:: - If you use VPC peering, your clusters will not have public endpoints and you can only access them from peered VPCs. - After a cluster has been provisioned with VPC peering, you cannot change the cluster to have public endpoints. Supported Features ------------------ |ccloud-ent| and |ccloud| Dedicated Clusters support the following VPC peering features: - Connect up to five VPCs to a single |ccloud| VPC. - |ccloud| VPC can contain any number of clusters. Limitations ----------- - For |gcp-long|, the customer VPC and |ccloud| VPC must be in the same region. - Transitive VPC and VNet peering are not supported. If you peer Network A to Network B, and peer Network B to |ccloud-ent|, applications running in Network A will not be able to access |ccloud-ent|. You can use shared |gcp| and |aws-long| VPCs to enable transitive VPC connectivity. For more information, see `AWS Working with Shared VPCs `__ and `GCP Shared VPC overview `__. To achieve transitivity, you can also link an |aws| Transit Gateway to a |ccloud| |aws| cluster. - You cannot directly connect from an on-premises datacenter to |ccloud|. To do this, you must first be routed to a shared services VPC or VNet that you own, and that is peered to |ccloud|. If you are interested in this environment for |ccloud|, contact your Confluent sales representative. Getting Started --------------- To implement VPC peering in your environment, order |ccloud-ent| or a |ccloud| Dedicated Cluster with VPC peering enabled. You will be asked to provide the following information to your Confluent representative. ----------------- |ccloud| on |aws| ----------------- Provide the following information to your Confluent representative. - The |aws| account ID associated with the VPC that you are peering to |ccloud|. - The VPC ID that you are peering with |ccloud|. - The |aws| region of the VPC that you are peering with |ccloud|. - The `CIDR `__ block of the VPC that you are peering with |ccloud|. This is used by |ccloud| to route traffic back to your network. Must be a private range. - The VPC `CIDR `__ block for |ccloud| to use. It must be a private /16 CIDR, per https://tools.ietf.org/html/rfc1918. It must not match or overlap with the CIDR used by your VPC. - You might need to increase your route quota when you use VPC peering in |aws-long| (|aws|), because the Confluent and |aws| routes are shared. For more information about VPC peering with |aws|, see `What Is Amazon VPC? `__. |ccloud| on |aws| Using Transit Gateway ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can enable |ccloud| for use on the |aws| Transit Gateway. To enable, you must provide the following information to your Confluent representative: #. The full |aws| Resource Name (ARN) for the AWS Resource Access Manager (RAM) Share-ID of the Transit Gateways that you want |ccloud| attached to. - Create a `transit gateway `__. The transit gateway must have ``Default route table association``, ``Default route table propagation``, and ``Auto accept shared attachments`` enabled. - Create a shared transit gateway using `AWS Resource Access Manager (RAM) share `__. The RAM share ID must be shared with the Confluent AWS Account: ``050879227952``. #. The VPC CIDR block for |ccloud| to use. - It must be a private /16 CIDR. - It must not match or overlap with the CIDR used by your VPC. - ``10.255.0.0/16`` and ``198.18.0.0/15`` are reserved by Confluent. - You might need to increase your route quota when you use VPC peering in |aws-long|, because the Confluent and |aws| routes are shared. After the |ccloud| clusters have been provisioned: #. Confluent accepts RAM share and attaches the |ccloud| VPC to the |aws| Transit Gateway. #. You can set up the desired routing in the |aws| Transit Gateway to route traffic to |ccloud|. ---------------------- |ccloud| on |gcp-long| ---------------------- Provide the following information to your Confluent representative. - The project ID associated with the VPC that you are peering to |ccloud|. - The network name of the VPC that you are peering with |ccloud|. - The VPC `CIDR `__ block for |ccloud| to use. It must be a private /16 CIDR, per https://tools.ietf.org/html/rfc1918. It must not match or overlap with the CIDR used by your VPC. - You might need to increase your route quota when you use VPC peering in |gcp-long| (|gcp|) because the |ccloud| and |gcp| routes are shared. For more information about VPC peering with |gcp|, see `VPC Network Peering `__. --------------------- |ccloud| on |az-long| --------------------- Provide the following information to your Confluent representative. - The Tenant ID associated with the VNet that you are peering to |ccloud|. Retrieve your Tenant/Directory ID in `portal.azure.com `__ under **Azure Active Directory -> Properties**. - The VNet ID of the VNet that you are peering with |ccloud|. Retrieve your VNet Resource ID in `portal.azure.com `__ under **Virtual Networks -> Target VNet -> Properties**. - The virtual network `CIDR `__ block for |ccloud| to use. It must be a private /16 CIDR, per https://tools.ietf.org/html/rfc1918. It must not match or overlap with any of the CIDRs used by your virtual network. Your Confluent representative will provide a setup script you must run before Confluent can peer to your VNet. For more information about VNet peering with |az|, see `Virtual Network Peering `__. .. _vpc-dashboard-access: Configuring Access to the UI Dashboard -------------------------------------- When VPC peering is enabled, the |ccloud| UI dashboard components like topic management and KSQL are set up with private endpoints that are not publicly reachable. You must configure internal access to these components. .. important:: You might have to configure multiple endpoints for topic management, consumer lag, and KSQL. ---------------- Example Topology ---------------- In this this example topology, the customer network is running outside of a cloud VPC using HAProxy to connect to |ccloud|. .. figure:: ../images/cloud-vpc-topology.png Example topology ------------------------------- |haproxy| Example Configuration ------------------------------- Here is an example |haproxy| configuration that configures access to topic management. #. Create an |haproxy| configuration file (``/etc/haproxy/haproxy.cfg``) with these values: - Bind port ``*:443`` for front end listening - Configure SNI routing of topic management endpoint ``pkac-***`` - Configure front end to back end mapping - Use the IP address ``10.10.1.115`` for the front end of HAProxy :: global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners stats timeout 30s user haproxy group haproxy daemon # Default SSL material locations ca-base /etc/ssl/certs crt-base /etc/ssl/private # Default ciphers to use on SSL-enabled listening sockets. # For more information, see ciphers(1SSL). This list is from: # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ # An alternative list with additional directives can be obtained from # https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS ssl-default-bind-options no-sslv3 defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http frontend kafka mode tcp bind *:9092 bind *:443 log global tcp-request inspect-delay 5s tcp-request content accept if { req.ssl_hello_type 1 } acl is_kafka0 req.ssl_sni -i b0-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_kafka1 req.ssl_sni -i b1-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_kafka2 req.ssl_sni -i b2-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_kafka3 req.ssl_sni -i b3-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_kafka4 req.ssl_sni -i b4-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_kafka5 req.ssl_sni -i b5-pkc-438qj.eu-west-1.aws.confluent.cloud acl is_topic1 req.ssl_sni -i pkac-4nvdd.eu-west-1.aws.confluent.cloud use_backend kafka0 if is_kafka0 use_backend kafka1 if is_kafka1 use_backend kafka2 if is_kafka2 use_backend kafka3 if is_kafka3 use_backend kafka4 if is_kafka4 use_backend kafka5 if is_kafka5 use_backend topic1 if is_topic1 backend kafka0 mode tcp server kafka0 b0-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend kafka1 mode tcp server kafka1 b1-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend kafka2 mode tcp server kafka2 b2-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend kafka3 mode tcp server kafka3 b3-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend kafka4 mode tcp server kafka4 b4-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend kafka5 mode tcp server kafka5 b5-pkc-438qj.eu-west-1.aws.confluent.cloud:9092 check backend topic1 mode tcp server topic1 pkac-4nvdd.eu-west-1.aws.confluent.cloud:443 check #. Configure the DNS entry to point to the |haproxy| front end for topic management endpoint. Here is an example that uses Amazon Route 53. Note that ``pkac-****`` is mapped to the front end of |haproxy|: :: $ aws route53 list-resource-record-sets --hosted-zone-id Z03406652PN3OVDPNQJP0 { "ResourceRecordSets": [ { "TTL": 172800, "ResourceRecords": [ { "Value": "ns-1536.awsdns-00.co.uk." }, { "Value": "ns-0.awsdns-00.com." }, { "Value": "ns-1024.awsdns-00.org." }, { "Value": "ns-512.awsdns-00.net." } ], "Type": "NS", "Name": "eu-west-1.aws.confluent.cloud." }, { "TTL": 900, "ResourceRecords": [ { "Value": "ns-1536.awsdns-00.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400" } ], "Type": "SOA", "Name": "eu-west-1.aws.confluent.cloud." }, { "TTL": 300, "ResourceRecords": [ { "Value": "10.10.1.115" } ], "Type": "A", "Name": "pkac-4nvdd.eu-west-1.aws.confluent.cloud." } ] } .. tip:: For more information on |haproxy| hardware and operating system requirements, see `HAProxy operating system and hardware requirements `__. .. _sr-ccloud-vpc: Using |sr-ccloud| in a VPC Peered Environment --------------------------------------------- If you have VPC peered environment and you want to use |sr-ccloud|, you must open outbound calls (egress) to a public |sr| endpoint. This is because |sr-ccloud| is a multi-tenant |sr|. Prerequisites |sr-ccloud| is :ref:`enabled and configured `. |sr-ccloud| is currently available as a preview. For more information, see :ref:`sr-prv`. #. Click **Schemas** on the left-side panel. On step 2 you should see the |sr| endpoint. For example ``https//confluent.us-east-2.aws.confluent.cloud``. .. image:: ../images/cloud-sr-view.png #. Open outbound calls to the |sr-ccloud| endpoint. Follow the instructions based on your cloud provider. |aws| VPC Configure outbound call access for these |aws| VPC networking components: - Follow the instructions in the |aws| `Internet Gateway documentation `_. - Follow the instructions in the |aws| `NAT Gateway documentation `_. - Follow the instructions in the |aws| `NAT Instance documentation `_. |gcp| VPC Configure outbound call access for |gcp| `networking components `_. #. .. include:: includes/sr-verify.rst