Custom Domains and Network Isolation in Confluent Private Cloud Gateway

Confluent Private Cloud Gateway enables you to map Kafka listener endpoints to custom domains and isolate network traffic using private networking solutions. These capabilities enhance security, flexibility, and compliance for private Kafka clusters by allowing you to expose branded, abstracted endpoints to clients, restrict access according to your policies, and separate internal broker networks from external access.

Custom Domain: Each Custom Domain is an isolated pathway, with its own entry point (endpoint), routing logic, and security enforcement mechanisms. By establishing and managing multiple Custom Domains, operators can create isolated network segments for different clients, departments, or applications, all controlled via the Gateway, removing the need for physical separation or complex client configuration.

Custom Domains provide the framework for Network Isolation.

Custom Domains can be configured using DNS records (for example, in AWS Route 53 or Azure Private Zones) or in the /etc/hosts file and mapped to the Gateway’s IP addresses (for local development purposes only).
Network Isolation: When Kafka brokers reside in a private network, unreachable directly by external clients, Confluent Gateway acts as the proxy access point, enforcing controlled entry.

Both Custom Domains and Network Isolation help you enable external partner access for your Kafka clusters/data as per your compliance policies.

Set up Network Isolation and Custom Domains for a private Kafka cluster

To set up a Custom Domain and Network Isolation:

Set up host resolution for the Custom Domain.
Configure the Confluent Gateway to use the Custom Domain.
Restart the Gateway to apply the changes.

For a complete example of setting up Network Isolation with a Custom Domain, see the Gateway Images repository.

Set up host resolution for the Custom Domain

Set up host resolution for the Custom Domain. Ensure that network and DNS settings allow clients to resolve this Custom Domain to the Gateway’s actual IP address.
- Case: Kafka clients and the Confluent Gateway are in the same VPC
  
  Create a private hosted zone / DNS zone in the VPC and create DNS records pointing to the private IP of the Confluent Gateway instance.
- Case: Kafka clients and the Confluent Gateway are in different VPCs
  1. Enable connectivity between the VPCs using Peering, Transit Gateway, or Private Link.
  2. Use the same private hosted zone / DNS zone, and associate it with both VPCs.
  3. Create DNS records pointing to the private IP of the Confluent Gateway instance.
- Case: Kafka clients are on an external network
  1. Create a public hosted zone / DNS zone.
  2. Create DNS records pointing to the public IP of the Confluent Gateway instance.
For details on creating a DNS zone and DNS records, see:

Configure the Confluent Gateway to use the Custom Domain

Update your Gateway configuration so that the route(s) use the Custom Domain name as their endpoint.
```
routes:
  - name:
    endpoint:        --- [1]
```
- [1] Specify the same Custom Domain name you used in step 1 above, and specify the port.

Restart the Gateway to apply the changes

Restart the Confluent Gateway instance to apply the changes:

docker compose -f docker-compose.yaml up -d

or

kubectl apply -f <Gateway_CR.yaml>

The restart results in client disconnections and reconnections.

Example setup

This sample setup demonstrates how to use a private hosted zone in Route 53 to achieve Custom Domain routing and Network Isolation when both Confluent Gateway and Kafka clients reside in the same AWS VPC, and no public access is required.

Browse to AWS Console → Route 53 → Create Hosted Zone.
Select Private Hosted Zone, and enter your Custom Domain name (for example, mycluster-proxy-route53.kafka).
Associate the VPC where both Confluent Gateway and Kafka clients run (for example, vpc-0f2b932796e3e97a4).
Ensure the VPC has DNS Resolution and DNS Hostnames enabled.
Inside the hosted zone, create A records pointing to the private IP of the Confluent Gateway EC2 instance (10.0.3.140):
- mycluster-proxy-route53.kafka → 10.0.3.140
- *.mycluster-proxy-route53.kafka → 10.0.3.140
  
  This is needed when brokerIdentificationStrategy.type is host in gateway.yaml.

Modify the Confluent Gateway configuration file to use the Custom Domain.

The following is an example configuration using the sample values from the above steps:

secretStores:
  - name: AWS
    provider:
      type: AWS
      config:
        region: us-east-1
        accessKey: <AWS_ACCESS_KEY>
        secretKey: <AWS_SECRET_KEY>
        separator: /
streamingDomains:
  - name: sample-domain
    type: kafka
    kafkaCluster:
      name: kafka-cluster-1
      bootstrapServers:
        - id: SASL_SSL-1
          endpoint: <KAFKA_BROKER_HOST:PORT>
routes:
  - name: sasl-plain-to-sasl-plain
    endpoint: mycluster-proxy-route53.kafka:10000
    brokerIdentificationStrategy:
      type: host
      pattern: "broker$(nodeId).mycluster-proxy-route53.kafka"
    streamingDomain:
      name: sample-domain
      bootstrapServerId: SASL_SSL-1

Start the Confluent Gateway instance and test producing and consuming.