Configuring Access to the Confluent Cloud Web UI with VPC peering

When VPC peering is enabled, the Confluent Cloud web UI components like topic management and ksqlDB are set up with private endpoints that are not publicly reachable. You must configure internal access to these components.

Important

You might have to configure multiple endpoints for topic management, consumer lag, and ksqlDB.

Example Topology

In this example topology, the customer network is running outside of a cloud VPC using HAProxy to connect to Confluent Cloud. Other proxies such as NGINX or Envoy could be used instead, but the sample configuration below is based on HAProxy.

../_images/cloud-vpc-topology.png

Example topology

HAProxy Example Configuration

Here is an example HAProxy configuration that configures access to topic management.

  1. Append these values to your HAProxy configuration file (/etc/haproxy/haproxy.cfg):

    • Bind port *:443 for front end listening
    • Configure front end to back end mapping
    frontend confluent-cloud-topic-mgmt
    mode tcp
    bind *:443
    log global
    default_backend confluent-cloud-topic-mgmt
    
    backend confluent-cloud-topic-mgmt
    mode tcp
    server topic1 <your API endpoint>:443 check
    

    The Confluent Cloud UI will display the required API endpoint in an alert message. You can also run the ccloud kafka cluster describe command to determine the API endpoint.

  2. Configure the DNS entry to point to the HAProxy front end for topic management endpoint. Here is an example that uses Amazon Route 53. Note that pkac-**** is mapped to the front end of HAProxy:

    $ aws route53 list-resource-record-sets --hosted-zone-id Z03406652PN3OVDPNQJP0
    {
        "ResourceRecordSets": [
            {
                "TTL": 172800,
                "ResourceRecords": [
                    {
                        "Value": "ns-1536.awsdns-00.co.uk."
                    },
                    {
                        "Value": "ns-0.awsdns-00.com."
                    },
                    {
                        "Value": "ns-1024.awsdns-00.org."
                    },
                    {
                        "Value": "ns-512.awsdns-00.net."
                    }
                ],
                "Type": "NS",
                "Name": "eu-west-1.aws.confluent.cloud."
            },
            {
                "TTL": 900,
                "ResourceRecords": [
                    {
                        "Value": "ns-1536.awsdns-00.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
                    }
                ],
                "Type": "SOA",
                "Name": "eu-west-1.aws.confluent.cloud."
            },
            {
                "TTL": 300,
                "ResourceRecords": [
                    {
                        "Value": "10.10.1.115"
                    }
                ],
                "Type": "A",
                "Name": "pkac-4nvdd.eu-west-1.aws.confluent.cloud."
            }
        ]
    }
    

Tip

  • For more information on HAProxy hardware and operating system requirements, see HAProxy operating system and hardware requirements.

  • You can also edit the /etc/hosts file in Linux or macOS, for example:

    10.10.1.115 pkac-4nvdd.eu-west-1.aws.confluent.cloud
    

ksqlDB Example Configuration

For the ksqlDB UI to work in a VPC-peered Confluent cluster, each ksqlDB application has its own URL. You must configure the proxy to include the URL of every application by appending the following values to your HAProxy configuration file. Replace <your-topic-endpoint> and <your-ksqlDB-endpoint> with the appropriate values for your application.

frontend kafka
    mode tcp
    bind *:443
    log global

    tcp-request inspect-delay 5s
    tcp-request content accept if { req.ssl_hello_type 1 }

    acl is_topic req.ssl_sni -i <your-topic-endpoint>
    acl is_ksqlDB req.ssl_sni -i <your-ksqlDB-endpoint>

    use_backend topic if is_topic
    use_backend ksqlDB if is_ksqlDB

backend topic
    mode tcp
    server topic <your-topic-endpoint>:443 check

backend ksqlDB
    mode tcp
    server ksqlDB <your-ksqlDB-endpoint>:443 check