Access Confluent Cloud Console with Private Networking

When you enable a Confluent Cloud cluster with private networking, some Confluent Cloud Console workflows, including topic management and ksqlDB query management, use cluster endpoints that are not publicly accessible.

To use Confluent Cloud Console with your client computer, you need a connection to the network that hosts the private cluster endpoints (for AWS, Azure, or Google Cloud) and must update your DNS configuration for topic or ksqlDB endpoints to use that connection.

Resolve Confluent Cloud Console connectivity issues

Use the following tables to find a solution that might resolve your connectivity issues related to Confluent Cloud Console access. For details, click the applicable scenario and complete the required steps.

Scenario From an internet-based client From a VPC or VNet using VPN tunnel, Direct Connect, ExpressRoute, or Cloud Interconnect From a VPC or VNet
Proxy N/A No additional configuration required
Reverse SSH tunnel N/A No additional configuration required
Local DNS resolution N/A N/A

Configure a proxy

To enable access to Confluent Cloud Console, you can use a proxy (NGINX, HAProxy, Envoy, or others) for DNS resolution and to relay communication between your clients and Confluent Cloud resources.

In the following example topology, the client is running outside of a VPC or VNet and uses a NGINX proxy to connect to Confluent Cloud. Other proxies can be used, but the configuration example uses NGINX.

Simplified proxy configuration

To install and configure the proxy, complete the following steps:

  1. Install NGINX on an instance in the VPC or VNet connected to Confluent Cloud.

  2. On your client, modify the NGINX configuration file (/etc/nginx/nginx.conf) to use SNI from incoming TLS sessions for routing traffic to the appropriate servers on ports 443 and 9092.

    events {}
    stream {
      map $ssl_preread_server_name $targetBackend {
         default $ssl_preread_server_name;
     }
    
     server {
       listen 443;
    
       proxy_connect_timeout 1s;
       proxy_timeout 7200s;
       resolver 127.0.0.53;
    
       proxy_pass $targetBackend:443;
       ssl_preread on;
     }
    }
    
  3. Configure your DNS provider to resolve requests to Confluent Cloud to the proxy server. In this example, /etc/hosts is updated for local DNS resolution on the client used to access Confluent Cloud Console.

    ##
    # Host Database
    #
    # localhost is used to configure the loopback interface
    # when the system is booting.  Do not change this entry.
    ##
    127.0.0.1       localhost
    255.255.255.255 broadcasthost
    ::1             localhost
    <Public IP Address of VM instance> <bootstrapURL>
    
  4. Ensure that the NGINX status is active.

    You can verify the NGINX status by running the following command:

    $ sudo service nginx status
    
    nginx.service - A high performance web server and a reverse proxy server.
    
    Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
    
    Active: active (running) since Sun 2022-06-05 03:11:58 UTC; 9min ago
    

    If NGINX is not active, run the following command:

    sudo service NGINX start
    
  5. Go to the Confluent Cloud Console and verify that topics and ksqlDB query management are now visible under the Topics and ksqlDB tabs for your cluster.

Configure a reverse SSH tunnel

Complete the steps below to configure a reverse SSH tunnel and custom DNS records to access private cluster endpoints from a client on the internet or the on-premises network. Any DNS solution can be used.

Local traffic destined for your cluster is forwarded through a bastion hosted in your VPC or VNet over the specified port through the reverse SSH tunnel.

  1. Provision a bastion virtual machine in your VPC or VNet.

  2. Start the SSH service in your bastion.

  3. Establish a reverse SSH tunnel from the client machine to the bastion by executing the command below.

    You can find the bootstrap URL in the Confluent Cloud Console under Cluster Settings or in the red error banner (appearing if connectivity is not established). For ksqlDB, the URL can be found under Settings in the ksqlDB section.

    sudo ssh -L 443:<boostrapURL>:443 <username>@<instance-public-DNS>
    
    sudo ssh -L 443:<ksqlDB endpoint>:443 <username>@<instance-public-DNS>
    
  4. Configure DNS to resolve Kafka API or ksqlDB query requests.

    The following example uses /etc/hosts for local DNS resolution.

    ##
    # Host Database
    #
    # localhost is used to configure the loopback interface
    # when the system is booting.  Do not change this entry.
    ##
    127.0.0.1       localhost
    255.255.255.255 broadcasthost
    ::1             localhost
    <Public IP Address of VM instance> <bootstrapURL>
    <Public IP Address of VM instance> <ksqlDB>
    

Configure your local DNS resolution

If connectivity can be established between the web browser and Confluent Cloud, DNS provider updates are needed to route web browser requests to the UI endpoint and then to the private endpoint connected to Confluent Cloud.

Name Value

<UI endpoint>

Example: lkaclkc-k5w62-z6wj0p.westus2.azure.glb.confluent.cloud

<Private endpoint IP address>

<KsqlEndpoint>

Example: pksqlc-2jrvq-z6wj0p.westus2.azure.glb.confluent.cloud

<Private endpoint IP address>