Access Confluent Cloud Console with Private Networking¶

When you enable a Confluent Cloud cluster with private networking, some Confluent Cloud Console workflows, including topic management and ksqlDB query management, use cluster endpoints that are not publicly accessible.

To use Confluent Cloud Console with your client computer, you need a connection to the network that hosts the private cluster endpoints (for AWS, Azure, or Google Cloud) and must update your DNS configuration for topic or ksqlDB endpoints to use that connection.

Use the following tables to find the right solution for Confluent Cloud service access. For details, click the applicable scenario and complete the required steps.

Scenario	From an internet-based client	From a VPC or VNet using VPN tunnel, Direct Connect, ExpressRoute, or Cloud Interconnect	From a VPC or VNet
Proxy	✔	N/A	No additional configuration required
Reverse SSH tunnel	✔	N/A	No additional configuration required
Local DNS resolution	N/A	✔	N/A

Scenario	From an internet-based client	From a VPC or VNet using VPN tunnel, Direct Connect, ExpressRoute, or Cloud Interconnect	From a VPC or VNet
Proxy	✔	✔	No additional configuration required
Reverse SSH tunnel	✔	✔	No additional configuration required
Local DNS resolution	N/A	N/A	N/A

Configure a proxy¶

To enable access to Confluent services in a privately-networked Confluent Cloud cluster from your local machine, you can use a proxy (NGINX, HAProxy, Envoy, or others). The proxy relays communication between your local clients and Confluent Cloud resources.

Important

Confluent Cloud hostnames must be maintained over a TCP proxy, otherwise TLS hostname validation will fail, and TLS will not be established. You must have a DNS record, or an entry in your /etc/host file, that points to the IP address of the proxy instance that routes to the appropriate Confluent endpoint.

In the following example topology, the client is running outside of a VPC or VNet and uses a NGINX proxy to connect to Confluent Cloud.

To install and configure the proxy, complete the following steps. The configuration steps use NGINX as an example:

Provision Virtual Machine (VM) in your VPC or VNet that is connected to Confluent Cloud.

You can set up with the default properties.
Install NGINX on an instance in the VPC or VNet connected to Confluent Cloud.

If you are on Ubuntu or Debian, use the below command to install the latest version of NGINX onto your VM. If you are using RedHat, use the yum command.
```
sudo apt update
```
```
sudo apt install nginx
```

Update the NGINX configuration file (/etc/nginx/nginx.conf) to use SNI from incoming TLS sessions for routing traffic to the appropriate servers on ports 443 and 9092.

events {}
stream {
  map $ssl_preread_server_name $targetBackend {
     default $ssl_preread_server_name;
 }

 server {
   listen 9092;

   proxy_connect_timeout 1s;
   proxy_timeout 7200s;

   # Run 'nslookup <ConfluentCloud_endpoint> 127.0.0.53' on nginx host to
   # verify resolver and check /var/log/nginx/error.log for any resolving
   # issues using 127.0.0.53
   resolver 127.0.0.53;

   # On lookup failure, reconfigure to use the cloud provider's resolver
   # resolver 169.254.169.253; # for AWS
   # resolver 168.63.129.16;  # for Azure
   # resolver 169.254.169.254;  # for Google

   proxy_pass $targetBackend:9092;
   ssl_preread on;
 }

 server {
   listen 443;
   proxy_connect_timeout 1s;
   proxy_timeout 7200s;
   resolver 127.0.0.53;
   proxy_pass $targetBackend:443;
   ssl_preread on;
 }

 log_format stream_routing '[$time_local] remote address $remote_addr'
                    'with SNI name "$ssl_preread_server_name" '
                    'proxied to "$upstream_addr" '
                    '$protocol $status $bytes_sent $bytes_received '
                    '$session_time';
 access_log /var/log/nginx/stream-access.log stream_routing;
}

Configure your DNS provider to resolve requests to Confluent Cloud to the proxy server.

In the following example, the /etc/hosts file is updated to include the public IP address of your VM mapped to your Confluent Cloud cluster endpoint. This will route traffic that is destined for your Confluent Cloud through the proxy.

Note that you update the /etc/hosts file of your local machine, not of the bastion host/proxy server.
```
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost
255.255.255.255 broadcasthost
::1             localhost
<Public IP Address of VM instance> <Kafka-REST-Endpoint>
```
The <Kafka-REST-Endpoint> can be retrieved in the Cluster settings page or in the red banner that shows up on your Confluent Cloud Console.
Verify that NGINX is running and is able to direct traffic to your cluster endpoint.
```
sudo systemctl status nginx
```
If NGINX is not active, run the following command:
```
sudo systemctl service nginx start
```
Go to the Confluent Cloud Console and verify that topics and ksqlDB query management are now visible under the Topics and ksqlDB tabs for your cluster.

Configure a reverse SSH tunnel¶

To a privately-networked Confluent Cloud cluster from a local client on the internet or the on-premises network, you can use a reverse SSH tunnel.

At a high level, the reverse SSH tunnel captures all traffic going towards your privately-networked Confluent Cloud cluster, then it routes that traffic over the specified port through a bastion server deployed in your VPC/Vnet that has access to your Confluent Cloud cluster.

Provision a bastion virtual machine (VM) in your VPC or VNet that is connected to Confluent Cloud with the SSH service.

Upon “SSH public key source”, select an existing SSH key pair. Paste in your existing public key onto the “SSH public key” field. If you need to locate it, you can find it via going into your terminal and typing in the below command. Otherwise, create a new SSH public key pair.
```
cat ~/.ssh/id_rsa.pub
```
You can set up with the default values for the rest of the properties.
Test a basic connectivity using the following command:
```
sudo ssh <your_cloud_provider_username>@<public_ip_of_vm> -i ~/.ssh/id_rsa
```
Your output should show a successful SSH into your VM.
Verify that the VMs have connectivity to your private endpoint using one of the following commands.

$BOOTSTRAP can be retrieved in the Cluster settings on your Confluent Cloud Console.
- Create an HTTP connection to the specified bootstrap URL ($BOOTSTRAP):
```
openssl s_client \
  -connect $BOOTSTRAP:9092 \
  -servername $BOOTSTRAP \
  -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | \
grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
```
- Verify the connectivity to the specified bootstrap URL ($BOOTSTRAP):
```
telnet $BOOTSTRAP 9092
```
Establish a reverse SSH tunnel from the client machine to the bastion VM for each Confluent Cloud components you are trying to connect from your local client.

There are 3 different endpoints to reach different components of your Confluent Cloud cluster:
- 443 is the Kafka REST API endpoint.
- 9092 is the broker endpoint to direct traffic to from your local host.
- ksqlDB endpoint.
The Kafka REST endpoint (<Kafka-REST-Endpoint>) can be found in the Cluster settings page. The ksqlDB endpoint (<ksqlDB-Endpoint>) is displayed in the Confluent Cloud Console under Cluster Settings > ksqlDB > Settings.
```
sudo ssh -L 127.0.0.2:443:<Kafka-REST-Endpoint>:443 <username>@<VM-instance-public-DNS> -i ~/.ssh/id_rsa

sudo ssh -L 127.0.0.3:443:<ksqlDB-Endpoint>:443 <username>@<VM-instance-public-DNS> -i ~/.ssh/id_rsa
```
Note that these connections must be maintained in order for the tunnel to work. However, in cases, such as a system restart, the tunnel and the DNS loopback aliases may not persist.
Configure DNS to resolve traffic to the Confluent Cloud component endpoints.

In the following example, /etc/hosts is updated to include the local IP addresses mapped to your Confluent Cloud cluster endpoints.
```
127.0.0.1 <Bootstrap-URL>
127.0.0.2 <Kafka-REST-Endpoint>
127.0.0.3 <ksqlDB-endpoint>
```

Configure your local DNS resolution¶

If connectivity cannot be established between the web browser and Confluent Cloud, DNS provider updates are needed to route web browser requests to the Kafka REST API endpoint and then to the private endpoint connected to Confluent Cloud.

Name Value

<Kafka-REST-Endpoint>

Example: lkc-k5w62-z6wj0p.westus2.azure.glb.confluent.cloud

Example: pksqlc-2jrvq-z6wj0p.westus2.azure.glb.confluent.cloud