Test Connectivity to Confluent Cloud

Kafka broker hosts in Confluent Cloud do not respond to ping commands. Use alternative ways presented in this guide to test connectivity to the Kafka cluster and the Kafka REST endpoint before whitelisting those endpoints.

Note that you need to whitelist both endpoints (ports 9092 and 443) on your firewall for the Confluent CLI to work.

Run through the following steps to validate Confluent Cloud connectivity is working correctly.

1. Test connectivity to the Confluent Cloud cluster bootstrap endpoint:
   • For public endpoints, VPC peering, VNet peering, and AWS Transit Gateway
   • For AWS PrivateLink, Azure Private Link, and Google Cloud Private Service Connect
2. If connectivity can be successfully established, test data plane operations by producing/consuming using:
   • Confluent CLI
   • Other tools

Test connectivity to Confluent Cloud

For public networking, VPC peering, VNet peering, and AWS Transit Gateway, test connectivity to the Confluent Cloud cluster using one of the tools, openSSL, Netcat, or Telnet.

For public endpoint clusters, run the command from any computer that has internet access.

For the cluster in private network environments, such as VPC peering, VNet peering, and AWS Transit Gateway, run the tests from within your VPC or VNet that is connected to the Confluent Cloud cluster.

Note that the host addresses of the Kafka bootstrap server and the REST endpoint are the same, and only the port numbers differ:

• Use port 9092 to test the connection to the Kafka bootstrap server.
• Use port 443 to test the connection to the Kafka REST endpoint.

To only test TCP connectivity, use Telnet and Netcat (or its successor Socat):

• Netcat

  nc -zv <bootstrap-url> 9092
  
  nc -zv <bootstrap-url> 443
  

• Telnet

  telnet <bootstrap-url> 9092
  
  telnet <bootstrap-url> 443
  

In addition to TCP connectivity, to also test TLS handshake and the certificate, use openSSL. With openSSL, you can send an SNI header:

• openSSL

  openssl s_client -servername <bootstrap-url> -connect <bootstrap-url>:9092
  
  openssl s_client -servername <bootstrap-url> -connect <bootstrap-url>:443
  

  For details, see the openSSL documentation for the -connect option.

It is recommended that you use openSSL to test TCP and TLS because with the TCP testing only, it is difficult to make the distinction among the various causes when a connection fails, such as:

• Timeout because of routing problems
• Established connection, but you as the client not initiating the TLS handshake
• Envoy disconnecting your connection because you do not send the SNI header

Troubleshoot connectivity issues

If connectivity to the bootstrap endpoint cannot be established, check your firewalls and other security configurations and restrictions that could prevent the connection to the Confluent Cloud cluster bootstrap endpoint.

Test private link connectivity to Confluent Cloud

From an instance within your VPC or VNet, or anywhere the DNS is set up, run through the following steps to validate Kafka connectivity through PrivateLink, Private Link, or Private Service Connect is working correctly.

1. Set an environment variable with the cluster bootstrap URL.

   export BOOTSTRAP=$<bootstrap-server-url>
   

   The Bootstrap URL displayed in Confluent Cloud Console includes the port (9092). The BOOTSTRAP value should include the full hostname, but do not include the port. This is so that you can run the openssl s_client -connect <host>:<port> command with the required values.

   For example:

   # Public DNS resolution for AWS
   export BOOTSTRAP=lkc-2v531-lg1y3.us-west-1.aws.glb.confluent.cloud
   
   # Public DNS resolution for Azure
   export BOOTSTRAP=lkc-222v1o-4kgzg.centralus.azure.glb.confluent.cloud
   
   # Public DNS resolution for GCP
   export BOOTSTRAP=lkc-xxxxxx-xxxxxx.asia-southeast1.gcp.glb.confluent.cloud
   
   # Private DNS resolution
   export BOOTSTRAP=lkc-2v531.domz6wj0p.us-west-1.aws.confluent.cloud
   

2. Test connectivity to your cluster by running the openssl s_client -connect <host>:<port> command, specifying the $BOOTSTRAP environment variable for the <host> value.

   To run the openssl s_client -connect command, the -connect option requires that you specify the host and the port number. For details, see the openssl s_client documentation.

   Test the connection to the Kafka bootstrap server using the port and 9092:

   openssl s_client -connect $BOOTSTRAP:9092 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
   

3. If the return output is -----BEGIN CERTIFICATE----- Verify return code: 0 (ok), connectivity to the bootstrap is confirmed.

4. Test the connection to the Kafka REST endpoint using the port 443:

   openssl s_client -connect $BOOTSTRAP:443 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
   

5. If the return output is -----BEGIN CERTIFICATE----- Verify return code: 0 (ok), connectivity to the bootstrap is confirmed.

Troubleshoot private link connectivity issues

If connectivity to the bootstrap endpoint cannot be established, check the following:

• Your security group has inbound/outbound rules for ports 443 and 9092 from your desired source (your VPC CIDR). The Protocol should be TCP for all the rules.

• You created a Private DNS zone with the correct Confluent Cloud DNS Domain Name. The Domain Name can be found in the Confluent Cloud Console, under the Network page.

• Check your firewalls and other security configurations and restrictions that could prevent the connection to the Confluent Cloud cluster bootstrap endpoint.

• If your VPC is not configured to use the standard DNS resolver of the cloud provider, the instance won’t be able to resolve the private DNS zone.

  As a test, run dig with the resolver to use. For example:

  dig @<DNS resolver> <hostname>
  

  If resolution fails, re-configure the VPC to resolve using the standard DNS server.

  The cloud provider’s standard resolvers are:

  • For AWS: 169.254.169.253
  • For Azure: 168.63.129.16
  • For Google Cloud: 169.254.169.254

If you still have issues with private link connectivity after addressing the above items, run the debug connectivity script (debug-connectivity.sh) for your provider in Confluent Cloud Networking repo and provide the output to Confluent Support for assistance with your private link connectivity setup.

Test connectivity to Kafka using Confluent CLI

After connectivity is successfully established, test data plane operations by producing/consuming using the Confluent CLI.

If using private networking, run the steps from an instance within the VPC or VNet to validate Kafka connectivity works correctly.

1. Sign in to Confluent CLI with your Confluent Cloud credentials.

   confluent login
   

2. List the clusters in your organization.

   confluent kafka cluster list
   

3. Select the cluster with a PrivateLink, Private Link, Private Service Connect you wish to test.

   confluent kafka cluster use ...
   

   For example:

   confluent kafka cluster use lkc-a1b2c
   

4. Create a cluster API key to authenticate with the cluster.

   confluent api-key create --resource ... --description ...
   

   For example:

   confluent api-key create --resource lkc-a1b2c --description "connectivity test"
   

5. Select the API key you just created.

   confluent api-key use ... --resource ...
   

   For example:

   confluent api-key use WQDMCIQWLJDGYR5Q --resource lkc-a1b2c
   

6. Create a test topic.

   confluent kafka topic create test
   

7. Start consuming events from the test topic.

   confluent kafka topic consume test
   

8. Open another terminal tab or window.

9. Start a producer.

   confluent kafka topic produce test
   

10. Type anything into the produce tab and hit Enter; press Ctrl+D or Ctrl+C to stop the producer.

11. The tab running consume will print what was typed in the tab running produce.

Troubleshoot connectivity to Kafka brokers

Failed connection to Kafka brokers in private link cluster

Issue: You get an error message similar to the following from a data plane operation over a private link, such as produce or consume.

|FAIL|Confluent-CLI_v3.8.0#producer-1|
[thrd:sasl_ssl://e-000d.europe-west2-b.dom8w9ed2pn.europe-west2.gcp.c]:
sasl_ssl://e-000d.europe-west2-b.dom8w9ed2pn.europe-west2.gcp.confluent.cloud:9092/2:
SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker
listener? (after 0ms in state SSL_HANDSHAKE)

Possible causes:

• The private endpoint in use does not correspond to the correct availability zone.
• Zonal affinity between your VPC and Confluent Cloud is not respected. See Provision PrivateLink endpoints in AWS.
• The private DNS zone is not configured correctly. See Create a DNS zone and DNS records.

If you need to review your private link connectivity configurations, see:

• Configure connectivity to Confluent Cloud using AWS PrivateLinks.
• Configure connectivity to Confluent Cloud using Azure Private Links.
• Configure connectivity to Confluent Cloud using Google Cloud Private Service Connects.

Test connectivity to Kafka using other tools

After connectivity is successfully established, you can use the other clients and tools to test producing/consuming messages when Confluent CLI is not a viable option. Examples are kafka-topics, kafka-console-consumer, native command line tools, or Java and other clients. The following are a few of the test workflows you can use as references:

• For using Java to test connectivity, see Simple Java client examples or Getting Started with Apache Kafka and Java tutorial.
• For using the console consumer tools to test connectivity, see How to produce and consume messages using the console tools.