Test Connectivity to Confluent Cloud¶
Kafka broker hosts in Confluent Cloud do not respond to ping
commands. Use
alternative ways presented in this guide to test connectivity to the Kafka cluster
and the Kafka REST endpoint before whitelisting those endpoints.
Note that you need to whitelist both endpoints (ports 9092
and 443
) on
your firewall for the Confluent CLI to work.
Run through the following steps to validate Confluent Cloud connectivity is working correctly.
- Test TCP connectivity to the Confluent Cloud cluster bootstrap endpoint:
- If TCP connectivity can be successfully established, test data plane operations by producing/consuming using:
Test TCP connectivity to Confluent Cloud¶
For public networking, VPC peering, VNet peering, and AWS Transit Gateway, test TCP connectivity to the Confluent Cloud cluster using one of the tools, openSSL, Netcat, or Telnet.
For public endpoint clusters, run the command from any computer that has internet access.
For the cluster in private network environments, such as VPC peering, VNet peering, and AWS Transit Gateway, run the tests from within your VPC or VNet that is connected to the Confluent Cloud cluster.
Note that the host addresses of the Kafka bootstrap server and the REST
endpoint are the same, and only the port numbers differ. Use port 9092
to
test the connection to the Kafka bootstrap server, and port 443
to test
the connection to the Kafka REST endpoint.
If successful, you will see a connection success message, and if there is a problem reaching the cluster, the commands will time out with an error.
openSSL
openssl s_client -connect <bootstrap-url>:9092 openssl s_client -connect <bootstrap-url>:443
For details, see the openSSL documentation for the -connect option.
Netcat
nc -zv <bootstrap-url> 9092 nc -zv <bootstrap-url> 443
Telnet
telnet <bootstrap-url> 9092 telnet <bootstrap-url> 443
Note
You might need to update the network security tools and firewalls to allow connectivity. If you have issues connecting after following these steps, confirm which network security systems your organization uses and whether their configurations need to be changed.
Test private link TCP connectivity to Confluent Cloud¶
From an instance within your VPC or VNet, or anywhere the DNS is set up, run through the following steps to validate Kafka connectivity through PrivateLink, Private Link, or Private Service Connect is working correctly.
Set an environment variable with the cluster bootstrap URL.
export BOOTSTRAP=$<bootstrap-server-url>
The Bootstrap URL displayed in Confluent Cloud Console includes the port (
9092
). TheBOOTSTRAP
value should include the full hostname, but do not include the port. This is so that you can run theopenssl s_client -connect <host>:<port>
command with the required values.For example:
# Public DNS resolution for AWS export BOOTSTRAP=lkc-2v531-lg1y3.us-west-1.aws.glb.confluent.cloud # Public DNS resolution for Azure export BOOTSTRAP=lkc-222v1o-4kgzg.centralus.azure.glb.confluent.cloud # Public DNS resolution for GCP export BOOTSTRAP=lkc-xxxxxx-xxxxxx.asia-southeast1.gcp.glb.confluent.cloud # Private DNS resolution export BOOTSTRAP=lkc-2v531.domz6wj0p.us-west-1.aws.confluent.cloud
Test connectivity to your cluster by running the
openssl s_client -connect <host>:<port>
command, specifying the$BOOTSTRAP
environment variable for the<host>
value.To run the
openssl s_client -connect
command, the-connect
option requires that you specify the host and the port number. For details, see the openssl s_client documentation.Test the connection to the Kafka bootstrap server using the port and
9092
:openssl s_client -connect $BOOTSTRAP:9092 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
If the return output is
-----BEGIN CERTIFICATE----- Verify return code: 0 (ok)
, connectivity to the bootstrap is confirmed.Note
You might need to update the network security tools and firewalls to allow connectivity. If you have issues connecting after following these steps, confirm which network security systems your organization uses and whether their configurations need to be changed.If you still have issues, run the debug connectivity script (
debug-connectivity.sh
) for your provider in Confluent Cloud Networking repo and provide the output to Confluent Support for assistance with your Private Service Connect setup.Test the connection to the Kafka REST endpoint using the port
443
:openssl s_client -connect $BOOTSTRAP:443 -servername $BOOTSTRAP -verify_hostname $BOOTSTRAP </dev/null 2>/dev/null | grep -E 'Verify return code|BEGIN CERTIFICATE' | xargs
If the return output is
-----BEGIN CERTIFICATE----- Verify return code: 0 (ok)
, connectivity to the bootstrap is confirmed.
Troubleshoot TCP connectivity¶
If TCP connectivity to the bootstrap endpoint cannot be established, check the following:
Your security group has inbound/outbound rules for ports 443 and 9092 from your desired source (your VPC CIDR). The Protocol should be TCP for all the rules.
You created a Private DNS zone with the correct Confluent Cloud DNS Domain Name. The Domain Name can be found in the Confluent Cloud Console, under the Network page.
Check your firewalls and other security configurations and restrictions that could prevent the connection to the Confluent Cloud cluster bootstrap endpoint.
If your VPC is not configured to use the default cloud provider DNS resolver, the instance won’t able to resolve the private DNS zone. As a test, run
dig
with the resolver to use. For example:dig @10.16.16.2 hostname lkc-6gpq2-4r6p4.eu-west-1.aws.glb.confluent.cloud
You should configure the VPC to resolve the standard DNS server.
Test connectivity to Confluent Cloud using Confluent CLI¶
After TCP connectivity is successfully established, test data plane operations by producing/consuming using the Confluent CLI.
If using private networking, run the steps from an instance within the VPC or VNet to validate Kafka connectivity works correctly.
Sign in to Confluent CLI with your Confluent Cloud credentials.
confluent login
List the clusters in your organization.
confluent kafka cluster list
Select the cluster with a PrivateLink, Private Link, Private Service Connect you wish to test.
confluent kafka cluster use ...
For example:
confluent kafka cluster use lkc-a1b2c
Create a cluster API key to authenticate with the cluster.
confluent api-key create --resource ... --description ...
For example:
confluent api-key create --resource lkc-a1b2c --description "connectivity test"
Select the API key you just created.
confluent api-key use ... --resource ...
For example:
confluent api-key use WQDMCIQWLJDGYR5Q --resource lkc-a1b2c
Create a test topic.
confluent kafka topic create test
Start consuming events from the test topic.
confluent kafka topic consume test
Open another terminal tab or window.
Start a producer.
confluent kafka topic produce test
Type anything into the produce tab and hit
Enter
; pressCtrl+D
orCtrl+C
to stop the producer.The tab running consume will print what was typed in the tab running produce.
Troubleshoot connectivity to Kafka brokers¶
Failed connection to Kafka brokers in private link cluster¶
Issue: You get an error message similar to the following from a data plane operation over a private link, such as produce or consume.
|FAIL|Confluent-CLI_v3.8.0#producer-1|
[thrd:sasl_ssl://e-000d.europe-west2-b.dom8w9ed2pn.europe-west2.gcp.c]:
sasl_ssl://e-000d.europe-west2-b.dom8w9ed2pn.europe-west2.gcp.confluent.cloud:9092/2:
SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker
listener? (after 0ms in state SSL_HANDSHAKE)
Possible causes:
- The private endpoint in use does not correspond to the correct availability zone.
- Zonal affinity between your VPC and Confluent Cloud is not respected. See Provision PrivateLink endpoints in AWS.
- The private DNS zone is not configured correctly. See Create a DNS zone and DNS records.
If you need to review your private link connectivity configurations, see:
Test connectivity to Confluent Cloud using other tools¶
After TCP connectivity is successfully established, you can use the other
clients and tools to test producing/consuming messages when Confluent CLI is
not a viable option. Examples are kafka-topics
, kafka-console-consumer
,
native command line tools, or Java and other clients. The following are a few of
the test workflows you can use as references:
- For using Java to test connectivity, see Simple Java client examples or Getting Started with Apache Kafka and Java tutorial.
- For using the console consumer tools to test connectivity, see How to produce and consume messages using the console tools.