Configure Mutual TLS (mTLS) Authentication on Confluent Cloud

Requirements

Confluent Cloud

Note

Temporary limitation: When you use mTLS authentication, custom connectors are not supported. Existing and new mTLS connections to the cluster fail if custom connectors are configured. To resume mTLS authentication after configuring a custom connector, delete all custom connectors on the cluster.

Kafka or Confluent Platform client

  • mTLS authentication is supported for Kafka and Confluent Platform clients that authenticate to Dedicated clusters in Confluent Cloud. Schema Registry, ksqlDB, Flink, and Kafka REST APIs are not supported.
  • Java or librdkafka Kafka clients. For supported versions, see Build Kafka Client Applications on Confluent Cloud.

Confluent CLI

  • Confluent CLI 4.4.0 or later.

Mutual TLS (mTLS)

  • A valid X.509 certificate chain PEM file, used to configure a trusted certificate authority in Confluent Cloud. The PEM file must meet the following requirements:

    • Minimum of one root or intermediate certificate.

    • Maximum of four certificates in the chain.

    • If using librdkafka clients, the certificate chain must include the signing certificate of the client certificate; otherwise, the TLS handshake fails during mTLS authentication.

      Note: Any authenticating librdkafka clients misconfigured with both SASL_SSL SASL_SSL security protocol and ssl.certificate.location / ssl.key.location configurations fail in an SSL handshake after a Certificate Authority is successfully configured in Confluent Cloud. Once a Certificate Authority is configured, all Dedicated clusters in the organization are enabled to use mTLS client authentication.

    • The file must be named using a .pem extension.

      Example: acme-internal-client-ca.pem

  • A PKCS12 or JKS keystore, used by the client application to authenticate to Confluent Cloud clusters. The keystore must contain the following:

    • A single client private key.

    • The client certificate.

    • The associated CA certificate chain that signs the client certificate.

    • The file must be named using a .p12 or .jks extension.

      PKCS12 example: acme.client.keystore.p12

      JKS example: acme.client.keystore.jks

  • A client configuration file containing the keystore settings for TLS authentication. This file is used by the client application to authenticate to your Kafka clusters.

    PKCS12 example for client.properties

    security.protocol=SSL
ssl.keystore.location=path/to/<client-keystore-filename>.p12
ssl.keystore.type=PKCS12
ssl.keystore.password=<keystore password>
ssl.key.password=<key password>

    JKS example for client.properties

    security.protocol=SSL
ssl.keystore.location=path/to/<client-keystore-filename>.jks
ssl.keystore.type=JKS
ssl.keystore.password=<keystore password>
ssl.key.password=<key password>

  • Only direct Certificate Revocation Lists (CRLs) are supported, meaning the CRL issuer must be the same as the configured Certificate Authority.

Steps to configure mTLS authentication on Confluent Cloud

Follow these steps to configure mTLS authentication on Confluent Cloud. You can use the Confluent Cloud Console, Confluent CLI, or the Confluent Cloud APIs to create a trusted certificate authority (CA) and an identity pool.

For the steps below, the examples are based on a hypothetical organization named Acme, Inc. and its internal client applications.

Step 1 - Create a trusted Certificate Authority

To create a trusted Certificate Authority (CA) using the Confluent Cloud Console, Confluent CLI, or the Confluent Cloud APIs, see Create a Certificate Authority.

Optionally, you can use certificate revocation lists (CRLs) to revoke certificates. For details, see Certificate revocation for mTLS authentication on Confluent Cloud.

Step 2 - Create an identity pool to map client certificates to granular cluster access

Create an identity pool to map a client certificate with a Confluent identity, allowing for granular permissions. This enables you to configure varying levels of access for different client certificates issued by your Certificate Authority to the data within your Confluent Dedicated clusters.

To create an identity pool for your Certificate Authority and assign role bindings for your identity pools, see Create an identity pool for mTLS.

Step 3 - Verify and use mTLS authentication

After you create a trusted client certificate authority and an identity pool with sufficient permissions, a valid client certificate can be used for authentication and authorization. Unlike OAuth/OIDC, you do not need to specify the certificate identity pool in the client configuration. The identity pool is automatically mapped based on the identity pool filters.

Important

A client certificate can map to multiple identity pools. If multiple pools have filters that evaluate true for the client certificate, then the client is given the union of permissions of all mapped pools. For details, see Using multiple identity pools.

To verify that the client can authenticate to the Dedicated Kafka cluster, you can use the client configuration file to list topics in the cluster. Here’s an example of how to use the configured mTLS authentication with a Kafka command-line tool.

Example

To run this example:

  • Replace <bootstrap URL> with the bootstrap URL for your Dedicated Kafka cluster.
  • The client.properties file is the client configuration file that you created in Step 3. It should contain the following settings:

PKCS12 example for client.properties

security.protocol=SSL
ssl.keystore.location=path/to/acme.client.keystore.p12
ssl.keystore.type=PKCS12
ssl.keystore.password=<keystore password>
ssl.key.password=<key password>

JKS example for client.properties

security.protocol=SSL
ssl.keystore.location=path/to/acme.client.keystore.jks
ssl.keystore.type=JKS
ssl.keystore.password=<keystore password>
ssl.key.password=<key password>

# list topics in the cluster
bin/kafka-topics.sh --bootstrap-server <bootstrap URL> \
  --command-config client.properties \
  --list

This should return a list of topics in the cluster. Note that you didn’t need to specify the certificate identity pools in the client configuration. Multiple identity pools can be selected based on the filters in the identity pools that evaluate to true for the client certificate. The correct identity pools are automatically selected based on the client certificate in the keystore.

Related content