Configure Network Encryption with Confluent for Kubernetes

This document describes how to configure network encryption with Confluent for Kubernetes. For security concepts in Confluent Platform, see Security.

To protect network communications of Confluent components,|co| supports Transport Layer Security (TLS), an industry-standard encryption protocol.

TLS relies on keys and certificates to establish trusted connections. This section describes how to manage keys and certificates in preparation to configure TLS encryption for Confluent Platform.

CFK supports the following mechanisms to enable TLS encryption:

  • Auto-generated certificates: CFK auto-generates the server certificates, using a given certificate authority
  • User provided certificates: User provides the private key, public key and certificate authority

For scenarios where you don’t need to use your own server certificates, we recommend you use the auto-generated certificate capability.

Configure auto-generated TLS certificates

Provide a root certificate authority as a Kubernetes Secret named ca-pair-sslcerts. Provide the certificate authority public and private key in the following format:

kubectl create secret tls ca-pair-sslcerts \
  --cert=/path/to/ca.pem \
  --key=/path/to/ca-key.pem

Configure each component to use auto-generated certificates.

spec:
  tls:
    autoGeneratedCerts: true

CFK will create the required server certificates and store them as Kubernetes secrets, for Confluent components to use:

kubectl get secrets
NAME                             TYPE
...
zookeeper-generated-jks          kubernetes.io/tls
kafka-generated-jks              kubernetes.io/tls
...

The generated server certificates expires in 365 days.

For a tutorial scenario on using auto-generated certs, see the quickstart tutorial.

Configure user-provided TLS certificates

When you provide the TLS certificates, CFK takes the provided files and configures Confluent components accordingly.

The following TLS certificate information needs to be provided:

  • Certificate authorities to trust
  • Server certificate private key
  • Server certificate public key

It’s important to ensure that the server certificate Subject Alternative Name (SAN) list is properly defined. The list needs to cover all hostnames that the server component will be accessed on.

With Kubernetes, there is an internal network (.<namespace>.svc.cluster.local) that needs to be considered, if TLS for internal communication network encryption is enabled, which is recommended. If enabling external network communication, the external domain name needs to be included in the SAN list.

For an example of how to create certificates with approproate SAN configurations, see the Create your own certificates tutorial.

This TLS certificate information can be provided by the user in one of three ways:

  1. As .pem files
  2. As .tls and .crt files
  3. As .jks files

1. Provide as .pem files

Provide the following files:

  • ca.pem: This contains the list of certificate authorities to trust.
  • server.pem: This contains the server certificate.
  • server-key.pem: This contains the server certificate private key.

Create a Kubernetes secret with the following keys:

kubectl create secret generic kafka-tls \
--from-file=fullchain.pem=server.pem \
--from-file=cacerts.pem=ca.pem \
--from-file=privkey.pem=server-key.pem

Configure in the component CR:

spec:
  tls:
    secretRef: kafka-tls

2. Provide as .tls and .crt files

Provide the following files:

  • tls.crt: This contains the server certificate.
  • tls.key: This contains the server certificate private key.
  • ca.crt: This contains the list of certificate authorities to trust.

Note: .tls and .crt secret files are what Cert Manager, a popular open source tool to manage certificates, creates.

Create a Kubernetes secret with the following keys:

kubectl create secret generic kafka-tls \
--from-file=tls.crt=tls.crt \
--from-file=ca.crt=ca.crt \
--from-file=tls.key=tls.key

Configure in the component CR:

spec:
  tls:
    secretRef: kafka-tls

3. Provide as .jks files

Provide the following files:

  • keystore.jks: PKCS12 format keystore, containing component server key.

  • truststore.jks: PKCS12 format truststore, containing the certificates to trust.

  • jksPassword.txt: Password for the JKS.

    Create the jksPassword.txt file with jksPassword=<password_for_jks>:

    echo -n "jksPassword=<password_for_jks>" > jksPassword.txt
    

Create a Kubernetes secret with the following keys:

kubectl create secret generic kafka-tls \
--from-file=keystore.jks=keystore.jks \
--from-file=truststore.jks=truststore.jks \
--from-file=jksPassword.txt=jksPassword.txt

Configure in the component CR:

spec:
  tls:
    secretRef: kafka-tls