Manage TLS Certificates for Confluent Platform Using Confluent for Kubernetes

This topic describes how to manage Transport Layer Security (TLS) certificates for Confluent Platform using Confluent for Kubernetes (CFK).

Requirements and considerations

Plan rotation before certificate expiry.
There should be an overlap window between new and old certificates during the renewal process.
Test the rotation process first in a lower-tier, non-production environment.

Rotate user-provided server certificates

In CFK, to rotate user-provided certificates that have been provided as Kubernetes secrets, you can update the contents of the Kubernetes secrets.

For example, to rotate the server certificates that you provided in .pem files:

Update the server.pem certificate and server-key.pem certificate private key.

Update the Kubernetes secret using the following command:

kubectl create secret generic component-tls \
--from-file=fullchain.pem=server.pem \
--from-file=cacerts.pem=ca.pem \
--from-file=privkey.pem=server-key.pem \
--save-config --dry-run=client -oyaml | \
kubectl apply -f -

CFK will watch for the Kubernetes secret changes, will notice that the content has changed, and will then perform a rolling restart of the component to configure the new server certificate.

Note that CFK only watches the secrets provided in the .pem format. This feature is not supported for the secrets provided in the .jks format.

Rotate certificate authority for user-provided certificates

You can rotate the certificate authority (CA) that has been provided as Kubernetes secrets using the steps described in this section.

The examples use the .pem files for providing certificates.

Step 1. Append new certificate authority

Append the new intermediate/root CA to the ca.pem file. It will co-exist with the old intermediate/root CA.

file name: ca.pem
content:
-----BEGIN CERTIFICATE-----
<old root CA certificate>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<new root CA certificate>
-----END CERTIFICATE-----

Update the secret:

kubectl create secret generic component-tls \
--from-file=fullchain.pem=server.pem \
--from-file=cacerts.pem=ca.pem \
--from-file=privkey.pem=server-key.pem \
--save-config --dry-run=client -oyaml | \
kubectl apply -f -

CFK will watch for the Kubernetes secret changes, notice that the content has changed, and then perform a rolling restart for the component to configure the new root CA for use alongside the old root CA.

If this CA is for Kafka and the dynamic loading is enabled, the cluster will not roll.

Step 2. Generate server certificates with new certificate authority

Generate the new server certificates with the new CA, and replace the server.pem certificate and the server-key.pem certificate private key.

Update the Kubernetes secret:

kubectl create secret generic component-tls \
--from-file=fullchain.pem=server.pem \
--from-file=cacerts.pem=ca.pem \
--from-file=privkey.pem=server-key.pem \
--save-config --dry-run=client -oyaml | \
kubectl apply -f -

CFK will watch for the Kubernetes secret changes, will notice that the content has changed, and will then rolling restart the component to configure the new server certificate.

If this certificate is for Kafka and the dynamic loading is enabled, the cluster will not roll.

Rotate certificate authority used for auto-generated certificates

In CFK, you can provide a certificate authority (CA) for Confluent Platform components to use for auto-generated certificates.

CA certificate rotation is a critical security operation that replaces an existing CA with a new one while maintaining service availability.

Follow the steps described in this section to renew the truststore and auto-generated certificates. This process maintains service availability throughout without downtime.

Wait for each step to complete before moving to the next step.

Important considerations

Do not apply both force-managed-truststore-renewal and force-managed-cert-renewal annotations simultaneously.
Always renew truststores for all components before renewing certificates.
Wait for each component’s rollout to complete before proceeding.
Plan rotation before certificate expiry.
Securely store and manage CA private keys.
Test the rotation process in a non-production environment first.

Step 1. Update CFK configuration

Create CA certificates.
Create your old (if it does not already exist) and new CA certificates.

Create Kubernetes or Vault secrets for the CA certificates.

For example:

# Create secret for new CA
kubectl -n <namespace> create secret tls ca-pair-sslcerts-new \
  --cert=new/ca.pem --key=new/ca-key.pem

Update your CFK configuration to include both old and new CA certificates.
caCertificate and oldCaCertificate support both secretRef (Kubernetes secret) and directoryPathInContainer (Vault directory path). When both are provided, directoryPathInContainer will be used.
For example:
```
managedCerts:
  enabled: true
  caCertificate:
    secretRef: ca-pair-sslcerts-new  # Your new CA secret
  oldCaCertificate:
    secretRef: ca-pair-sslcerts      # Your old CA secret
```

Apply the configuration.

helm upgrade --install -f values.yaml operator confluent-for-kubernetes \
  --namespace <namespace>

Step 2. Renew truststore

Apply the truststore renewal annotation to each Confluent Platform component.

kubectl annotate <Component CR type> <Component CR name> platform.confluent.io/force-managed-truststore-renewal="true"

Verify that all components truststores now have dual CAs, both old and new CA.

kubectl exec <pod-name> -- keytool -list \
  -keystore /mnt/sslcerts/truststore.jks \
  -storepass mystorepassword | grep -c "Certificate fingerprint"

You should see two certificates in each truststore.

Step 3. Renew certificates

Apply the certificate renewal annotation to each Confluent Platform component.

kubectl annotate <Component CR type> <Component CR name> platform.confluent.io/force-managed-cert-renewal="true"

Verify that all certificates are now signed by the new CA:

kubectl exec <pod-name> -- openssl x509 -in /mnt/sslcerts/tls.crt -text -noout | grep "Issuer:"

The issuer should show your new CA.

Manage auto-generated certificates

To manage auto-generated certificates in Confluent for Kubernetes (CFK), use the following annotations in the Confluent Platform component CRs.

Force a renewal of auto-generated certificates:
```
kubectl annotate <Component CR type> <Component CR name> \
  platform.confluent.io/force-managed-cert-renewal="true"
```
The annotation triggers a reconciliation on the Confluent Platform component. The process causes the certificates to be renewed, and the component to be rolled.
After the certificates are renewed, the annotation is reset to false.
Add additional custom SAN for auto-generated certificates. This will be added to SANs defined at the global level in managedCerts.sans in the CFK Helm values (in values.yaml).
```
kubectl annotate secret ca-pair-sslcerts \
  platform.confluent.io/managed-cert-add-sans="<Comma separated list of SANs>"
```
For example, the following annotation will add the SANs, kafka.operator, kafka.operator.local, to auto-generated certificates for Kafka:
```
kubectl annotate secret ca-pair-sslcerts \
  platform.confluent.io/managed-cert-add-sans="kafka.operator, kafka.operator.local"
```
After the annotation is set, use the platform.confluent.io/force-managed-cert-renewal="true" annotation to force a renewal of certificates. If not forced, SANs will be updated during the next reconciliation.
Specify the Kubernetes secret name for the CA certificates at the CR level. This will override the global config defined in managedCerts.caCertificate in the CFK Helm values (in values.yaml). This can be used even if managedCerts.enable is set to false.
```
kubectl annotate <Component CR type> <Component CR name> \
  platform.confluent.io/managed-cert-ca-pair-secret=<Secret name>
```
The secret should have tls.crt and tls.key with the CA certificate and key.

Configure the optional settings for auto-generated certificates at the CR level:

kubectl annotate <Component CR type> <Component CR name> \
  platform.confluent.io/managed-cert-duration-in-days=<Number of days>

kubectl annotate <Component CR type> <Component CR name> \
  platform.confluent.io/managed-cert-renew-before-in-days=<Number of days>

For example:

kubectl annotate kafka kafka platform.confluent.io/managed-cert-duration-in-days=60

kubectl annotate kafka kafka platform.confluent.io/managed-cert-renew-before-in-days=30

After the annotations are set, use the platform.confluent.io/force-managed-cert-renewal="true" to force a renewal of certificates.