Configure Encryption for Confluent Platform with Ansible Playbooks

Ansible Playbooks for Confluent Platform supports the PLAINTEXT (no encryption) and TLS encryption methods with PLAINTEXT being the default.

TLS encryption

Configure TLS for all components

To enable TLS encryption for all components, add the following in the hosts.yml file.

all:
  vars:
    ssl_enabled: true

Configure TLS for individual components

To selectively enable or disable TLS encryption for specific components, set the following settings to true or false in addition to the global ssl_enabled setting.

• zookeeper_ssl_enabled
• kafka_controller_ssl_enabled
• kafka_connect_ssl_enabled
• kafka_rest_ssl_enabled
• schema_registry_ssl_enabled
• control_center_ssl_enabled
• ksql_ssl_enabled

For example, if you want TLS enabled for all components except for Schema Registry, set:

all:
  vars:
    ssl_enabled: true
    schema_registry_ssl_enabled: false

By default, the certs for this configuration are self-signed. To deploy custom certificates, you can provide either custom certs or custom keystores and truststores.

Certificates

You can enable TLS encryption using one of the following.

• Self-signed certs: A Certificate Authority will be generated by the Ansible playbooks and used to sign the certs for each host.

  Use self-signed certificates only for test and development environments. Due to security concerns, self-signed certificates are not recommended for production environments.

• Custom certs: You provide signed certs and keys for each host as well as the Certificate Authority Certificate used to sign the certs. See Use custom certs for TLS.

• Custom certs existing on the hosts: You use the signed certs and keys already existing on the hosts. See Use custom certs already existing on hosts.

• Custom keystores and truststores: You provide keystores and truststores for each host on the control node. See Use custom keystores and truststores for TLS.

• Custom keystores and truststores existing on the hosts: You use the keystores and truststores already existing on the hosts. See Use custom keystores and truststores already existing on hosts.

To speed up the certificate and key store creations, you can install and run the random number generator service (rng-tools) on all hosts.

Use custom certs for TLS

To provide custom certs for each host, you need to have the the following on the Ansible control node:

• The Certificate Authority certificate
• The signed certificates
• The keys for each host

Complete the following steps to update hosts.yml.

1. Specify that custom certs are provided.

   all:
     vars:
       ssl_custom_certs: true
   

2. Enter the path to the Certificate Authority Certificate used to sign each host certificate. For example:

   all:
     vars:
       ssl_ca_cert_filepath: "/tmp/certs/ca.crt"
   

3. Set the signed certificate path and key file path for each host. For example:

   all:
     vars:
       ssl_signed_cert_filepath: "/tmp/certs/{{inventory_hostname}}-signed.crt"
       ssl_key_filepath: "/tmp/certs/{{inventory_hostname}}-key.pem"
   

   The {{inventory_hostname}} internal variable holds the hostnames set in the inventory file. You can use the variable to keep the inventory file shorter in the filename for each signed certificate and key file.

   As an alternative, you can set the variables directly under a host. For example:

   schema_registry:
      hosts:
         ip-192-24-10-207.us-west.compute.internal:
            ssl_signed_cert_filepath: "/tmp/certs/192-24-10-207-signed.crt
            ssl_key_filepath: "/tmp/certs/192-24-10-207-key.pem
   

Use custom certs already existing on hosts

You can set up TLS in Ansible Playbooks for Confluent Platform with the custom certs that are already present on the Confluent Platform components hosts.

You need the Certificate Authority certificate, the signed certificates, and keys existing on each host.

Complete the following steps to update hosts.yml.

1. Specify that you are using the custom certs existing on the hosts.

   all:
     vars:
       ssl_custom_certs: true
       ssl_custom_certs_remote_src: true
   

2. Enter the path to the Certificate Authority Certificate used to sign each host certificate.

   ssl_ca_cert_filepath:
   

3. Set the signed certificate path and key file path for each host.

   ssl_signed_cert_filepath:
   ssl_key_filepath:
   

   For example:

   schema_registry:
      hosts:
         ip-192-24-10-207.us-west.compute.internal:
            ssl_ca_cert_filepath: "/tmp/certs/ca.crt"
            ssl_signed_cert_filepath: "/tmp/certs/192-24-10-207-signed.crt
            ssl_key_filepath: "/tmp/certs/192-24-10-207-key.pem
   

Use custom keystores and truststores for TLS

To provide custom keystores and truststores for each host, you need to have keystores and truststores (and their passwords) for each host on the Ansible control node and their passwords.

Complete the following steps to update hosts.yml.

1. Specify that custom keystores and truststores are provided.

   all:
     vars:
       ssl_provided_keystore_and_truststore: true
   

2. Provide the keystore and truststore filepaths and passwords. For example:

   all:
     vars:
       ssl_keystore_filepath: "/tmp/certs/{{inventory_hostname}}-keystore.jks"
       ssl_keystore_key_password: mystorepassword
       ssl_keystore_store_password: mystorepassword
       ssl_truststore_filepath: "/tmp/certs/truststore.jks"
       ssl_truststore_password: truststorepass
   

   The {{inventory_hostname}} internal variable holds the hostnames set in the inventory file. Using the {{inventory_hostname}} variable and setting the same password for each host, you can set these variable only once in the hosts.yml file.

   As an alternative, you can set these variables under each host. For example:

   schema_registry:
     hosts:
       ip-192-24-10-207.us-west.compute.internal:
         ssl_keystore_filepath: "/tmp/certs/{{inventory_hostname}}-keystore.jks"
         ssl_keystore_key_password: mystorepassword
         ssl_keystore_store_password: mystorepassword
         ssl_truststore_filepath: "/tmp/certs/truststore.jks"
         ssl_truststore_password: truststorepass
   

Use custom keystores and truststores already existing on hosts

You can set up TLS in Ansible Playbooks for Confluent Platform with the keystores and truststores that are already present on the Confluent Platform components hosts.

In this scenario, you fully own the keystore and truststore and take responsibility to ensure the correct certificates are in place and to set and manage the JKS password. If you are using the certificates for mTLS authentication, ensure that the correct SANs are in the certificates and that the principal name is correctly set.

To use existing keystores and truststores:

1. Create the keystore and truststore JKS files, a separate one per component. You put these files on the component hosts and provide the file path for each file per component in the inventory as shown in the third step below.

   Keystores and truststores must be configured as follows:

   • Keystores and truststores should ideally be in the PKCS12 format. If it’s in the JKS format, it’ll be internally copied and converted to a temporary pem file to enable the extraction of key/certs.
   • Keystore and truststore must have the read permission (permissions >= 0640).
   • Keystore and truststore must be owned by the relevant user and group.

2. Set the following variables in the inventory.

   In addition to the variables mentioned in Use custom keystores and truststores for TLS, you need to set ssl_provided_keystore_and_truststore_remote_src: true.

   all:
     vars:
       ssl_enabled: true
       ssl_mutual_auth_enabled: true
       ssl_provided_keystore_and_truststore: true
       ssl_provided_keystore_and_truststore_remote_src: true
   

3. Set the following variables for each host separately in the inventory:

   ssl_keystore_filepath:
   ssl_keystore_key_password:
   ssl_keystore_store_password:
   ssl_keystore_alias:
   ssl_truststore_filepath:
   ssl_truststore_password:
   ssl_truststore_ca_cert_alias:
   

   For example:

   schema_registry:
     hosts:
       ip-192-24-10-207.us-west.compute.internal:
         ssl_keystore_filepath: "/tmp/certs/{{inventory_hostname}}-keystore.jks"
         ssl_keystore_key_password: mystorepassword
         ssl_keystore_store_password: mystorepassword
         ssl_keystore_alias: <alias for host specific certificate, only required if multiple certs are in the provided keystore>
         ssl_truststore_filepath: "/tmp/certs/truststore.jks"
         ssl_truststore_password: truststorepass
         ssl_truststore_ca_cert_alias: <alias for the CA certificate in the truststore>
   

   The internal variable {{inventory_hostname}} holds the hostnames set in the inventory file.

4. (For RBAC only) To use existing keystores and truststores for the mTLS authentication with RBAC, the following are the additional steps you need to perform.

   1. Generate pem files in addition to keystores/truststores and copy them to the hosts.

      PEM files must be 2048 bits in size.

   2. Set the following variables in the inventory under each host:

      create_mds_certs: false
      token_services_public_pem_file: # Path to public.pem on the host node
      token_services_private_pem_file: # Path to tokenKeypair.pem
      

Enable FIPS in Confluent Platform

The Federal Information Processing Standard (FIPS) is a U.S. government computer security standard that is used to validate and certify software cryptographic modules.

You can use Ansible Playbooks for Confluent Platform (Confluent Ansible) to enable FIPS on Confluent Platform.

Confluent Ansible uses the in-built FIPS compliant OpenSSL with Red Hat.

Note

If you need to have the latest OpenSSL 3.0 on RHEL 8, you need to install FIPS-compliant OpenSSL 3. Confluent Ansible would still work on such setups.

The following are the requirements when enabling FIPS using Confluent Ansible:

• FIPS-enabled RHEL 7, RHEL 8, or RHEL 9 system

  For enabling FIPS on RHEL, see the Red Hat documentation.

• Java

  • Java 11 or 17 for RHEL 8 and RHEL 9
  • Java 8 or 11 for RHEL 7

• Random number generator service running

• TLS enabled in Confluent Platform

Upgrading between a non-FIPS Confluent Platform cluster and a FIPS-enabled Confluent Platform cluster is not supported.

For more information about enabling and using FIPS in Confluent Platform, see FIPS compliance in Confluent Platform.

To configure Confluent Ansible to run Confluent Platform in FIPS mode:

1. Enable FIPS mode on RHEL 7, RHEL 8, or RHEL 9.

2. If using an RBAC-enabled cluster, ensure that LDAP is FIPS compliant.

3. In your inventory file, add the flag to enable FIPS:

   all:
      vars:
        fips_enabled: true
   

4. In your inventory file, enable SSL for all listeners:

   all:
      vars:
        ssl_enabled: true
   

5. If using custom SSL certificates (ssl_custom_certs: true), ssl_signed_cert_filepath must be the path to the certificate chain. Using a single signed certificate alone will not work.

6. Install Confluent Platform:

   ansible-playbook -i <inventory.yml> confluent.platform.all
   

Next step

• Configure Authentication for Confluent Platform with Ansible Playbooks.
• Configure Authorization for Confluent Platform with Ansible Playbooks.
• Configure Advanced Confluent Platform Settings with Ansible Playbooks.
• Install Confluent Platform with Ansible Playbooks.