.. _ansible-security: -------------------- Configuring Security -------------------- The topics in this section describes how to configure security for |ansible|. ============== TLS encryption ============== By default, the |cp| components are installed using the PLAINTEXT protocol with no data encryption. You can enable TLS encryption with one of the following options: * **Self Signed Certs:** A Certificate Authority will be generated by the playbooks and used to sign the certs for each host. * **Custom Certs:** You provide signed certs and keys for each host as well as the Certificate Authority Cert used to sign the certs. * **Custom Keystores and Truststores:** You provide keystores and truststores for each host. .. note:: By default the TLS connection is **one-way TLS**. Mutual TLS is also configurable. Configuring TLS for all components ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To enable TLS encryption for all components, add the following in the ``hosts.yml`` file. :: ssl_enabled: true TLS encryption variables should be added under the *all* group in the ``hosts.yml`` file. Configuring TLS for individual components ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To selectively enable or disable TLS encryption for specific components, set the following settings to ``true`` or ``false`` in addition to the global ``ssl_enabled`` setting. * ``kafka_connect_ssl_enabled`` * ``kafka_rest_ssl_enabled`` * ``schema_registry_ssl_enabled`` * ``control_center_ssl_enabled`` * ``ksql_ssl_enabled`` For example, if you want TLS enabled for all components except for Schema Registry, set: :: ssl_enabled: true schema_registry_ssl_enabled: false By default, components are configured with one-way TLS. To turn on TLS mutual authentication, set the following parameter to true: :: ssl_mutual_auth_enabled: true By default, the certs for this configuration are self-signed. To deploy custom certificates, there are two options available: **Custom Certificates** and **Custom Keystores and Truststores**. Custom certificates ^^^^^^^^^^^^^^^^^^^ To provide custom certificates for each host, you need the Certificate Authority certificate, the signed certificates, and keys for each host on the Ansible host. Complete the following steps to use custom certificates. #. Set ``ssl_custom_certs`` to ``true``. :: ssl_custom_certs: true #. Enter the path to the Certificate Authority Cert used to sign each host certificate. :: ssl_ca_cert_filepath: "/tmp/certs/ca.crt" #. Set the following variables for each host. :: ssl_signed_cert_filepath: "/tmp/certs/{{inventory_hostname}}-signed.crt" ssl_key_filepath: "/tmp/certs/{{inventory_hostname}}-key.pem" The variable ``{{inventory_hostname}}`` in the example shows that Ansible can read the hostnames set in the inventory file. For this reason, you can keep the inventory file shorter if you put the hostname in the filename for each signed certificate and key file. As an alternative, you can set the variables directly under a host: :: schema_registry: hosts: ip-192-24-10-207.us-west.compute.internal: ssl_signed_cert_filepath: "/tmp/certs/192-24-10-207-signed.crt ssl_key_filepath: "/tmp/certs/192-24-10-207-key.pem Custom Keystores and Truststores ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To provide Custom Keystores and Truststores for each host, you will need to have keystores and truststores for each host on the Ansible host, as well as their passwords. Complete the following steps to use Custom Keystores and Truststores. #. Set this variable to true. :: provided_keystore_and_truststore: true #. Set the following variables for each host. :: ssl_keystore_filepath: "/tmp/certs/{{inventory_hostname}}-keystore.jks" ssl_keystore_key_password: mystorepassword ssl_keystore_store_password: mystorepassword ssl_truststore_filepath: "/tmp/certs/truststore.jks" ssl_truststore_password: truststorepass Using the ``{{inventory_hostname}}`` variable and setting the same password for each host, you can set these variable once under the *all* group in the ``hosts.yml`` file. As an alternative, you can set these variables under each host as shown in the alternative Custom Certificates example. =================== SASL authentication =================== Confluent Ansible can configure SASL/PLAIN or SASL/GSSAPI (or Kerberos). By default, the |cp| components are installed without any SASL authentication. SASL authentication options ^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can enable the following SASL options: * **PLAIN:** SASL/PLAIN uses a simple username and password for authentication. * **SCRAM:** SASL/SCRAM uses usernames and password stored in |zk|. Credentials get created during installation. * **Kerberos:** SASL with Kerberos provides authentication using your Kerberos or Active Directory server. .. note:: Kerberos Key Distribution Center (KDC) and Active Directory KDC configurations are not currently configured by the Ansible playbook. Configuring SASL/PLAIN ^^^^^^^^^^^^^^^^^^^^^^ To configure SASL/PLAIN security, set the following option in the *all* group in the ``hosts.yml`` file. :: sasl_protocol: plain During installation, users are created for each component. This includes an admin user for the |ak| brokers and a client user for use by external components. To configure additional users, add the following section to the *all* group in the ``hosts.yml`` file. The following example shows an additional three users added. :: sasl_plain_users: user1: principal: user1 password: my-secret user2: principal: user2 password: my-secret user3: principal: user3 password: my-secret Configuring SASL/SCRAM ^^^^^^^^^^^^^^^^^^^^^^ To configure SASL/SCRAM security, set the following option in the *all* group in the ``hosts.yml`` file. :: sasl_protocol: scram During installation, users are created for each component. This includes an admin user for the |ak| brokers and a client user for use by external components. To configure additional users, add the following section to the *all* group in the ``hosts.yml`` file. :: sasl_scram_users: user1: principal: user1 password: my-secret Configuring SASL with Kerberos ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To configure SASL with Kerberos, you need to create principals within your organization's KDC server for each component and for each host in each component. You also need to generate keytabs for these principals. The keytab files must be present on the Ansible host. .. note:: You need to set up your own Key Distribution Center (KDC) independently of the playbook and provide your own keytabs. Add the following configuration parameters in in the ``all`` section of the ``hosts.yaml`` file. This installs Kerberos packages and configures the ``/etc/krb5.conf`` file on each host. * ``kerberos_kafka_broker_primary``: The *primary* part of the |ak| broker principal. For example, for the principal, ``kafka/kafka1.hostname.com@EXAMPLE.COM``, the primary is ``kafka``. * ``kerberos_configure``: Boolean for Ansible to install Kerberos packages and to configure the ``/etc/krb5.conf`` file. The default is ``true``. * ``realm``: The *realm* part of the |ak| broker Kerberos principal. * ``kdc_hostname``: Hostname of machine with KDC running. * ``admin_hostname``: Hostname of machine with KDC running. The following example shows the Kerberos configuration settings for the Kerberos principal, ``kafka/kafka1.hostname.com@EXAMPLE.COM``. :: all: vars: ...omitted #### Kerberos Configuration #### kerberos_kafka_broker_primary: kafka kerberos_configure: true kerberos: realm: example.com kdc_hostname: ip-192-24-45-82.us-west.compute.internal admin_hostname: ip-192-24-45-82.us-west.compute.internal .. note:: If the hosts already have the ``/etc/krb5.conf`` file configured, make sure to set ``kerberos_configure`` to false. The following examples show the variables that need to be added to the ``hosts.yml`` file for each component to use Kerberos for authentication. Each host must have these variables set in the ``hosts.yml`` file. **Zookeeper section** Use the following example to enter the ``zookeeper_kerberos_keytab_path`` and ``zookeeper_kerberos_principal`` variables. Each host needs to have these two variables. Three hosts are shown in the example. :: zookeeper: hosts: ip-192-24-34-224.us-west.compute.internal: zookeeper_kerberos_keytab_path: /tmp/keytabs/zookeeper-ip-192-24-34-224.us-west.compute.internal.keytab zookeeper_kerberos_principal: zookeeper/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM ip-192-24-37-15.us-west.compute.internal: zookeeper_kerberos_keytab_path: /tmp/keytabs/zookeeper-ip-192-24-34-224.us-west.compute.internal.keytab zookeeper_kerberos_principal: zookeeper/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM ip-192-24-34-224.us-west.compute.internal: zookeeper_kerberos_keytab_path: /tmp/keytabs/zookeeper-ip-192-24-34-224.us-west.compute.internal.keytab zookeeper_kerberos_principal: zookeeper/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **Kafka broker section** Use the following example to enter the ``kafka_broker_kerberos_keytab_path`` and ``kafka_broker_kerberos_principal`` variables. Each host needs to have these two variables. The example shows three hosts. :: kafka_broker: hosts: ip-192-24-34-224.us-west.compute.internal: kafka_broker_kerberos_keytab_path: /tmp/keytabs/kafka-ip-192-24-34-224.us-west.compute.internal.keytab kafka_broker_kerberos_principal: kafka/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM ip-192-24-37-15.us-west.compute.internal: kafka_broker_kerberos_keytab_path: /tmp/keytabs/kafka-ip-192-24-34-224.us-west.compute.internal.keytab kafka_broker_kerberos_principal: kafka/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM ip-192-24-34-224.us-west.compute.internal: kafka_broker_kerberos_keytab_path: /tmp/keytabs/kafka-ip-192-24-34-224.us-west.compute.internal.keytab kafka_broker_kerberos_principal: kafka/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **Schema Registry section** Use the following example to enter the ``schema_registry_kerberos_keytab_path`` and ``schema_registry_kerberos_principal`` variables. Each host needs to have these two variables. The example shows one host. :: schema_registry: hosts: ip-192-24-34-224.us-west.compute.internal: schema_registry_kerberos_keytab_path: /tmp/keytabs/schemaregistry-ip-192-24-34-224.us-west.compute.internal.keytab schema_registry_kerberos_principal: schemaregistry/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **Kafka Connect section** Use the following example to enter the ``kafka_connect_kerberos_keytab_path`` and ``kafka_connect_kerberos_principal`` variables. Each host needs to have these two variables. The example shows one host. :: kafka_connect: hosts: ip-192-24-34-224.us-west.compute.internal: kafka_connect_kerberos_keytab_path: /tmp/keytabs/connect-ip-192-24-34-224.us-west.compute.internal.keytab kafka_connect_kerberos_principal: connect/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **REST proxy section** Use the following example to enter the ``kafka_rest_kerberos_keytab_path`` and ``kafka_rest_kerberos_principal`` variables. Each host needs to have these two variables. The example shows one host. :: kafka_rest: hosts: ip-192-24-34-224.us-west.compute.internal: kafka_rest_kerberos_keytab_path: /tmp/keytabs/restproxy-ip-192-24-34-224.us-west.compute.internal.keytab kafka_rest_kerberos_principal: restproxy/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **KSQL section** Use the following example to enter the ``ksql_kerberos_keytab_path`` and ``ksql_kerberos_principal`` variables. Each host needs to have these two variables. The example shows one host. :: ksql: hosts: ip-192-24-34-224.us-west.compute.internal: ksql_kerberos_keytab_path: /tmp/keytabs/ksql-ip-192-24-34-224.us-west.compute.internal.keytab ksql_kerberos_principal: ksql/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM **Control Center section** Use the following example to enter the ``control_center_kerberos_keytab_path`` and ``control_center_kerberos_principal`` variables. Each host needs to have these two variables. The example shows one host. :: control_center: hosts: ip-192-24-34-224.us-west.compute.internal: control_center_kerberos_keytab_path: /tmp/keytabs/controlcenter-ip-192-24-34-224.us-west.compute.internal.keytab control_center_kerberos_principal: controlcenter/ip-192-24-34-224.us-west.compute.internal@REALM.EXAMPLE.COM ============================================ Security for internal and external listeners ============================================ |ansible| configures two listeners on the broker: an internal listener for the broker to communicate with other brokers and an external listener for the other |cp| components and external clients. By default both of these listeners inherit the security settings you configure for ``ssl_enabled`` and ``sasl_protocol``. If you only need a single listener, add the following variable to the *all* group in ``hosts.yml`` file. :: kafka_broker_configure_additional_brokers: false To configure different security settings for the internal and external listeners, add the following variables to the *all* group in the ``hosts.yml`` file. :: kafka_broker_custom_listeners: internal: name: INTERNAL port: 9091 ssl_enabled: false ssl_mutual_auth_enabled: false sasl_protocol: none external: name: EXTERNAL port: 9092 ssl_enabled: true ssl_mutual_auth_enabled: false sasl_protocol: scram You can create additional custom listeners within the ``kafka_broker_custom_listeners`` section shown in the example above. Simply add another listener block. The example below shows an additional custom client listener. :: client_listener: name: CLIENT port: 9093 ssl_enabled: true ssl_mutual_auth_enabled: true sasl_protocol: plain .. Hide the section until testing is done. ====================================== Configuring FIPS operational readiness ====================================== This feature is supported with Confluent Server, but not for standard |ak| deployments Use the ``fips_enabled`` setting to configure |cp| with :ref:`fips-operational-readiness`. All the listeners on the brokers must have the ``ssl_enabled`` set to ``true``, and you must be running |cs|. Set these settings in *all* group in the ``hosts.yml`` file. :: ssl_enabled: true confluent_server_enabled: true fips_enabled: true