.. _kafka_adding_security: Adding Security to a Running Cluster ------------------------------------ Migrating Brokers and Clients ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can secure a running cluster via one or more of the supported protocols discussed previously. This is done in phases: (1) Incrementally restart the cluster nodes to open additional secured port(s). (2) Restart clients using the secured rather than ``PLAINTEXT`` port (assuming you are securing the client-broker connection). (3) Incrementally restart the cluster again to enable broker-to-broker security (if this is required) (4) A final incremental restart to close the ``PLAINTEXT`` port. The specific steps for configuring security protocols are described in the respective sections for :ref:`SSL ` and :ref:`SASL `. Follow these steps to enable security for your desired protocol(s). The security implementation lets you configure different protocols for both broker-client and broker-broker communication. These must be enabled in separate restarts. A ``PLAINTEXT`` port must be left open throughout so brokers and/or clients can continue to communicate. When performing an incremental restart, take into consideration the recommendations for doing :ref:`rolling restarts ` to avoid downtime for end users. As an example, if you want to encrypt both broker-client and broker-broker communication with SSL. In the first incremental restart, open a SSL port on each node: .. codewithvars:: bash listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092 Then restart the clients, changing their config to point at the newly opened, secured port: .. codewithvars:: bash bootstrap.servers=[broker1:9092,...] security.protocol=SSL ...etc In the second incremental server restart, instruct |ak-tm| to use SSL as the broker-broker protocol (which will use the same SSL port): .. codewithvars:: bash listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092 security.inter.broker.protocol=SSL In the final restart, secure the cluster by closing the ``PLAINTEXT`` port: .. codewithvars:: bash listeners=SSL://broker1:9092 security.inter.broker.protocol=SSL Alternatively, you might choose to open multiple ports so that different protocols can be used for broker-broker and broker-client communication. If you want to use SSL encryption throughout (i.e. for broker-broker and broker-client communication) but want to add SASL authentication to the broker-client connection also. You would achieve this by opening two additional ports during the first restart: .. codewithvars:: bash listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093 Again you restart the clients, changing their config to point at the newly opened, SASL & SSL secured port: .. codewithvars:: bash bootstrap.servers=[broker1:9093,...] security.protocol=SASL_SSL ...etc The second server restart would switch the cluster to use encrypted broker-broker communication via the SSL port you previously opened on port 9092: .. codewithvars:: bash listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093 security.inter.broker.protocol=SSL The final restart secures the cluster by closing the ``PLAINTEXT`` port. .. codewithvars:: bash listeners=SSL://broker1:9092,SASL_SSL://broker1:9093 security.inter.broker.protocol=SSL .. _zookeeper_adding_security: Migrating |zk| ~~~~~~~~~~~~~~~~~~~ If you are running a version of |ak| that does not support security or simply with security disabled, and you want to make the cluster secure, then you must execute the following steps to enable |zk| authentication with minimal disruption to your operations: #. Perform a rolling restart setting the JAAS login file, which enables brokers to authenticate. At the end of the rolling restart, brokers are able to manipulate znodes with strict ACLs, but they will not create znodes with those ACLs #. Perform a second rolling restart of brokers, this time setting the configuration parameter ``zookeeper.set.acl`` to true, which enables the use of secure ACLs when creating znodes #. Execute the ZkSecurityMigrator tool. To execute the tool, there is this script: ``bin/zookeeper-security-migration`` with ``zookeeper.acl`` set to secure. This tool traverses the corresponding sub-trees changing the ACLs of the znodes It is also possible to turn off authentication in a secure cluster. To do it, follow these steps: #. Perform a rolling restart of brokers setting the JAAS login file, which enables brokers to authenticate, but setting ``zookeeper.set.acl`` to false. At the end of the rolling restart, brokers stop creating znodes with secure ACLs, but are still able to authenticate and manipulate all znodes #. Execute the ZkSecurityMigrator tool. To execute the tool, run this script ``bin/zookeeper-security-migration`` with ``zookeeper.acl`` set to unsecure. This tool traverses the corresponding sub-trees changing the ACLs of the znodes #. Perform a second rolling restart of brokers, this time omitting the system property that sets the JAAS login file Here is an example of how to run the migration tool: .. codewithvars:: bash bin/zookeeper-security-migration --zookeeper.acl=secure --zookeeper.connect=localhost:2181 Run this to see the full list of parameters: .. codewithvars:: bash bin/zookeeper-security-migration --help Migrating the |zk| ensemble ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is also necessary to enable authentication on the |zk| ensemble. To do it, you must perform a rolling restart of the ensemble and set a few properties. Please refer to the |zk| documentation for more detail: #. `Apache ZooKeeper documentation `_ #. `Apache ZooKeeper wiki `_