Adding Security to a Running Cluster¶
Migrating Brokers and Clients¶
You can secure a running cluster using one or more of the supported protocols discussed previously. This is done in phases:
- Incrementally restart the cluster nodes to open additional secured port(s).
- Restart clients using the secured rather than
PLAINTEXTport (assuming you are securing the client-broker connection).
- Incrementally restart the cluster again to enable broker-to-broker security (if this is required).
- A final incremental restart to close the
The security implementation lets you configure different protocols for both
broker-client and broker-broker communication. These must be enabled in separate
PLAINTEXT port must be left open throughout so brokers and/or
clients can continue to communicate.
When performing an incremental restart, take into consideration the recommendations for doing rolling restarts to avoid downtime for end users.
As an example, if you want to encrypt both broker-client and broker-broker communication with SSL, in the first incremental restart, open an SSL port on each node:
Then restart the clients, changing their configuration to point at the newly-opened, secured port:
bootstrap.servers=[broker1:9092,...] security.protocol=SSL ...etc
In the second incremental server restart, instruct Apache Kafka® to use SSL as the broker-broker protocol (which will use the same SSL port):
In the final restart, secure the cluster by closing the
Alternatively, you might choose to open multiple ports so that different protocols can be used for broker-broker and broker-client communication. If you want to use SSL encryption throughout (i.e. for broker-broker and broker-client communication), but also want to add SASL authentication to the broker-client connection, open two additional ports during the first restart:
Again, restart the clients, changing their configuration to point at the newly-opened, SASL and SSL secured port:
bootstrap.servers=[broker1:9093,...] security.protocol=SASL_SSL ...etc
The second server restart would switch the cluster to use encrypted broker-broker communication using the SSL port you previously opened on port 9092:
The final restart secures the cluster by closing the
If you are running a version of Kafka that does not support security or you have security disabled and you want to make the cluster secure, then you must perform the following steps to enable ZooKeeper authentication with minimal disruption to your operations:
Migrating ZooKeeper security when the Kafka cluster is not running (no controller node in ZooKeeper) results in a controller node being created, but populated with a null value. In this scenario, leader election may not work properly in subsequent starts of the Kafka cluster.
- Perform a rolling restart setting the JAAS login file, which enables brokers to authenticate. At the end of the rolling restart, brokers are able to manipulate znodes with strict ACLs, but they will not create znodes with those ACLs.
- Perform a second rolling restart of brokers, this time setting the configuration
zookeeper.set.aclto true, which enables the use of secure ACLs when creating znodes.
- Execute the ZkSecurityMigrator tool using the script:
secure. This tool traverses the corresponding sub-trees, changing the ACLs of the znodes.
If you want to turn off authentication in a secure cluster:
- Perform a rolling restart of brokers setting the JAAS login file, which
enables brokers to authenticate, but setting
false. At the end of the rolling restart, brokers stop creating znodes with secure ACLs, but are still able to authenticate and manipulate all znodes.
- Run the ZkSecurityMigrator tool using the script
unsecure. This tool traverses the corresponding sub-trees, changing the ACLs of the znodes.
- Perform a second rolling restart of brokers, this time omitting the system property that sets the JAAS login file.
Here is an example of how to run the migration tool:
bin/zookeeper-security-migration --zookeeper.acl=secure --zookeeper.connect=localhost:2181
Run this command to see the full list of parameters: