Migrate from ZooKeeper to KRaft (EA)

Migrating from ZooKeeper to KRaft means migrating existing metadata from Kafka brokers that are using ZooKeeper to manage metadata, to brokers that are using a KRaft quorum controller to manage metadata in Apache Kafka®. This topic will walk you through how to perform the migration.

Important

The ZooKeeper to KRaft migration feature is an early access feature (EA) and should be used only for evaluation and non-production testing purposes or to provide feedback to Confluent, particularly as it becomes more widely available in follow-on preview editions.

Early Access Program features are intended for evaluation use in development and testing environments only, and not for production use. The warranty, SLA, and Support Services provisions of your agreement with Confluent do not apply to Early Access Program features. Confluent may discontinue providing preview releases of the Early Access Program features at any time in Confluent’s sole discretion.

Prerequisites

  • Upgrade to Confluent Platform 7.5.6. Before you migrate from ZooKeeper to KRaft you should upgrade to the latest Confluent Platform version. See Upgrade Confluent Platform for the considerations and steps to do this.

  • To help with debugging, enable TRACE level logging for metadata migration. Add the following line to the log4j.properties file found in the CONFLUENT_HOME/etc/kafka/ directory:

    log4j.logger.org.apache.kafka.metadata.migration=TRACE

Step 1: Retrieve the cluster ID

You must format storage for your Kafka cluster with the ID of the existing cluster. You can get this ID with the zookeeper-shell tool. For example:

./bin/zookeeper-shell localhost:2181

Connecting to localhost:2181
Welcome to ZooKeeper!

get /cluster/id

{"version":"1","id":"WZEKwK-bS62oT3ZOSU0dgw"}

Save this ID, you will use it later after you configure the KRaft controllers.

Step 2: Configure and provision a KRaft controller quorum

Deploy a set of KRaft controllers that will take over from ZooKeeper. Configure and start each of the KRaft controllers with the following:

  • A node.id that is unique across all brokers and controllers.
  • Migration enabled with zookeeper.metadata.migration.enable=true.
  • ZooKeeper connection configuration.
  • Other KRaft-mode required properties, such as controller.quorum.voters, and controller.listener.names.

Following is an example controller.properties file, for a controller listening on port 9093:

process.roles=controller
node.id=3000
controller.quorum.voters=3000@localhost:9093
controller.listener.names=CONTROLLER
listeners=CONTROLLER://:9093

# Enable the migration
  zookeeper.metadata.migration.enable=true

# ZooKeeper client configuration
  zookeeper.connect=localhost:2181

# Other configuration entries ...

Step 3: Format storage with the ID you saved previously

Format storage with the ID and the controller configuration file. For example:

./bin/kafka-storage format --config ./etc/kafka/kraft/controller.properties --cluster-id WZEKwK-bS62oT3ZOSU0dgw

You might see output like the following:

Formatting /tmp/kraft-controller-logs with metadata version 3.5-IV0

Step 4: Enable migration on the brokers

Once the KRaft controllers are started, you will reconfigure each broker for KRaft migration and restart the broker. You can do a rolling restart to help ensure cluster availability during the migration. Metadata migration automatically starts when all of the brokers have been restarted.

Set the following for brokers:

  • inter.broker.protocol.version to version 3.5.
  • Enable migration with zookeeper.metadata.migration.enable=true.
  • The ZooKeeper connection configuration (zookeeper.connect).
  • Any other KRaft-mode required properties, such as controller.quorum.voters and controller.listener.names.

Following is an example configuration file for a broker that is ready for the KRaft migration.

broker.id=0
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://localhost:9092
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT

# Set the IBP
  inter.broker.protocol.version=3.5

# Enable the migration
  zookeeper.metadata.migration.enable=true

# ZooKeeper client configuration
  zookeeper.connect=localhost:2181

# KRaft controller quorum configuration
  controller.quorum.voters=3000@localhost:9093
  controller.listener.names=CONTROLLER

When all of the Kafka brokers running with ZooKeeper for metadata management are restarted with migration properties set, the migration automatically begins. When migration is complete, you should see the following entry in a log at INFO level.

Completed migration of metadata from ZooKeeper to KRaft.

Step 5: Migrate the brokers

At this point, the metatdata migration should be complete, but the brokers are still running in ZooKeeper mode.

The KRaft controller is running in migration mode, and it will send controller remote procedure calls (RPCs) such as UpdateMetadata and LeaderAndIsr to the ZooKeeper-mode brokers.

To migrate the brokers to KRaft, you need to reconfigure them as KRaft brokers and restart them.

Using the migration broker configuration as an example, replace the broker.id with node.id (maintaining the same identifier) and add process.roles=broker, which signals KRaft mode.

To finish the broker migration:

  • Remove the ZooKeeper configuration entries.
  • If you are using ACLs, change the authorizer class. For more information, see ACL concepts.
  • Restart the broker.

Following is an example of how a server.properties file for a migrated broker might look. Note that ZooKeeper-specific properties are commented out.

process.roles=broker
node.id=0
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://localhost:9092
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT

# Remove the IBP, KRaft uses "metadata.version" feature flag
# inter.broker.protocol.version=3.5

# Remove the migration enabled flag
# zookeeper.metadata.migration.enable=true

# Remove ZooKeeper client configuration
# zookeeper.connect=localhost:2181

# Keep the KRaft controller quorum configuration
  controller.quorum.voters=3000@localhost:9093
  controller.listener.names=CONTROLLER

# If using ACLs, change the authorizer from AclAuthorizer used for ZooKeeper to the StandardAuthorizer used for KRaft.
  authorizer.class.name=kafka.security.authorizer

When you have finished updating the configuration and restarting each broker, the cluster is running in KRaft mode.

Step 6: Take KRaft controllers out of migration mode

The final step of the migration is to remove the zookeeper.metadata.migration.enable property to take the controllers out of migration mode, and to remove the ZooKeeper configuration entry. Following is an example controller.properties file for a controller that is migrated to KRaft mode and is listening on port 9093. Note that the ZooKeeper-specific properties are commented out.

process.roles=controller
node.id=3000
controller.quorum.voters=1@localhost:9093
controller.listener.names=CONTROLLER
listeners=CONTROLLER://:9093

# Disable migration
# zookeeper.metadata.migration.enable=true

# Remove ZooKeeper client configuration.
# zookeeper.connect=localhost:2181

After this step, restart each controller and your cluster should be migrated to KRaft mode.

Related content