Upgrade Confluent Platform with Ansible Playbooks
Ansible Playbooks for Confluent Platform includes playbooks to upgrade all Confluent Platform components. The playbooks go
host by host, shutting down the component, upgrading packages, and validating
application health before moving onto the next one.
You must complete the upgrades in the following order:
- Upgrade Zookeeper.
- Upgrade Kafka brokers.
- Upgrade (in any order):
- Schema Registry
- REST Proxy
- Upgrade Confluent Control Center.
- Upgrade external clients.
- Upgrade Kafka log format. This step ensures that the log is formatted properly for the new version of Confluent Platform after all upgrades have been completed.
The upgrade playbooks have the following requirements:
- Component must have been originally installed and configured using Ansible Playbooks for Confluent Platform.
- You must have the same
hosts.yml file used during the installation.
- Upgrade playbooks are supported in Confluent Platform version 5.3.0 and later.
Step 1. Check out the upgrade branch
Check out the branch of
cp-ansible you want to upgrade to. Branches are in
<version>-post, for example, 6.1.0-post.
git checkout 6.1.0-post
Step 2. Upgrade ZooKeeper
The ZooKeeper upgrade is designed to run on Kafka clusters with three nodes or more.
As it upgrades the hosts it checks for quorum and health on each host before
moving on to the next. Enter the command below to upgrade ZooKeeper:
ansible-playbook -i /path/to/hosts.yml upgrade_zookeeper.yml
Step 3. Upgrade Kafka
The Kafka broker upgrade playbook upgrades each broker in sequence. The upgrade
takes special care around the
Enter the following command and note the current version of
confluent-server installed on your hosts:
rpm -qa | grep confluent
upgrade_kafka_broker playbook and pass in the version installed:
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker.yml -e kafka_broker_upgrade_start_version=5.5.1
This playbook upgrades the Kafka brokers in sequence.
log.message.format.version property is not updated when the Kafka upgrade completes. This property is changed at the end of this procedure, after all clients have been upgraded.
Step 4. Upgrade other components
If you need to upgrade specific hosts instead of all of them, you can limit the upgrade. This can be useful when your components are behind a load balancer. In this case, remove a specific host from the load balancer pool, upgrade it, then add it back. This ensures no traffic is disrupted. Enter the following command to limit the upgrade to one or more specific hosts.
ansible-playbook -i /path/to/hosts.yml upgrade_<component>.yml --limit "<host1>,<host2>"
Use the commands below to update the other components:
Step 6. Update the Kafka log format
After all of the applicable clients have been updated, run the following command to update update the log format:
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker_log_format.yml -e kafka_broker_upgrade_end_version=6.1