Upgrade with Ansible Playbooks for Confluent Platform¶
Ansible Playbooks for Confluent Platform includes playbooks to upgrade all Confluent Platform components. The playbooks go host by host, shutting down the component, upgrading packages, and validating application health before moving onto the next one.
You must complete the upgrades in the following order:
- Upgrade Zookeeper.
- Upgrade Kafka brokers.
- Upgrade (in any order):
- Schema Registry
- REST Proxy
- Upgrade Confluent Control Center.
- Upgrade external clients.
- Upgrade Kafka log format. This step ensures that the log is formatted properly for the new version of Confluent Platform after all upgrades have been completed.
The upgrade playbooks have the following requirements:
- Component must have been originally installed and configured using Ansible Playbooks for Confluent Platform.
- You must have the same
hosts.ymlfile used during the installation.
- Upgrade playbooks are supported in Confluent Platform version 5.3.0 and later.
- The Kafka broker upgrade playbook requires the jolokia agent be running on all Kafka brokers. Jolokia is required for the upgrade to work.
Step 1. Check out the upgrade branch¶
Check out the branch of cp-ansible you want to upgrade to. Branches will be in
<version>-post. For example
cd /path/to/cp-ansible git fetch git checkout 5.5.1-post
Step 2. Upgrade ZooKeeper¶
The ZooKeeper upgrade is designed to run on Kafka clusters with three nodes or more. As it upgrades the hosts it checks for quorum and health on each host before moving on to the next. Enter the command below to upgrade ZooKeeper:
ansible-playbook -i /path/to/hosts.yml upgrade_zookeeper.yml
Step 3. Upgrade Kafka¶
The Kafka broker upgrade playbook upgrades each broker in sequence. The upgrade
takes special care around the
log.message.format.version properties. First, enter the following command
and note the current version of
installed on your hosts:
# rpm -qa | grep confluent confluent-server-5.3.0-1.noarch
Next, run the upgrade_kafka_broker playbook and pass in the version installed:
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker.yml -e kafka_broker_upgrade_start_version=5.3.0
This playbook upgrades the Kafka brokers in sequence.
log.message.format.version property is not updated when the Kafka upgrade completes. This property is changed at the end of this procedure, after all clients have been upgraded.
Step 4. Upgrade other components¶
If you need to upgrade specific hosts instead of all of them, you can limit the upgrade. This can be useful when your components are behind a load balancer. In this case, remove a specific host from the load balancer pool, upgrade it, then add it back. This ensures no traffic is disrupted. Enter the following command to limit the upgrade to one or more specific hosts.
ansible-playbook -i /path/to/hosts.yml upgrade_<component>.yml --limit "<host1>,<host2>"
Use the commands below to update the other components:
ansible-playbook -i /path/to/hosts.yml upgrade_schema_registry.yml
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_connect.yml
ansible-playbook -i /path/to/hosts.yml upgrade_ksql.yml
The minimum version you can upgrade ksqlDB from is 5.5.0.
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_rest.yml
Confluent Control Center
ansible-playbook -i /path/to/hosts.yml upgrade_control_center.yml
Step 5. Upgrade clients¶
Confluent Platform Ansible playbooks do not currently support upgrading clients. Review Preparation for additional information.
Step 6. Update the Kafka log format¶
After all of the applicable clients have been updated, run the following command to update update the log format:
ansible-playbook -i /path/to/hosts.yml upgrade_kafka_broker_log_format.yml -e kafka_broker_upgrade_end_version=5.5