Install Confluent Platform with Ansible Playbooks

This topic describes the installation steps for Confluent Platform.

Install all Confluent Platform components

  1. Configure required settings in the hosts.yml file as described in Configure Ansible Playbooks for Confluent Platform.

  2. (Optional) Validate hosts before installing Confluent Platform:

    ansible-playbook -i hosts.yml confluent.platform.validate_hosts

    To validate hosts for an individual component, use the --tag flag as below:

    ansible-playbook -i hosts.yml confluent.platform.validate_hosts --tags=<component-name>

    <component-name> can be zookeeper, kafka_broker, kafka_controller, schema_registry, kafka_rest, kafka_connect, ksql, control_center, or control_center_next_gen.

  3. Install or update Control Center.

  4. Install or update all Confluent Platform:

    ansible-playbook -i hosts.yml confluent.platform.all

Install individual Confluent Platform components

You can use the Ansible tags to install individual components. You must follow the correct order given below to satisfy the dependencies among the components. For example, Kafka brokers will not install until the certificate authority is generated and either ZooKeeper or KRaft controller is installed.

Note

As of Confluent Platform 7.5, ZooKeeper is deprecated for new deployments. Confluent recommends KRaft mode for new deployments. For more information, see KRaft Overview.

Similarly, when you update a component section in the hosts.yml file, you can optionally update the component and the preceding components in the order.

  1. Generate the certificate authority (CA) if TLS encryption is enabled and using self-signed certificates. Additionally, a key pair for the MDS token, a private key and a public certificate, will be generated.

    ansible-playbook -i hosts.yml confluent.platform.all --tags=certificate_authority

  2. Install Control Center.

  3. Install ZooKeeper or KRaft controller. Do not install both.

    ansible-playbook -i hosts.yml confluent.platform.all --tags=zookeeper

    ansible-playbook -i hosts.yml confluent.platform.all --tags=kafka_controller

  4. Install Kafka.

    ansible-playbook -i hosts.yml confluent.platform.all --tags=kafka_broker

  5. Install Schema Registry, REST Proxy, Connect, and ksqlDB in any order.

    ansible-playbook -i hosts.yml confluent.platform.all --tags=schema_registry
ansible-playbook -i hosts.yml confluent.platform.all --tags=kafka_rest
ansible-playbook -i hosts.yml confluent.platform.all --tags=kafka_connect
ansible-playbook -i hosts.yml confluent.platform.all --tags=ksql

  6. Install Control Center (Legacy).

    ansible-playbook -i hosts.yml confluent.platform.all --tags=control_center

Install Confluent Control Center

Confluent Control Center is available as a separate package and follows independent versions.

Control Center requires the bcrypt function. Install bcrypt with the following command on the control node:

pip install bcrypt

After you prepare the host and configure, run the following command to install Control Center:

ansible-playbook -i hosts.yml confluent.platform.all --tags=control_center_next_gen

The Control Center package includes Prometheus and Alert Manager.

Now when you install Kafka and KRaft as described above in Install all Confluent Platform components or Install individual Confluent Platform components, Kafka and KRaft start sending metrics to Control Center, and you will see the metrics in Control Center.

Install Confluent Platform without root access

Ansible Playbooks for Confluent Platform is a collection of tasks, some of which do not require root access. You can run those tasks as a non-privileged user. However, there are certain steps where you do need root access.

If you do not have root access, you can use Ansible tags to filter out and skip running the tasks that need root access.

The skipped tasks that need root access are still required to be executed, for example, using a customized script or another Ansible tower that has root access.

The following tags are used to separate out tasks that require root and non-root access:

  • systemd: Tag for systemd-related tasks

  • sysctl: Tag for sysctl-related tasks on the Kafka brokers

  • privileged: Tag for tasks that require elevated permissions

  • filesystem: Tag for filesystem-related tasks across components

    Use this tag to skip configuration folder management, such as creating, updating, and changing permissions on files and folders, usually along with the other tags, such as privileged and systemd.

  • configuration: Tag for configuration-related tasks, such as creating, editing, changing permissions for component config files

If you do not have root access, skip the root access tasks with the --skip-tags flag and the above tags, for example, --skip-tags privileged,sysctl,systemd,filesystem,configuration.

For details on Ansible tags, see Confluent Ansible Playbook tags.

For a sample inventory, see non-root sample inventory.

Next step