Prepare Ansible Inventory File to Install Confluent Platform

Before running the Ansible playbooks, you need to generate an inventory file. The inventory file specifies the hosts in which to provision Confluent Platform components. For more information about the Ansible inventory file, see Ansible Inventory Basics.

Generate an inventory file

To generate an inventory file, gather all of the Fully Qualified Domain Names (FQDNs) of your hosts and create a file called hosts.yml on your Ansible control node, setting each hostname under the desired groups as shown below.

The built-in inventory_hostname variable of each host is set to the hostname of the host. The hostname can be internal or external addresses that can be reachable from the control node.

For example, when using KRaft:

kafka_controller:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:
    ip-172-31-37-15.us-east-2.compute.internal:
    ip-172-31-34-231.us-east-2.compute.internal:

kafka_broker:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:
    ip-172-31-34-247.us-east-2.compute.internal:
    ip-172-31-34-248.us-east-2.compute.internal:

schema_registry:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:

kafka_rest:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:

ksql:
  hosts:
    ip-172-31-37-15.us-east-2.compute.internal:
    ip-172-31-37-16.us-east-2.compute.internal:

kafka_connect:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:

control_center:
  hosts:
    ip-172-31-37-15.us-east-2.compute.internal:

When using ZooKeeper, replace the KRaft controller (kafka_controller) hosts in the above example with ZooKeeper hosts as shown below:

zookeeper:
  hosts:
    ip-172-31-34-246.us-east-2.compute.internal:
    ip-172-31-37-15.us-east-2.compute.internal:
    ip-172-31-34-231.us-east-2.compute.internal:

Use ansible_host for SSH connections

When your inventory_hostname does not work with SSH, you can specify one additional hostname for SSH connection using the ansible_host variable.

In the following example, ip-172-31-40-189.us-west-2.compute.internal is the inventory_hostname, and ec2-34-217-174-252.us-west-2.compute.amazonaws.com is used for SSH.

kafka_broker:
  hosts:
    ip-172-31-40-189.us-west-2.compute.internal:
      ansible_host: ec2-34-217-174-252.us-west-2.compute.amazonaws.com

Verify connection to Confluent Platform hosts

After generating an inventory file, set connection variables so that the Ansible control node can connect to each Confluent Platform host.

Most commonly, Ansible uses SSH for its connections. For more information about setting up connection variables, see Connecting to hosts: behavioral inventory parameters.

Add the following section to hosts.yml:

all:
  vars:
    ansible_connection: ssh
    ansible_user: ec2-user
    ansible_become: true
    ansible_ssh_private_key_file: /tmp/certs/ssh_priv.pem

Use the following command to verify that Ansible can connect over SSH:

ansible -i /path/to/hosts.yml all -m ping

The above command validates that a Python Interpreter is available for use on all the hosts, and it returns pong on success.

If you cannot reach the host even after providing the SSH private key path in the hosts.yml file for Ansible, the SSH private key file might have incorrect permissions.

Use the chmod command to update the permissions of the SSH private key file to read-only for the owner.

chmod 400 <ansible_ssh_private_key_file>

It is recommended that you store your inventory file in its own Git repository. You may have a Git repo with an inventory file for each of your deployments.