<a id="ansible-airgap-deployment"></a>

# Deploy Confluent Platform in Air-Gapped Environment Using Ansible Playbooks

This topic guides you through the steps to use Ansible Playbooks to deploy Confluent Platform
without internet connectivity to outside networks.

This scenario requires:

* A distribution server with internet connectivity

  You download the Ansible Playbooks for Confluent Platform (Confluent Ansible) collection and the Confluent Platform packages
  from the Confluent site to your distribution server.

  You need to install `ansible-galaxy` on the distribution server.
* A control node to run Ansible

  This internal repository should be accessible from your Confluent Platform hosts and the
  distribution server.
* Internal hosts to run Confluent Platform.

  The Confluent Platform hosts must be able to reach the control node at port 8080.

At the high level, the required steps are:

1. Download the Confluent Platform packages and other required files, such as monitoring JARs,
   Confluent CLI, and the Cryptography pip package, along with Ansible
   collections. (This step requires internet connectivity.)
2. Put the packages and other required files that you downloaded in the previous
   step on your repository server.
3. Configure the variables for Ansible to use an internal custom repository
   server for Confluent Platform installation.
4. Install Confluent Platform.

## Deploy using tarball archives

### Step 1: Distribute Confluent Ansible collection

Build the Confluent Ansible collection and install the collection on
the control node.

1. Create a directory with the following structure on the distribution server:
   ```bash
   mkdir -p <path_to_cp-ansible>/ansible_collections/confluent/
   ```

   You can put `<path_to_cp-ansible>` anywhere in your directory
   structure, but the directory structure under
   `<path_to_cp-ansible>` should be set up exactly as specified
   above.
2. Clone the Ansible Playbooks for Confluent Platform repo into the `platform` directory inside the directory
   you created in the previous step:
   ```bash
   git clone https://github.com/confluentinc/cp-ansible \
     <path_to_cp-ansible>/ansible_collections/confluent/platform
   ```
3. Check out the `8.3.0-post` branch:
   ```bash
   cd <path_to_cp-ansible>/ansible_collections/confluent/platform
   git fetch
   git checkout 8.3.0-post
   ```
4. Create the collection:
   ```bash
   ansible-galaxy collection build
   ```

   The command builds a tarball of the collection in the current directory.
5. Upload the collection tarball to your control node:
   ```bash
   scp <collection tarball> <control-node>:/path/to/confluent-collection
   ```
6. Install the collection on the control node:
   ```bash
   ansible-galaxy collection install /path/to/confluent-collection/<collection tarball>
   ```

   For more information about Ansible collection distribution, see [Building your
   collection tarball](https://docs.ansible.com/ansible/devel/dev_guide/developing_collections_distributing.html#building-your-collection-tarball).
7. Download and install the `cryptography` pip package.
   1. Download the `cryptography` pip package to the distribution server:
      ```bash
      pip download cryptography -d <local directory on distribution server>
      ```
   2. Transfer the `cryptography` package to all the Confluent Platform hosts (managed
      nodes):
      ```bash
      scp <local directory on distribution server> <managed-node>:/<some-path-on-managed-node-to-keep-pip-packages>
      ```
   3. Install the `cryptography` package:
      ```bash
      cd <some-path-on-managed-node-to-keep-pip-packages>
      pip install <crpyptography-wheel-file-name> -f ./ --no-index
      ```

      * `-f` argument takes in the local path of directory where `.whl`
        files are present.
      * `--no-index` is for ignoring the remote repo and only looking inside the
        current directory.

### Step 2: Download Confluent Platform archives

1. Download the Confluent Platform archives to the distribution server:
   ```bash
   curl -O http://packages.confluent.io/archive/8.3/confluent-8.3.0.tar.gz
   ```

   If enabling monitoring, download the required JARs:
   ```bash
   curl -o /tmp/confluent-packages/jolokia-jvm-1.6.2-agent.jar \
     http://search.maven.org/remotecontent?filepath=org/jolokia/jolokia-jvm/1.6.2/jolokia-jvm-1.6.2-agent.jar
   ```

   ```bash
   curl -o /tmp/confluent-packages/jmx_prometheus_javaagent-0.12.0.jar \
     https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar
   ```
2. Move the packages and files to the control node:
   ```bash
   scp confluent-packages.tar.gz <control-node>:/path/to/confluent-packages.tar.gz
   ```
3. On the control node, extract the contents of the Confluent Platform archive:
   ```bash
   tar xzf confluent-8.3.0.tar.gz
   ```

### Step 3: Install Confluent Ansible

1. Configure Confluent Ansible as described in [Configure Ansible Playbooks for Confluent Platform](ansible-configure.md#ansible-configure).
2. Add the following in the Ansible inventory file.
   * To install Confluent Platform using a tarball hosted on a web server:
     ```yaml
     all:
       vars:
         repository_configuration: custom
         installation_method: archive
         confluent_archive_file_source: <web server url>/path/to/confluent-8.3.0.tar.gz
     ```
   * To install Confluent Platform using a tarball placed on the control node:
     ```yaml
     all:
       vars:
         installation_method: archive
         confluent_archive_file_source: /path/to/confluent-8.3.0.tar.gz
         confluent_archive_file_remote: false
     ```
3. Configure monitoring in the Ansible inventory file:
   * To enable Jolokia:
     ```yaml
     jolokia_enabled: true
     jolokia_url_remote: false
     jolokia_jar_url: /path/to/jolokia-jvm-1.6.2-agent.jar
     ```
   * To enable JMX exporter:
     ```yaml
     jmxexporter_enabled: true
     jmxexporter_url_remote: false
     jmxexporter_jar_url: /path/to/jmx_prometheus_javaagent-0.12.0.jar
     ```
4. Install Confluent Ansible as described in the [Installation Guide](ansible-install.md#ansible-install).

## Deploy using RHEL or CentOS packages

### Step 1: Distribute Confluent Ansible collection

Build the Confluent Ansible collection and install the collection on the
control node.

1. Create a directory with the following structure on the distribution server:
   ```bash
   mkdir -p <path_to_cp-ansible>/ansible_collections/confluent/
   ```

   You can put `<path_to_cp-ansible>` anywhere in your directory
   structure, but the directory structure under
   `<path_to_cp-ansible>` should be set up exactly as specified
   above.
2. Clone the Ansible Playbooks for Confluent Platform repo into the `platform` directory inside the directory
   you created in the previous step:
   ```bash
   git clone https://github.com/confluentinc/cp-ansible \
     <path_to_cp-ansible>/ansible_collections/confluent/platform
   ```
3. Check out the `8.3.0-post` branch:
   ```bash
   cd <path_to_cp-ansible>/ansible_collections/confluent/platform
   git fetch
   git checkout 8.3.0-post
   ```
4. Create the collection:
   ```bash
   ansible-galaxy collection build
   ```

   The command builds a tarball of the collection in the current directory.
5. Upload the collection tarball to your control node:
   ```bash
   scp <collection tarball> <control-node>:/path/to/confluent-collection
   ```
6. Install the collection on your control node:
   ```bash
   ansible-galaxy collection install /path/to/confluent-collection/<collection tarball>
   ```

   For more information about Ansible collection distribution, see [Building
   your collection tarball](https://docs.ansible.com/ansible/devel/dev_guide/developing_collections_distributing.html#building-your-collection-tarball).
7. Download and install the `cryptography` pip package.
   1. Download the `cryptography` pip package to the distribution server:
      ```bash
      pip download cryptography -d <local directory on distribution server>
      ```
   2. Transfer the `cryptography` package to all the Confluent Platform hosts (managed
      nodes):
      ```bash
      scp <local directory on distribution server> <managed-node>:/<some-path-on-managed-node-to-keep-pip-packages>
      ```
   3. Install the `cryptography` package:
      ```bash
      cd <some-path-on-managed-node-to-keep-pip-packages>
      pip install <crpyptography-wheel-file-name> -f ./ --no-index
      ```

      * `-f` argument takes in the local path of directory where `.whl`
        files are present.
      * `--no-index` is for ignoring the remote repo and only looking inside the
        current directory.

### Step 2: Download Confluent Platform packages

On the distribution server:

1. Install the `epel-release` package:
   ```bash
   yum -y install epel-release
   ```
2. Create the `confluent.repo` file for pulling the Confluent Platform packages:
   ```bash
   echo "[Confluent.dist]
   baseurl = https://packages.confluent.io/rpm/8.3
   enabled = 1
   gpgcheck = 1
   gpgkey = https://packages.confluent.io/rpm/8.3/archive.key
   name = Confluent repository (dist)

   [Confluent]
   baseurl = https://packages.confluent.io/rpm/8.3
   enabled = 1
   gpgcheck = 1
   gpgkey = https://packages.confluent.io/rpm/8.3/archive.key
   name = Confluent repository" \
   > /etc/yum.repos.d/confluent.repo
   ```
3. Create a directory and download the Confluent Platform packages into that directory:
   ```bash
   mkdir /tmp/confluent-packages/
   ```

   ```bash
   yum clean all

   yum install --downloadonly --downloaddir=/tmp/confluent-packages/ \
     confluent-platform \
     confluent-security \
     nc \
     ansible \
     createrepo \
     java-21-openjdk
   ```
4. Download the required JARs if you want to enable monitoring:
   ```bash
   curl -L -o /tmp/confluent-packages/jolokia-jvm-1.6.2-agent.jar \
     http://search.maven.org/remotecontent?filepath=org/jolokia/jolokia-jvm/1.6.2/jolokia-jvm-1.6.2-agent.jar

   curl -o /tmp/confluent-packages/jmx_prometheus_javaagent-0.12.0.jar \
     https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar
   ```

### Step 3: Host Confluent Platform packages on control node

Copy the Confluent Platform packages to the control node and run a simple repository server
over a Python web server.

1. On the distribution server, package and move the the files to the control
   node:
   ```bash
   cd /tmp

   tar -zcvf confluent-packages.tar.gz confluent-packages

   scp /tmp/confluent-packages.tar.gz <control-node>:/tmp/confluent-packages.tar.gz
   ```
2. On the control node, decompress and install the packages:
   ```bash
   cd /tmp
   tar xvzf confluent-packages.tar.gz -C /tmp
   ```

   ```bash
   cd /tmp/confluent-packages/
   yum -y localinstall createrepo-*.rpm
   ```
3. Create a repo, and host the control node with python3 http.server:
   ```bash
   createrepo .
   ```

   ```bash
   python3 -m http.server 8080
   ```

### Step 4: Install Confluent Ansible

Install Ansible on the control node, create a repo file for the rest of the Confluent Platform
hosts to use, and install Confluent Platform.

1. On the control node, install Confluent Ansible:
   ```bash
   cd /tmp/confluent-packages/
   yum -y localinstall ansible-*.rpm
   ```
2. On the control node, create a `.repo` file that each host will use.
   If your organization has a repository server, that file should be configured
   for the existing repository server.
   ```bash
   echo "[confluent-repo]
   name=Confluent Repo
   baseurl=http://<ansible-control-node>:8080
   enabled=1
   gpgcheck=0" \
   > /tmp/confluent.repo
   ```
3. Configure Confluent Ansible as described in [Configure Ansible Playbooks for Confluent Platform](ansible-configure.md#ansible-configure), and add
   the following in the Ansible inventory file:
   ```yaml
   all:
     vars:
       repository_configuration: custom
       custom_yum_repofile_filepath: /tmp/confluent.repo
   ```
4. Configure monitoring in the Ansible inventory file:
   * To enable Jolokia:
     ```yaml
     jolokia_enabled: true
     jolokia_url_remote: false
     jolokia_jar_url: http://<control-node>:8080/jolokia-jvm-1.6.2-agent.jar
     ```
   * To enable JMX exporter:
     ```yaml
     jmxexporter_enabled: true
     jmxexporter_url_remote: false
     jmxexporter_jar_url: http://<control-node>:8080/jmx_prometheus_javaagent-0.12.0.jar
     ```
5. Install Confluent Ansible as described in the [Installation Guide](ansible-install.md#ansible-install).

## Deploy using Ubuntu or Debian packages

### Step 1: Distribute Confluent Ansible collection

Build the Confluent Ansible collection and install the collection on the control
node.

1. Create a directory with the following structure on the distribution server:
   ```bash
   mkdir -p <path_to_cp-ansible>/ansible_collections/confluent/
   ```

   You can put `<path_to_cp-ansible>` anywhere in your directory
   structure, but the directory structure under
   `<path_to_cp-ansible>` should be set up exactly as specified
   above.
2. Clone the Ansible Playbooks for Confluent Platform repo into the `platform` directory inside the directory
   you created in the previous step:
   ```bash
   git clone https://github.com/confluentinc/cp-ansible \
     <path_to_cp-ansible>/ansible_collections/confluent/platform
   ```
3. Check out the `8.3.0-post` branch:
   ```bash
   cd <path_to_cp-ansible>/ansible_collections/confluent/platform
   git fetch
   git checkout 8.3.0-post
   ```
4. Create the collection:
   ```bash
   ansible-galaxy collection build
   ```

   The command builds a tarball of the collection in the current directory.
5. Upload the collection tarball to your control node:
   ```bash
   scp <collection tarball> <control-node>:/path/to/confluent-collection
   ```
6. Install the collection on your control node:
   ```bash
   ansible-galaxy collection install /path/to/confluent-collection/<collection tarball>
   ```

   For more information about Ansible collection distribution, see [Building
   your collection tarball](https://docs.ansible.com/ansible/devel/dev_guide/developing_collections_distributing.html#building-your-collection-tarball).
7. Download and install the `cryptography` pip package.
   1. Download the `cryptography` pip package to the distribution server:
      ```bash
      pip download cryptography -d <local directory on distribution server>
      ```
   2. Transfer the `cryptography` package to all the Confluent Platform hosts (managed
      nodes):
      ```bash
      scp <local directory on distribution server> <managed-node>:/<some-path-on-managed-node-to-keep-pip-packages>
      ```
   3. Install the `cryptography` package:
      ```bash
      cd <some-path-on-managed-node-to-keep-pip-packages>
      pip install <crpyptography-wheel-file-name> -f ./ --no-index
      ```

      * `-f` argument takes in the local path of directory where `.whl`
        files are present.
      * `--no-index` is for ignoring the remote repo and only looking inside the
        current directory.

### Step 2: Download Confluent Platform packages

1. Download the Confluent Platform packages and copy them into a directory for distribution:
   ```bash
   # Downloads debs and their dependencies into /var/cache/apt/archives
   apt-get install --download-only --yes \
     confluent-platform \
     confluent-security \
     ansible \
     openjdk-11-jdk \
     rsync \
     ca-certificates \
     apt-transport-https \
     python-apt \
     dpkg-dev
   ```

   ```bash
   mkdir /tmp/confluent-packages

   cp /var/cache/apt/archives/*.deb /tmp/confluent-packages
   ```
2. Add the Confluent repository key and the repository on the distribution
   server:
   ```bash
   wget -qO - https://packages.confluent.io/deb/8.3/archive.key | sudo apt-key add -
   ```

   ```bash
   add-apt-repository "deb [arch=amd64] https://packages.confluent.io/deb/8.3 stable main"
   ```

   ```bash
   apt install software-properties-common --yes

   apt-add-repository --yes --update ppa:ansible/ansible

   apt-get update
   ```
3. Download the required JARs if you want to enable monitoring:
   ```bash
   curl -o /tmp/confluent-packages/jolokia-jvm-1.6.2-agent.jar \
     http://search.maven.org/remotecontent?filepath=org/jolokia/jolokia-jvm/1.6.2/jolokia-jvm-1.6.2-agent.jar

   curl -o /tmp/confluent-packages/jmx_prometheus_javaagent-0.12.0.jar \
     https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar
   ```

### Step 3: Host Confluent Platform packages on control node

Copy the Confluent Platform packages to the control node and run a simple repository server
over a Python web server.

1. On the distribution server, package and move the the files to the control
   node:
   ```bash
   cd /tmp

   tar -zcvf confluent-packages.tar.gz confluent-packages

   scp /tmp/confluent-packages.tar.gz <control-node>:/tmp/confluent-packages.tar.gz
   ```
2. On the control node, decompress and install the packages:
   ```bash
   cd /tmp
   tar xvzf confluent-packages.tar.gz -C /tmp

   cd /tmp/confluent-packages/
   apt install ./dpkg-dev*.deb --yes

   dpkg-scanpackages ./ /dev/null | gzip > Packages.gz
   ```
3. On the control node, host a repository server with http.server:
   ```bash
   python3 -m http.server 8080
   ```

### Step 4: Install Confluent Ansible

Install Ansible on the control node, create a repo file for the rest of the Confluent Platform
hosts to use, and install Confluent Ansible.

1. On the control node, install Ansible:
   ```bash
   cd /tmp/confluent-packages/
   apt install ./ansible-*.deb --yes
   ```
2. Create a `.list` file that each Confluent Platform host will use:
   ```bash
   echo "deb [trusted=yes] http://<control-node>:8080 ./" > /tmp/custom_confluent.list
   ```
3. Configure Confluent Ansible as described in [Configure Ansible Playbooks for Confluent Platform](ansible-configure.md#ansible-configure).
4. Add the following in the Ansible inventory file:
   ```yaml
   all:
     vars:
       repository_configuration: custom
       custom_apt_repo_filepath: /tmp/custom_confluent.list
   ```
5. Configure monitoring in the Ansible inventory file:
   * To enable Jolokia:
     ```yaml
     jolokia_enabled: true
     jolokia_url_remote: false
     jolokia_jar_url: http://<control-node>:8080/jolokia-jvm-1.6.2-agent.jar
     ```
   * To enable JMX exporter:
     ```yaml
     jmxexporter_enabled: true
     jmxexporter_url_remote: false
     jmxexporter_jar_url: http://<control-node>:8080/jmx_prometheus_javaagent-0.12.0.jar
     ```
6. Install Confluent Ansible as described in the [Installation Guide](ansible-install.md#ansible-install).