Migrate Confluent Platform to Confluent Server

Confluent Platform includes a commercial component named confluent-server. This includes a Kafka broker and support for commercial features such as RBAC, Tiered Storage and Confluent for Kubernetes. Confluent Platform also includes a version of confluent-kafka, which is an open source Apache 2.0 licensed component. Both confluent-community and confluent-platform packages have a dependency on confluent-kafka. You can replace confluent-kafka with confluent-server package to use commercial features not available in confluent-kafka. For more information about the packages, see Confluent Platform Packages.

You can migrate from confluent-kafka to confluent-server using the instructions in the following sections.

Note that Confluent Server requires that the confluent.license property be set to a valid license string in each broker properties file, and this license is checked at broker start up. For more information, see Manage Confluent Platform Licenses.

Prerequisites

If you have already installed some packages using confluent-kafka, you must remove this package before installing confluent-server. You may need to run yum autoremove or apt autoremove to remove packages installed as dependencies by a previous install.

Red Hat Enterprise Linux

If your cluster was installed on Red Hat Enterprise Linux (RHEL), you can migrate from confluent-kafka to confluent-server or from confluent-server to confluent-kafka.

Migrating from confluent-kafka to confluent-server

Complete the following steps to migrate to confluent-server when using Red Hat Enterprise Linux (RHEL).

Step 1: Install Confluent Platform

  1. Install Confluent Platform. For more information, see Install Confluent Platform using Systemd on RHEL, CentOS, and Fedora-based Linux.
  2. Start your cluster and verify that it is working as expected.

Step 2: Migrate the controllers (KRaft mode)

In KRaft mode, you must migrate all of the controllers before you migrate the brokers. If you are running in ZooKeeper mode, you can skip this step.

  1. Log in to and stop each KRaft controller.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the configuration files:

    sudo cp /etc/kafka/kraft/controller.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Use yum swap to change to confluent-server:

    yum swap confluent-kafka-7.9.4 confluent-server

  4. Copy the configuration backup to the Kafka configuration directory:

    sudo cp /tmp/controller.properties /etc/kafka/kraft

  5. Copy your override file using the following command:

    sudo cp /tmp/override.conf /etc/systemd/system/confluent-server.service.d/

  6. If you are running Kerberos, copy your JAAS file to the configuration directory:

    sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/kraft

  7. Start your Kafka controllers:

    sudo systemctl start confluent-server

Step 3: Migrate the brokers

  1. Log in to and stop each Kafka broker.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the configuration files:

    sudo cp /etc/kafka/kraft/broker.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Use yum swap to change to confluent-server:

    yum swap confluent-kafka-7.9.4 confluent-server

  1. Copy your configuration files to the Kafka configuration directory:

    sudo cp /tmp/broker.properties /etc/kafka/kraft/

    If you are running Kerberos, copy your JAAS file to the Kafka configuration directory:

    sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/kraft/

  2. Copy your override file using the following command:

    sudo cp /tmp/override.conf /etc/systemd/system/confluent-server.service.d/

  3. Start your Kafka nodes.

    sudo systemctl start confluent-server

Migrating from confluent-server to confluent-kafka

Complete the following steps to change from confluent-server back to confluent-kafka when using Red Hat Enterprise Linux (RHEL).

Important

The confluent-server package is a superset of the confluent-kafka package. It includes all of the functionality of confluent-kafka, but also includes additional features.

If you are using Kafka in KRaft mode, you currently cannot migrate from confluent-server to confluent-kafka using the techniques described in this topic.

Step 1: Migrate the controllers (KRaft mode)

In KRaft mode, you must migrate all of the controllers before you migrate the brokers. If you are running in ZooKeeper mode, you can skip this step.

  1. Log in to and stop each Kafka controller.

    sudo systemctl stop confluent-server

  2. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/kraft/controller.properties /tmp
sudo cp /etc/systemd/system/confluent-server.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Use yum swap to change to confluent-kafka:

    yum swap confluent-kafka-7.9.4 confluent-kafka

  4. Start the controllers:

    sudo systemctl start confluent-kafka

Step 2: Migrate the brokers

  1. Log in to and stop each Kafka broker.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the configuration files:

    sudo cp /etc/kafka/kraft/broker.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Use yum swap to change to confluent-kafka:

    yum swap confluent-kafka-7.9.4 confluent-kafka

  1. Copy your configuration files to the Kafka configuration directory:

    sudo cp /tmp/broker.properties /etc/kafka/kraft/

    If you are running Kerberos, copy your JAAS file to the Kafka configuration directory:

    sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/kraft/

  2. Start your Kafka nodes:

    sudo systemctl start confluent-kafka

Ubuntu

If your cluster was installed on Ubuntu, you can migrate from confluent-kafka to confluent-server or from confluent-server to confluent-kafka.

Migrating from confluent-kafka to confluent-server

Complete the following steps to migrate to confluent-server when using Ubuntu.

Step 1: Install Confluent Platform.

  1. Install Confluent Platform. For more information, see Install Confluent Platform using Systemd on Ubuntu and Debian.
  2. Start your cluster and verify that it is working as expected.

Step 2: Migrate the controllers (KRaft mode)

In KRaft mode, you must migrate all of the controllers before you migrate the brokers. If you are running in ZooKeeper mode, you can skip this step.

  1. Log in to and stop each KRaft controller.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/kraft/controller.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Remove confluent-kafka from your Kafka controller hosts.

    sudo apt-get remove confluent-kafka-7.9.4

  4. Install confluent-server on your Kafka controller hosts.

    sudo apt-get install confluent-server

  5. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

  6. Copy your override file using the following command:

    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf
/etc/systemd/system/confluent-server.service.d

  7. Start the controllers.

    sudo systemctl start confluent-server

Step 3: Migrate the brokers

  1. Log in to and stop each Kafka brokers.

    sudo systemctl stop confluent-kafka

  1. Log in to and stop each Kafka broker.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the configuration files:

    sudo cp /etc/kafka/kraft/broker.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Remove confluent-kafka from your Kafka broker hosts.

    sudo apt-get remove confluent-kafka-7.9.4

  4. Install confluent-server on your Kafka broker hosts.

    sudo apt-get install confluent-server

  5. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

  6. Copy your override file using the following command:

    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf
/etc/systemd/system/confluent-server.service.d

  7. Start the Kafka brokers.

    sudo systemctl start confluent-server

Migrating from confluent-server to confluent-kafka

Complete the following steps to change from confluent-server back to confluent-kafka when using Ubuntu.

Important

The confluent-server package is a superset of the confluent-kafka package. It includes all of the functionality of confluent-kafka, but also includes additional features.

If you are using Kafka in KRaft mode, you currently cannot migrate from confluent-server to confluent-kafka using the techniques described in this topic.

Step 1: Migrate the controllers (KRaft mode)

In KRaft mode, you must migrate all of the controllers before you migrate the brokers. If you are running in ZooKeeper mode, you can skip this step.

  1. Log in to and stop each KRaft controller.

    sudo systemctl stop confluent-server

  2. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/kraft/controller.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you need to back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Remove confluent-server from your controller hosts.

    sudo apt-get remove confluent-server

  4. Install confluent-kafka on your controller hosts.

    sudo apt-get install confluent-kafka-7.9.4

  5. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

  6. Start the controllers.

    sudo systemctl start confluent-kafka

Step 2: Migrate the brokers

  1. Log in to and stop each Kafka broker.

    sudo systemctl stop confluent-server

  1. Log in to and stop each Kafka broker.

    sudo systemctl stop confluent-kafka

  2. As a precaution, back up the configuration files:

    sudo cp /etc/kafka/kraft/broker.properties /tmp
sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kraft/kafka_server_jaas.conf /tmp

  3. Remove confluent-server from your Kafka broker hosts.

    sudo apt-get remove confluent-server

  4. Install confluent-kafka on your Kafka broker hosts.

    sudo apt-get install confluent-kafka-7.9.4

  5. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

  6. Start your Kafka brokers.

    sudo systemctl start confluent-kafka

Related content