Migrate Confluent Platform to Confluent Server

Confluent Platform includes a commercial component named confluent-server. This includes a Kafka broker and support for commercial features such as RBAC, Tiered Storage and Confluent for Kubernetes. Confluent Platform also includes a version of confluent-kafka, which is an open source Apache 2.0 licensed component. Both confluent-community and confluent-platform packages have a dependency on confluent-kafka. You can replace confluent-kafka with confluent-server package to use commercial features not available in confluent-kafka. For more information about the packages, see Confluent Platform Packages.

You can migrate from confluent-kafka to confluent-server using the instructions in the following sections.

Note that Confluent Server requires that the confluent.license property be set to a valid license string in each broker properties file, and this license is checked at broker start up. For more information, see Manage Confluent Platform Licenses.

Prerequisites

If you have already installed some packages using confluent-kafka, you must remove this package before installing confluent-server. You may need to run yum autoremove or apt autoremove to remove packages installed as dependencies by a previous install.

Red Hat Enterprise Linux

If your cluster was installed on Red Hat Enterprise Linux (RHEL), you can migrate from confluent-kafka to confluent-server or from confluent-server to confluent-kafka.

Migrating from confluent-kafka to confluent-server

Complete the following steps to migrate to confluent-server when using Red Hat Enterprise Linux (RHEL).

  1. Install your cluster using Ansible Playbooks for Confluent Platform. This installs confluent-kafka by default. For more information, see Deploy with Ansible Playbooks.

  2. Start your cluster and verify that it is working as expected.

  3. Log in and stop each Kafka broker host.

    sudo systemctl stop confluent-kafka
    
  4. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/server.properties /tmp
    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
    

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
    
  5. Use yum swap to change to confluent-server:

    yum swap confluent-kafka-2.12 confluent-server
    
  6. Copy your Kafka broker configuration backup to the broker configuration directory:

    sudo cp /tmp/server.properties /etc/kafka/
    
  7. Copy your override file using the following command:

    sudo cp /tmp/override.con /etc/systemd/system/confluent-server.service.d/
    

    Note

    If you are running Kerberos, complete the next step. If not, skip the next step.

  8. Copy your JAAS file to the broker configuration directory:

    sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/
    
  9. Start your Kafka brokers:

    sudo systemctl start confluent-server
    

Migrating from confluent-server to confluent-kafka

Complete the following steps to change from confluent-server back to confluent-kafka when using Red Hat Enterprise Linux (RHEL).

  1. Log in to each Kafka broker host and stop each broker:

    sudo systemctl stop confluent-server
    
  2. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/server.properties /tmp
    sudo cp /etc/systemd/system/confluent-server.service.d/override.conf /tmp
    

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
    
  3. Use yum swap to change to confluent-kafka:

    yum swap confluent-kafka-2.12 confluent-kafka
    
  4. Copy your configuration files to the broker configuration directory:

    sudo cp /tmp/server.properties /etc/kafka/
    

    Note

    If you are running Kerberos, complete the next step. If not, skip the next step.

  5. Copy your JAAS file to the broker configuration directory:

    sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/
    
  6. Start your Kafka brokers:

    sudo systemctl start confluent-kafka
    

Ubuntu

If your cluster was installed on Ubuntu, you can migrate from confluent-kafka to confluent-server or from confluent-server to confluent-kafka.

Migrating from confluent-kafka to confluent-server

Complete the following steps to migrate to confluent-server when using Ubuntu.

  1. Install your cluster using Ansible Playbooks for Confluent Platform. This installs confluent-kafka by default. For more information, see Deploy with Ansible Playbooks.

  2. Start your cluster and verify that it is working as expected.

  3. Log in and stop each Kafka broker host.

    sudo systemctl stop confluent-kafka
    
  4. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/server.properties /tmp
    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
    

    If running with Kerberos, you must back up your JAAS file:

    sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
    
  5. Remove confluent-kafka from your Kafka broker hosts.

    sudo apt-get remove confluent-kafka-2.12
    
  6. Install confluent-server on your Kafka broker hosts.

    sudo apt-get install confluent-server
    
  7. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

    Note

    If you are running Kerberos, complete the next step. If not, skip the next step.

  8. Copy your override file using the following command:

    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf
    /etc/systemd/system/confluent-server.service.d
    
  9. Start your Kafka brokers:

    sudo systemctl start confluent-server
    

Migrating from confluent-server to confluent-kafka

Complete the following steps to change from confluent-server back to confluent-kafka when using Ubuntu.

  1. Log in and stop each Kafka broker host.

    sudo systemctl stop confluent-server
    
  2. As a precaution, back up the following configuration files:

    sudo cp /etc/kafka/server.properties /tmp
    sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
    

    If running with Kerberos, you need to back up your JAAS file:

    sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
    
  3. Remove confluent-server from your Kafka broker hosts.

    sudo apt-get remove confluent-server
    
  4. Install confluent-kafka on your Kafka broker hosts.

    sudo apt-get install confluent-kafka-2.12
    
  5. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

  6. Start your Kafka brokers.

    sudo systemctl start confluent-kafka