.. _migrate-confluent-server:

Migrate to |cs|
===============

.. include:: includes/confluent-server-intro.rst

.. note::
  If you have already installed some packages using ``confluent-kafka``, you must remove this package
  before installing ``confluent-server``. You may need to run ``yum autoremove`` or ``apt autoremove``
  to remove packages installed as dependencies by a previous install.

You can migrate from ``confluent-kafka`` to ``confluent-server`` using the
instructions in the following sections.

.. include:: includes/installing-cp.rst
     :start-after: tip_for_installation
     :end-before: tip-for-available-packages-start

.. include:: includes/cs-license.rst

Red Hat Enterprise Linux
^^^^^^^^^^^^^^^^^^^^^^^^

Complete the following steps to migrate to ``confluent-server`` when using Red
Hat Enterprise Linux (RHEL).

#. Install your cluster using cp-ansible. This installs ``confluent-kafka`` by default.

#. Start your cluster and verify that it is working as expected.

#. Log in and stop each |ak| broker host.

   ::
      
      sudo systemctl stop confluent-kafka

#. As a precaution, back up the following configuration files:

   ::
      
      sudo cp /etc/kafka/server.properties /tmp
      sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
      
      
   If running with Kerberos, you must back up your JAAS file:
   
   ::
      
      sudo cp /etc/kafka/kafka_server_jaas.conf /tmp

#. Use ``yum swap`` to change to ``confluent-server``:

   ::
      
      yum swap confluent-kafka-2.12 confluent-server

#. Copy your |ak| broker configuration backup to the broker configuration directory:

   ::
      
      sudo cp /tmp/server.properties /etc/kafka/

#. Copy your override file using the following command:

   ::
      
      sudo cp /tmp/override.con /etc/systemd/system/confluent-server.service.d/
      
   .. note::
      
      If you are running Kerberos, complete the next step. If not, skip the next
      step.

#. Copy your JAAS file to the broker configuration directory:

   ::
      
      sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/

#. Start your |ak| brokers:

   ::
      
      sudo systemctl start confluent-server


Complete the following steps to change from ``confluent-server`` back to
``confluent-kafka``.

#. Log in to each |ak| broker host and stop each broker:

   ::
      
      sudo systemctl stop confluent-server

#. As a precaution, back up the following configuration files:

   ::
      
      sudo cp /etc/kafka/server.properties /tmp
      sudo cp /etc/systemd/system/confluent-server.service.d/override.conf /tmp
      
   If running with Kerberos, you must back up your JAAS file:
   
   ::
      
      sudo cp /etc/kafka/kafka_server_jaas.conf /tmp

#. Use ``yum swap`` to change to ``confluent-kafka``:

   ::
      
      yum swap confluent-kafka-2.12 confluent-kafka

#. Copy your configuration files to the broker configuration directory:

   ::
      
      sudo cp /tmp/server.properties /etc/kafka/
      
   .. note::
      
      If you are running Kerberos, complete the next step. If not, skip the next
      step.

#. Copy your JAAS file to the broker configuration directory:

   ::
      
      sudo cp /tmp/kafka_server_jaas.conf /etc/kafka/

#. Start your |ak| brokers:

   ::
      
      sudo systemctl start confluent-kafka

Ubuntu
^^^^^^

Complete the following steps to migrate to ``confluent-server`` when using Ubuntu.

#. Install your cluster using cp-ansible. This installs ``confluent-kafka`` by default.

#. Start your cluster and verify that it is working as expected.

#. Log in and stop each |ak| broker host.

   ::
      
      sudo systemctl stop confluent-kafka
      
#. As a precaution, back up the following configuration files:

   ::
      
      sudo cp /etc/kafka/server.properties /tmp
      sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
      
   If running with Kerberos, you must back up your JAAS file:
   
   ::
      
      sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
      

#. Remove ``confluent-kafka`` from your |ak| broker hosts.

   ::
      
      sudo apt-get remove confluent-kafka-2.12
      
#. Install ``confluent-server`` on your |ak| broker hosts.

   ::
      
      sudo apt-get install confluent-server
      
#. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

   .. note::
      
      If you are running Kerberos, complete the next step. If not, skip the next
      step.
      
#. Copy your override file using the following command:

   ::
      
      sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf
      /etc/systemd/system/confluent-server.service.d
      
#. Start your |ak| brokers:

   ::
      
      sudo systemctl start confluent-server
      
Complete the following steps to change from ``confluent-server`` back to
``confluent-kafka``.

#. Log in and stop each |ak| broker host.

   ::
      
      sudo systemctl stop confluent-server
      
#. As a precaution, back up the following configuration files:

   ::
      
      sudo cp /etc/kafka/server.properties /tmp
      sudo cp /etc/systemd/system/confluent-kafka.service.d/override.conf /tmp
      
   If running with Kerberos, you need to back up your JAAS file:
   
   ::
      
      sudo cp /etc/kafka/kafka_server_jaas.conf /tmp
      
#. Remove ``confluent-server`` from your |ak| broker hosts.

   ::
      
      sudo apt-get remove confluent-server
      
#. Install ``confluent-kafka`` on your |ak| broker hosts.

   ::
      
      sudo apt-get install confluent-kafka-2.12
      
#. Select the default option to keep your configuration files in place. Get the backup configuration files (that you made earlier) if you accidentally replace the files.

#. Start your |ak| brokers.

   ::
      
      sudo systemctl start confluent-kafka


Upgrade from 5.2.x to 5.3.x
^^^^^^^^^^^^^^^^^^^^^^^^^^^

If using confluent-kafka 5.2.x only
"""""""""""""""""""""""""""""""""""

If you are currently using ``confluent-kafka`` 5.2.x and are not using commercial features like LDAP authorizer,
you can upgrade ``confluent-kafka`` using the regular :ref:`upgrade` process.



If using confluent-kafka 5.2.x and confluent-security 5.2.x
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

.. important::
  Note that ``confluent-security`` 5.2.x is not compatible with ``confluent-kafka`` 5.3.x, so you must uninstall or upgrade ``confluent-security`` when upgrading ``confluent-kafka``.


If you are using the existing commercially-licensed :ref:`LDAP authorizer<kafka_ldap_authorizer>`
from ``confluent-security`` 5.2.x and want to upgrade to 5.3.x, you must install ``confluent-server``,
which contains the LDAP authorizer. You must first uninstall 5.2.x versions of ``confluent-kafka`` and
``confluent-security`` and then install the new ``confluent-server`` package. This can be done on one server
at a time for rolling upgrades.


**Debian packages via APT**

#. Backup all configuration files from ``/etc``, including, for example, ``/etc/kafka``.
#.  Stop the broker and remove the existing packages and their dependencies.

    .. codewithvars:: bash

        # stop the current kafka broker process
          sudo kafka-server-stop

        # uninstall existing 5.2.x confluent-kafka
          sudo apt-get remove confluent-kafka-|scala_version|

        # uninstall existing 5.2.x confluent-security
          sudo apt-get remove confluent-security

        # To remove Confluent Platform and all its dependencies at once, run the following after stopping all services
          sudo apt-get autoremove confluent-platform-|scala_version|

#. Remove the repository files of the previous version

   .. codewithvars:: bash

        sudo add-apt-repository -r "deb https://packages.confluent.io/deb/5.2 stable main"

#. Add the |version| repository to ``/etc/apt/sources.list``.

   .. codewithvars:: bash

       sudo add-apt-repository "deb https://packages.confluent.io/deb/|version| stable main"

#. Refresh repository metadata.

   .. sourcecode:: bash

        sudo apt-get update

#. Install ``confluent-server``. (Note that if you modified the configuration files, apt will prompt you to resolve the conflicts. You should keep your original configuration.)

   .. sourcecode:: bash

        sudo apt-get install confluent-server

#. Start the |ak| broker. If no additional features are being used (e.g., RBAC), no changes are required for the existing ``server.properties`` file.

   .. sourcecode:: bash

        kafka-server-start -daemon /etc/kafka/server.properties


**RPM packages via Yum**

#. Backup all configuration files from ``/etc``, including ``/etc/kafka``.
#.  Stop the broker and remove the existing packages and their dependencies.

    .. codewithvars:: bash

        # stop the current kafka broker process
          sudo kafka-server-stop

        # uninstall existing 5.2.x confluent-kafka
          sudo yum remove confluent-kafka-|scala_version|

        # uninstall existing 5.2.x confluent-security
          sudo yum remove confluent-security

        # To remove Confluent Platform and all its dependencies at once, run the following after stopping all services
          sudo yum autoremove confluent-platform-|scala_version|

#. Remove the repository files of the previous version

   .. sourcecode:: bash

        sudo rm /etc/yum.repos.d/confluent.repo

#. Add the |version| repository to directory in a file named :litwithvars:`confluent-|version|.repo`.

   .. codewithvars:: ini

        [confluent-|version|]
        name=Confluent repository for |version|.x packages
        baseurl=https://packages.confluent.io/rpm/|version|
        gpgcheck=1
        gpgkey=https://packages.confluent.io/rpm/|version|/archive.key
        enabled=1


#. Refresh repository metadata.

   .. sourcecode:: bash

        sudo yum clean all

#. Install ``confluent-server``. Note that yum may override your existing configuration files, so you must restore them from backup after installing the packages.

   .. sourcecode:: bash

        sudo yum install confluent-server

#. Restore any updated configuration files and start |ak| broker. If no additional features are being used (e.g., RBAC), no changes are required for the existing ``server.properties`` file.

   .. sourcecode:: bash

        kafka-server-start -daemon /etc/kafka/server.properties

**TAR or ZIP archives**

Follow :ref:`upgrade` instructions to upgrade your broker using TAR or ZIP archives.
Use :litwithvars:`confluent-|release|-|scala_version|.tar.gz` or the zip archive :litwithvars:`confluent-|release|-|scala_version|.zip`
to install |ak| when using commercial features like :ref:`LDAP authorizer<kafka_ldap_authorizer>`.


Upgrade confluent-kafka and Remove confluent-security
"""""""""""""""""""""""""""""""""""""""""""""""""""""

If you are currently using ``confluent-kafka`` with :ref:`LDAP authorizer<kafka_ldap_authorizer>`
from ``confluent-security`` 5.2.x, but do not intend to upgrade to the commercially licensed ``confluent-server``
package, you can upgrade ``confluent-kafka`` and use it without the LDAP authorizer. You can use the built-in
``SimpleAclAuthorizer`` for authorization using ACLs as described in :ref:`kafka_authorization`. You must replace
group-based ACLs with equivalent user-based ACLs to ensure that all your existing authorization rules are applied.
Note that you must remove or upgrade ``confluent-security`` when you upgrade ``confluent-kafka``.