Migrate from Kafka or Confluent Platform Community Edition¶
If you are already running Apache Kafka® or the community features that Confluent offers, and want to use some of Confluent’s enterprise services, you can easily migrate from Kafka or Confluent Platform Community Edition to Confluent Server.
The next sections show you the steps to migrate to Confluent Server.
Note that Confluent Server requires that the
confluent.license property be set to a valid license string in each broker properties file,
and this license is checked at broker start up. For more information, see Manage Confluent Platform Licenses.
Migrate an Existing Kafka Deployment to Confluent Platform¶
This section describes migrating from open-source Kafka to Confluent Platform. You should start the migration by assessing your current Kafka deployment; the version of Kafka, and Kafka components and configuration files you are currently using.
The version of Kafka in Confluent Platform is fully compatible with the matching open source version and only contains additional patches for critical bugs when Confluent Platform and Kafka release schedules do not align. For version compatibility, see Confluent Platform and Apache Kafka compatibility. You should migrate to a compatible version, and if you want to upgrade, do so after you migrate.
An existing cluster can typically be upgraded easily by performing a rolling restart of Kafka brokers.
Download the Confluent Platform packages¶
Download and install the Confluent Platform archive that contains the entire platform. The next steps describe downloading a TAR or ZIP archive. For all of the installation options, see Install Confluent Platform On-Premises.
Download the correct version of the Confluent Platform TAR or ZIP archive from https://www.confluent.io/download/.
Extract the contents of the archive into a new Confluent Platform install directory. For ZIP files, use a GUI to extract the contents or run this command in a terminal:
For TAR files run this command:
tar xzf confluent-7.6.0.tar.gz
Copy all configuration files from the Kafka
./configdirectory into the
./etc/kafkafolder under the new Confluent Platform directory you created in the previous step.
Stop all Kafka services running in the Kafka directory. This might include
Start the corresponding services in the Confluent Platform directory, and in addition start any Confluent enterprise services you want to use, for example
confluent-control-center. Similar to Kafka, the start and stop scripts for all services are found in the
Repeat these steps on each server, one server at a time, to perform rolling migration.
If at any time you want to move back to Kafka, simply stop the services in the Confluent Platform installation directory and start them in the Apache Kafka® directory.
Add additional tools and features¶
Remaining Confluent Platform services can be added incrementally. For example, start by adding Schema Registry and updating your applications to use the Avro serializer. You can also add the REST Proxy to support applications that may not have access to good Kafka clients or Avro libraries. Run the HDFS connector to load data from Kafka into HDFS continuously.
For an overview of all the features that Confluent Platform provides, see Confluent Platform Features.
Migrate from Confluent Community to full Confluent Platform¶
This section covers migrating from Confluent Community to full Confluent Platform.
Download new packages¶
The first step of migration is to install additional components, or download new Confluent Platform packages.
DEB Packages via apt or RPM Packages via yum¶
If you installed Confluent Platform using only Confluent Community components with yum or apt-get, you can use a yum command to install additional enterprise components. This is because the Confluent Platform packages that contain everything in Confluent Platform using only Confluent Community components plus additional enterprise packages.
Install the additional enterprise packages in your existing community deployment by running:
sudo yum install confluent-platform
sudo apt-get install confluent-platform
TAR or ZIP archives¶
If you installed Confluent Platform using only Confluent Community components from TAR or ZIP archives, you must download and install a new Confluent Platform archive that contains the entire platform.
- Download the Confluent Platform TAR or ZIP archive from https://www.confluent.io/download/.
- Extract the contents of the archive into a new Confluent Platform install directory. For ZIP files, use a GUI to extract the contents or run this command in a terminal:
tar xzf confluent-7.6.0.tar.gz
Finish the migration¶
Use the following steps to finish the migration process.
- Copy all configuration files from
./etc/confluent-control-centerinto the new directory you created in previous step.
- Stop all Apache Kafka® services running in the Confluent Platform using only Confluent Community components directory. Depending on what was running, this will include
- Start the corresponding services in the new Confluent Platform directory, and in addition start any new Enterprise services you want to use - for example
- Repeat these steps on all Confluent servers, one server at a time, to perform rolling migration.
- If at any time you want to move back to Confluent Platform using only Confluent Community components, simply stop the services in the Confluent Platform installation directory and start them in the Confluent Platform using only Confluent Community components directory.