.. _installation: Installation ============ The Confluent Platform is available in a variety of formats. Ensure you have the `Requirements`_, then select one of these options: * :ref:`zip and tar archives ` -- Recommended for OS X and the :ref:`Quickstart` * :ref:`deb packages via apt ` -- Recommended for deploying services on Debian/Ubuntu * :ref:`rpm packages via yum ` -- Recommended for deploying services on RHEL/CentOS/Fedora * :ref:`deb/rpm packages with installer script ` If you're an application developer, you can include Confluent dependencies in your project via a repository: * :ref:`jars via maven repository ` -- Recommended for application developers Confluent does not currently support Windows. Windows users can download and use the zip and tar archives, but will have to run the jar files directly rather than use the wrapper scripts in the ``bin/`` directory. Requirements ------------ The only requirement is Oracle Java >= 1.7. Java installation varies by platform, so please check the JRE installation requirements for your platform. Before installing the Confluent Platform, double check your Java version: .. sourcecode:: bash $ java -version Various installation options are listed in the sections below. Select the approach you prefer and follow the given instructions to install the Confluent Platform. .. _installation_archive: ZIP and TAR archives -------------------- These archives contain the jars, driver scripts, and configuration files in plain zip and tar archives. These should be used on OS X and as a simple way to evaluate the platform. Start by downloading one of the archives. The complete list of downloads can be found on http://confluent.io/downloads/. For example, OS X users may choose http://packages.confluent.io/archive/3.0/confluent-3.0.0-2.11.tar.gz. After downloading, you can verify the integrity of the package by checking the the SHA-1 or SHA-256 signature of the package against the ``.sha1.txt`` or ``.sha256.txt`` provided in the same directory as the package. Next extract the contents of the archive. For zip files, use a GUI to extract the contents or run this command in a terminal: .. sourcecode:: bash $ unzip confluent-3.0.0-2.11.zip For tar files run this command: .. sourcecode:: bash $ tar xzf confluent-3.0.0-2.11.tar.gz Next, try the :ref:`Quickstart` to test out the Confluent Platform locally. .. _installation_apt: DEB packages via apt -------------------- The apt repositories provide packages for Debian-based Linux distributions such as Debian and Ubuntu. First install Confluent's public key, which is used to sign the packages in the apt repository. .. sourcecode:: bash $ wget -qO - http://packages.confluent.io/deb/3.0/archive.key | sudo apt-key add - Add the repository to your ``/etc/apt/sources.list``: .. sourcecode:: bash $ sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.0 stable main" Run apt-get update and install the Confluent Platform: .. sourcecode:: bash $ sudo apt-get update && sudo apt-get install confluent-platform-2.11 The number at the end of the package name specifies the Scala version. Currently supported versions are 2.11 (recommended) and 2.10. Individual components of the Confluent Platform are also available as standalone packages. See the `Available Packages`_ section for a listing of packages. Next, try the :ref:`Quickstart` to test out the Confluent Platform locally. .. _installation_yum: RPM packages via yum -------------------- The yum repositories provide packages for RHEL, CentOS, and Fedora-based distributions. First install Confluent's public key, which is used to sign packages in the yum repository. .. sourcecode:: bash $ sudo rpm --import http://packages.confluent.io/rpm/3.0/archive.key Add the repository to your ``/etc/yum.repos.d/`` directory in a file named ``confluent.repo``. If you are using RHEL/Centos/Oracle 6 .. sourcecode:: ini [Confluent.dist] name=Confluent repository (dist) baseurl=http://packages.confluent.io/rpm/3.0/6 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.0/archive.key enabled=1 [Confluent] name=Confluent repository baseurl=http://packages.confluent.io/rpm/3.0 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.0/archive.key enabled=1 If you are using RHEL/Centos/Oracle 7 .. sourcecode:: ini [Confluent.dist] name=Confluent repository (dist) baseurl=http://packages.confluent.io/rpm/3.0/7 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.0/archive.key enabled=1 [Confluent] name=Confluent repository baseurl=http://packages.confluent.io/rpm/3.0 gpgcheck=1 gpgkey=http://packages.confluent.io/rpm/3.0/archive.key enabled=1 It is recommended to `clear the yum caches `_ before proceeding: .. sourcecode:: bash $ sudo yum clean all The repository is now ready for use. You can install the entire Confluent Platform with: .. sourcecode:: bash $ sudo yum install confluent-platform-2.11 The number at the end of the package name specifies the Scala version. Currently supported versions are 2.11 (recommended) and 2.10. Individual components of the Confluent Platform are also available as standalone packages. See the `Available Packages`_ section for a listing of packages. Next, try the :ref:`Quickstart` to test out the Confluent Platform locally. .. _installation_package_installer: DEB/RPM packages with installer script -------------------------------------- Debian and RPM packages are also provided in self-contained archives with installers. These archives are available for Debian packages: .. sourcecode:: bash $ wget http://packages.confluent.io/archive/3.0/confluent-3.0.0-2.11-deb.tar.gz and RPM packages: .. sourcecode:: bash $ wget http://packages.confluent.io/archive/3.0/confluent-3.0.0-2.11-rpm.tar.gz Before running the installer script, install Confluent's public key for Debian-based systems: .. sourcecode:: bash $ wget -qO - http://packages.confluent.io/deb/3.0/archive.key | sudo apt-key add - or for RPM-based systems: .. sourcecode:: bash $ sudo rpm --import http://packages.confluent.io/rpm/3.0/archive.key Next extract the contents and run the installer, demonstrated here with the Debian archive: .. sourcecode:: bash $ tar xzf confluent-3.0.0-2.11-deb.tar.gz $ cd confluent-3.0.0 $ ./install.sh The script also provide an uninstall option: .. sourcecode:: bash $ ./install.sh --uninstall This only removes the contents of the package. Any data generated by running the services will be left in place. Next, try the :ref:`Quickstart` to test out the Confluent Platform locally. Available Packages ------------------ Confluent Platform ships with two groups of packages: 1. The *platform packages* of CP (``confluent-platform-``), which are umbrella packages to install the complete Confluent Platform, i.e. all its individual packages. See next point. 2. The *individual packages* of the Confluent Platform such as ``confluent-kafka-`` and ``confluent-schema-registry``. Here, the platform packages (``confluent-platform-``) as well as the individual Kafka packages (``confluent-kafka-``) -- and only these -- are available in two variants, named by the respective Scala version of Apache Kafka that is included in the packages. .. note:: **Why Scala, and why different Scala versions?** Apache Kafka is implemented in Scala and built for multiple versions of Scala. However, the Scala version only matters if you are actually using Scala for implementing your applications and you want a Kafka version built for the same Scala version you use. Otherwise any version should work, with 2.11 being recommended. In CP 3.0.0, ``scala_version`` can be one of: * **2.11 (recommended)** -- giving ``confluent-platform-2.11`` and ``confluent-kafka-2.11`` * 2.10 -- giving ``confluent-platform-2.10`` and ``confluent-kafka-2.10`` If you choose to install via an apt or yum repository, you can install individual components instead of the complete Confluent Platform. Use these packages when you only need one component on your server, e.g. on production servers. The following are packages corresponding to each of the Confluent Platform components: * ``confluent-platform-`` -- umbrella package that installs the complete Confluent Platform, i.e. all the individual packages mentioned below * ``confluent-platform-oss-`` -- umbrella package that installs only the open source Confluent Platform, i.e. all the individual packages mentioned below except confluent-control-center and confluent-support-metrics. * ``confluent-kafka-`` -- Apache Kafka, including Kafka Streams, Kafka Connect framework, and Kafka Java client * ``confluent-control-center`` -- GUI-based system for managing and monitoring Apache Kafka * ``confluent-kafka-connect-hdfs`` -- Hadoop HDFS Connector for Kafka Connect * ``confluent-kafka-connect-jdbc`` -- JDBC Connector for Kafka Connect * ``confluent-schema-registry`` -- Schema Registry * ``confluent-kafka-rest`` -- HTTP REST Proxy for Apache Kafka * ``confluent-support-metrics`` -- Part of "Proactive Support" - improves Confluent's support for the platform by collecting and reporting support metrics ("Metrics") to Confluent * ``confluent-camus`` -- Camus (deprecated) * ``librdkafka`` -- C/C++ Kafka client * DEB packages * ``librdkafka1`` -- library * ``librdkafka-dev`` -- development headers and library * ``librdkafka1-dbg`` -- debugging symbols * RPM packages * ``librdkafka1`` -- library * ``librdkafka-devel`` -- development headers and library * ``librdkafka1-debuginfo`` -- debugging symbols * ``librdkafka`` -- source rpm * Source packages - included in zip and tar archives under `src/` directory * ``avro`` -- C/C++ Avro serialization library * DEB packages * ``libavro-c1`` -- C library and source * ``libavro-cpp1`` -- C++ library and source * ``libavro-c-dev`` -- C development headers and library * ``libavro-cpp-dev`` -- C++ development headers and library * RPM packages * ``avro-c`` -- C library and source * ``avro-cpp`` -- C++ library and source * ``avro-c-tools`` -- command line tools * ``avro-c-devel`` -- C development headers and library * ``avro-cpp-devel`` -- C++ development headers and library * ``avro-c-debuginfo`` -- C debugging symbols * ``avro-cpp-debuginfo`` -- C++ debugging symbols * Source packages - included in zip and tar archives under `src/` directory * ``confluent-libserdes`` -- C/C++ Avro Serialization with Schema Registry support * DEB packages * ``confluent-libserdes-c1`` -- library and source * ``confluent-libserdes-dev`` -- development headers and library * RPM packages * ``confluent-libserdes`` -- library and source * ``confluent-libserdes-devel`` -- development headers and library * ``confluent-libserdes-debuginfo`` -- debugging symbols * Source packages - included in zip and tar archives under `src/` directory * ``confluent-kafka-python`` -- Python client library * Python package -- available on `PyPI `_ * Source packages -- included in zip and tar archives under `src/` directory The following packages may also be installed automatically as dependencies: * ``confluent-common`` * ``confluent-rest-utils`` .. _installation_maven: Maven repository for jars ------------------------- All jars included in the packages are also available in the Confluent maven repository. Here's a sample POM file showing how to add this repository: .. sourcecode:: xml confluent http://packages.confluent.io/maven/ The Confluent maven repository includes compiled versions of Kafka. To reference the Kafka version 0.10.0.0 that is included with CP 3.0.0, use the following in your ``pom.xml``: .. sourcecode:: xml org.apache.kafka kafka_2.11 0.10.0.0-cp1 .. note:: **Version names of Kafka in Apache vs. Kafka in Confluent Platform:** Confluent always contributes patches back to the Apache Kafka open source project. However, the exact versions (and version names) being included in Confluent Platform may differ from the Apache artifacts when Confluent Platform and Apache Kafka releases do not align. In the case they do differ, we keep the ``groupId`` and ``artifactId`` identical but append the suffix ``-cpX`` to the version identifier of the CP version (with ``X`` being a digit) in order to distinguish these from the Apache artifacts. For example, to reference the Kafka version 0.9.0.1 that is included with Confluent Platform 2.0.1, you would use the following snippet in your ``pom.xml``: .. sourcecode:: xml org.apache.kafka kafka_2.11 0.9.0.1-cp1 Further artifacts are available. For example, to use Confluent's serializers that integrate with the rest of the Confluent Platform you would include the following in your ``pom.xml``: .. sourcecode:: xml io.confluent kafka-avro-serializer 3.0.0