Installing Control Center on Kafka

Control Center is included natively in Confluent Platform, but you can also use it with a cluster running Apache Kafka®.

Important

The version of Kafka that ships with Confluent Platform is for all intents and purposes identical to Kafka releases, with the possible exception of critical bug fixes. It is recommended that you use Control Center with Confluent Platform.

Software requirements

Kafka Brokers running Kafka Version 0.10.2.0 or later
Stream Monitoring requires several new features of Kafka 0.10.2.0 to function. The Confluent Kafka distribution included with Confluent Platform 6.2 is recommended.
Kafka Java Producers and Consumers running 0.10.1.0 or later
Stream Monitoring requires several new features of Kafka 0.10.1.0 to function, including cluster ids. These are currently only available in the Kafka 0.10.1.0 Java clients.
Ability to add jars to the classpath of client applications
You do not need to recompile client applications, and may not need to reboot them if you are using a class injection system like Java Beans or Spring.
Ability to configure clients
You must have the ability to change the configurations of Kafka clients so that you can specify parameters for the Confluent Metrics Interceptor. (If you write your applications to dynamically load Kafka configuration settings, you don’t need to recompile to use this feature. If you can dynamically change the parameters of running applications, then you do not even need to restart them.
Ability to configure Kafka brokers
You must have the ability to change the configurations of Kafka brokers so that you can specify parameters for the Confluent Metrics Reporter.

Control Center uses Kafka to collect data. You can configure Control Center to run with a replication factor of 1 (thus requiring at least one Kafka broker in your cluster), but a replication factor of 3 is recommended for production use (requires at least three brokers).

Hardware requirements

A server that meets these minimal requirements:

  • 32 GB RAM (JVM default 6 GB); 8 or more cores
  • 300 GB storage, preferably SSDs
  • A supported operating system:
    • Ubuntu 14.04 or later
    • Debian 8 or later
    • RHEL/Centos 6.8 or later
  • Access to Kafka (specifically, the ability to consume messages and to communicate with ZooKeeper)
  • Access to Kafka Connect instances (if you want to configure Kafka Connect)
  • Ability to connect to the server from the user’s web browser

Users who need access to Control Center must be able to access the host that runs the application. You can configure the network port that Control Center uses to serve data. Because Control Center is a web application, you can use a proxy to control and secure access to it.

Install Control Center

Installation on Red Hat, CentOS, or Fedora

Installation instructions for installing the overall platform on Red Hat-style systems can be found at Manual Install using Systemd on RHEL and CentOS.

If you would like to install just Control Center on a server, first follow the instructions for configuring Confluent repositories with your package manager. Then, you can install Control Center with a line like:

sudo yum install confluent-control-center

Installation on Debian Linux

Installation instructions for the overall platform on Debian style systems can be found at Manual Install using Systemd on Ubuntu and Debian.

If you would like to install just Control Center on a server, first follow the instructions for configuring Confluent repositories with your package manager. Then, you can install Control Center with a line like:

sudo apt-get install confluent-control-center

Install and configure components

To use Control Center, the following Confluent Platform component are required.

Installing Confluent Metrics Reporter with Kafka

  1. Verify that the Metrics Reporter is available in the Kafka broker classpath. If you installed via the archive it should be located at ./share/java/kafka/confluent-metrics-6.2.15.jar. If you installed via deb or rpm you will need to install the confluent-rebalancer package.
  2. Follow the instructions to enable the Confluent Metrics Reporter. Example configuration parameters are provided in ./etc/kafka/server.properties of the Confluent archive.

Installing Kafka Connect

See Getting Started with Kafka Connect for more information about how to install and configure Kafka Connect. (You need to install Kafka Connect for Control Center to configure it.)

Installing Confluent Metrics Clients with Connect

If you install Confluent Platform, then you already have all the software you need to monitor Connect jobs with Stream Monitoring.

If you have installed Kafka Connect separately, you can still use Stream Monitoring to monitor Kafka Connect. To do this, you need to take these steps:

  1. Upgrade to Kafka 0.10.2.0+
  2. Add the Confluent Metrics Interceptor to the Kafka Connect CLASSPATH environment variable.
  3. Add the Confluent Metrics Interceptor to the client configurations in the Kafka Connect config files. You can append two lines like this in your configuration file to do this:
consumer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor
producer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor

Start Control Center

You can use a script for starting Control Center, called control-center-start. This script takes one argument: a configuration file (control-center.properties) that specifies the properties for Control Center. For information on what properties you must specify, see Control Center Configuration Reference. After you have configured the properties file, you can start Control Center.

<path-to-confluent>/bin/control-center-start etc/confluent-control-center/control-center.properties

If you would like to start Control Center as a daemon process, you can use the -daemon option:

<path-to-confluent>/bin/control-center-start -daemon etc/confluent-control-center/control-center.properties

You can navigate to the Control Center web interface at http://localhost:9021/.

License Keys

Confluent Control Center is proprietary software. If you have downloaded and installed Control Center from the Confluent Web Site and are not a Confluent Enterprise subscription customer, you have already agreed to the license agreement. If you are a Confluent Enterprise subscription customer, then you are bound by the terms of the agreement between your company and Confluent.

If you are not a customer, your download includes a 30 day free trial for Control Center. Control Center will stop functioning 30 days after you install the software. Any Kafka Connect Connectors that you created will still function; you can use the Kafka Connect API to change them, but you cannot use Control Center to make changes to connectors. Stream Monitoring will continue to log data, but the web application will not continue to function. If the Stream Monitoring functionality has become disabled, and you have applied a Confluent license, please contact Confluent through the support portal for assistance.

If you would like to learn more about how to become a Confluent Platform Enterprise subscription customer, please visit confluent.io. Customers are issued enterprise license keys and all of the functions of Control Center will be unlocked for the term of your subscription.

Note

Trial and developer licenses are shipped with Confluent Platform. A developer license allows using Control Center and other Confluent Platform proprietary components indefinitely for single-broker development environments. For more information about Confluent Platform licenses, see Managing Confluent Platform Licenses.

Removing Control Center

If you would like to remove Control Center from your system, please follow these instructions.

TAR and ZIP Archives

Inside the archive, Control Center resources can be found in the following locations:

  • Java Archives are found in share/java/confluent-control-center and share/java/monitoring-interceptors
  • Configuration files are found in etc/confluent-control-center
  • Executables are in bin/ with prefix control-center-

You can remove this all with a command like:

rm -rf share/java/confluent-control-center \
      share/java/monitoring-interceptors \
      etc/confluent-control-center \
      bin/control-center-*

Debian packages

If you installed with apt-get as a superuser, you can remove Control Center with a command like:

sudo apt-get remove confluent-control-center

Otherwise, follow the directions for the package management tool that you used.

Red Hat packages

If you installed with yum as a superuser, you can remove Control Center with a command like:

sudo yum remove confluent-control-center

Otherwise, follow the directions for the package management tool that you used.