This topic provides instructions for installing a production-ready Confluent Platform configuration in a multi-node environment
with a replicated ZooKeeper ensemble.
With this installation method, you connect to every node manually, download the archive, and run the Confluent Platform
Configure Confluent Platform
Configure Confluent Platform with the individual component properties files. By default these are located in
You must minimally configure the following components.
These instructions assume you are running ZooKeeper in replicated mode. A minimum of three servers are required for replicated
mode, and you must have an odd number of servers for failover. For more information, see the
Navigate to the ZooKeeper properties file (
/etc/kafka/zookeeper.properties) file and modify as shown.
This configuration is for a three node ensemble. This configuration file should be identical across all nodes in the ensemble.
clientPort are all set to typical single server values. The
govern how long following ZooKeeper servers can take to initialize with the current leader and how long they can be out of sync
with the leader. In this configuration, a follower can take 10000 ms to initialize and can be out of sync for up to 4000 ms
based on the
tickTime being set to 2000ms.
server.* properties set the ensemble membership. The format is
myid is the server identification number. There are three servers that each have a different
3 respectively. The
myid is set by creating a file named
myid in the
that contains a single integer in human readable ASCII text. This value must match one of the
myid values from the
configuration file. You will see an error if another ensemble member is already started with a conflicting
leaderport is used by followers to connect to the active leader. This port should be open between all ZooKeeper
electionport is used to perform leader elections between ensemble members. This port should be open between all ZooKeeper
autopurge.purgeInterval have been set to purge all but three snapshots every 24 hours.
Navigate to the ZooKeeper log directory (e.g.,
/var/lib/zookeeper/) and create a file named
consists of a single line that contains the machine ID in the format
<machine-id>. When the ZooKeeper server starts up, it knows
which server it is by referencing the
myid file. For example, server 1 will have a
myid value of
In a production environment, multiple brokers are required. During startup brokers register themselves in ZooKeeper to become
a member of the cluster.
Navigate to the Apache Kafka® properties file (
/etc/kafka/server.properties) and customize the following:
Connect to the same ZooKeeper ensemble by setting the
zookeeper.connect in all nodes to the same value. Replace all
localhost to the hostname or FQDN (fully qualified domain name) of your node. For example, if your
Configure the broker IDs for each node in your cluster using one of these methods.
Configure how other brokers and clients communicate with the broker using
listeners, and optionally
listeners: Comma-separated list of URIs and listener names to listen on.
advertised.listeners: Comma-separated list of URIs and listener names for other brokers and clients to use.
advertised.listeners parameter ensures that the broker advertises an address that is accessible from both
local and external hosts.
For more information, see Production Configuration Options.
Configure security for your environment.
Navigate to the Control Center properties file (
and customize the following:
# host/port pairs to use for establishing the initial connection to the Kafka cluster
# location for Control Center data
# the Confluent license
# ZooKeeper connection string with host and port of a ZooKeeper servers
This configuration is for a three node multi-node cluster. For more information, see Control Center configuration details. For information about Confluent Platform licenses, see Managing Confluent Platform Licenses.
Navigate to the Kafka server configuration file (
/etc/kafka/server.properties) and enable Confluent Metrics Reporter.
##################### Confluent Metrics Reporter #######################
# Confluent Control Center and Confluent Auto Data Balancer integration
# Uncomment the following lines to publish monitoring data for
# Confluent Control Center and Confluent Auto Data Balancer
# If you are using a dedicated metrics cluster, also adjust the settings
# to point to your metrics Kafka cluster.
# Uncomment the following line if the metrics cluster has a single broker
Add these lines to the Kafka Connect properties file (
/etc/kafka/connect-distributed.properties) to add support for the
# Interceptor setup
Navigate to the Schema Registry properties file (
and specify the following properties:
# Specify the address the socket server listens on, e.g. listeners = PLAINTEXT://your.host.name:9092
# The host name advertised in ZooKeeper. This must be specified if your running Schema Registry
# with multiple nodes.
# List of Kafka brokers to connect to, e.g. PLAINTEXT://hostname:9092,SSL://hostname2:9092
This configuration is for a three node multi-node cluster. For more information, see Running Schema Registry in Production.
Start Confluent Platform
Install Confluent Platform by using Kafka CLI commands.
ZooKeeper, Kafka, and Schema Registry must be started in this specific order, and must be started before any other components.
Start ZooKeeper. Run this command in its own terminal.
Start Kafka. Run this command in its own terminal.
Start Schema Registry. Run this command in its own terminal.
Start other Confluent Platform components as desired.
Confluent REST Proxy