System Requirements

Hardware

On-Premises

The following machine recommendations are for installing individual Confluent Platform components:

Component Nodes Storage Memory CPU
Control Center 1 300 GB, preferably SSDs 32 GB RAM (JVM default 6 GB) 8 cores or more
Broker 3
  • 12 X 1 TB disk. RAID 10 is optional
  • Separate OS disks from Apache Kafka® storage
64 GB RAM Dual 12 core sockets
Connect 2 Only required for installation 0.5 - 4 GB heap size depending on connectors Typically not CPU- bound. More cores is better than faster cores.
ksqlDB 2 Use SSD. Sizing depends on the number of concurrent queries and the aggregation performed. 20 GB RAM 4 cores
REST Proxy 2 Only required for installation 1 GB overhead plus 64 MB per producer and 16 MB per consumer 16 cores to handle HTTP requests in parallel and background threads for consumers and producers.
Schema Registry 2 Only required for installation 1 GB heap size Typically not CPU- bound. More cores is better than faster cores.
ZooKeeper 3-5
  • Transaction log: 512 GB SSD
  • Storage: 2 X 1 TB SATA, RAID 10

Each write to ZooKeeper must be persisted in the transaction log before the client gets an ack. Using SSD reduces the ZooKeeper write latency.

4 GB RAM 2-4 Cores

Cloud

Tip

If you don’t see your region listed, contact Confluent. Additional regions can be supported by request.

  • Amazon Web Services (AWS)
    • ap-northeast-1 (Tokyo)
    • ap-south-1 (Mumbai)
    • ap-southeast-1 (Singapore)
    • ap-southeast-2 (Sydney)
    • ca-central-1 (Canada Central)
    • eu-central-1 (Frankfurt)
    • eu-west-1 (Ireland)
    • eu-west-2 (London)
    • eu-west-3 (Paris)
    • sa-east-1 (São Paulo)
    • us-east-1 (N. Virginia)
    • us-east-2 (Ohio)
    • us-west-2 (Oregon)
  • Azure (Microsoft Azure)
    • australiaeast (New South Wales)
    • canadacentral (Canada)
    • centralus (Iowa)
    • eastus (Virginia)
    • eastus2 (Virginia)
    • francecentral (France)
    • northeurope (Ireland)
    • southeastasia (Singapore)
    • uksouth (London)
    • westus2 (Washington)
    • westeurope (Netherlands)
  • GCP (Google Cloud Platform)
    • asia-east1 (Taiwan)
    • asia-east2 (Hong Kong)
    • asia-northeast1 (Tokyo)
    • asia-south1 (Mumbai)
    • asia-southeast1 (Singapore)
    • asia-southeast-2 (Jakarta)
    • australia-southeast1 (Sydney)
    • europe-north1 (Finland)
    • europe-west1 (Belgium)
    • europe-west2 (London)
    • europe-west3 (Frankfurt)
    • europe-west4 (Netherlands)
    • northamerica-northeast1 (Montreal)
    • southamerica-east-1 (São Paulo)
    • us-central1 (Iowa)
    • us-east1 (S. Carolina)
    • us-east4 (N. Virginia)
    • us-west1 (Oregon)
    • us-west2 (Los Angeles)
    • us-west4 (Las Vegas)

Software

Operating Systems

Operating System 6.0.x 5.5.x 5.4.x 5.3.x 5.2.x 5.1.x 5.0.x 4.1.x 4.0.x 3.3.x
RHEL/CentOS 6.x (deprecated) no yes yes yes yes yes yes yes yes yes
RHEL/CentOS 7.x yes yes yes yes yes yes yes yes yes yes
RHEL/CentOS 8.x yes no no no no no no no no no
Debian 8 no yes yes yes yes yes yes yes yes yes
Debian 9 yes yes yes yes no no no no no no
Ubuntu 14.04 LTS (deprecated) no yes yes yes yes yes yes yes yes yes
Ubuntu 16.04 LTS yes yes yes yes yes yes yes yes yes yes
Ubuntu 18.04 LTS yes yes yes yes no no no no no no
Windows

Windows is not currently supported for Confluent Platform.

Windows 8.1 and later, as well as Windows 2016 and later, are supported by the C/C++ and .NET clients.

macOS
macOS 10.14 and later is supported for testing and development purposes only.
File descriptors
For the file descriptor requirement for Kafka, see File Descriptors and mmap.
ulimit

Control Center requires many open RocksDB files. Set the ulimit for the number of open files to a minimum value of 16384 using the ulimit -n command.

For the other Confluent Platform components, specifically Schema Registry and Replicator, you can leave the ulimit as the OS default.

RHEL with TLS

When installing Confluent Platform on RHEL8 with TLS encryption, you must add DH Key Size JVM Parameters for each component. For more information, see Strong crypto defaults in RHEL 8 and deprecation of weak crypto algorithms.

Set the following component-level environment variables to the argument:

  • Control Center: CONTROL_CENTER_OPTS=-Djdk.tls.ephemeralDHKeySize=2048
  • Schema Registry: SCHEMA_REGISTRY_OPTS=-Djdk.tls.ephemeralDHKeySize=2048
  • Kafka, ZooKeeper, and Connect: KAFKA_OPTS=-Djdk.tls.ephemeralDHKeySize=2048
  • REST Proxy: KAFKAREST_OPTS=-Djdk.tls.ephemeralDHKeySize=2048
  • ksqlDB: KSQL_OPTS=-Djdk.tls.ephemeralDHKeySize=2048

Java

Java 8 and Java 11 are supported in this version of Confluent Platform (Java 9 and 10 are not supported). For more information, see Java supported versions. You need to separately install the correct version of Java before you start the Confluent Platform installation process.

Docker

Optional: Docker version 1.11 or later running on a supported operating system. This is required if you are installing Confluent Platform by using the Docker images.

Network

Control Center relies heavily on Kafka, so a fast and reliable network is important for performance. Modern datacenter networking speed of 1 GbE, 10 GbE should be sufficient.

Ports used by Confluent Platform components

The table below enumerates the network services and ports exposed as part of Confluent Platform. Some components host just a single service (e.g. Schema Registry and its REST API) while others host multiple (e.g. Kafka broker and its multiple Kafka listeners, in addition to HTTP services like the Embedded REST Proxy).

All services listed below use the TCP protocol.

All ports listed below are the default ports, and in most cases you can configure each service to listen on a different port of your choice.

Ports that are indicated to be Internal Only need to be accessible by components within Confluent Platform, not by users or clients of the platform.

When deploying Confluent Platform, ensure that your networking rules allow for the required access to the various components and services.

Component and Service Default Port Internal Only?
ZooKeeper    
  • Peer-to-peer communication
2888 Yes
  • Peer-to-peer communication
3888 Yes
  • Client access
2181 No
Kafka Broker    
  • Inter-broker listener
9091 Yes
  • External listener
9092 No
  • Metadata Service (MDS)
8090 No
  • Confluent Server REST API
8090 No
(Standalone) REST Proxy 8082 No
Confluent Control Center 9021 No
Kafka Connect REST API 8083 No
ksqlDB Server REST API 8088 No
Schema Registry REST API 8081 No

Synchronize time

Clock synchronization on each Kafka broker is required to ensure the full system functions correctly. For example, secure network communication with the brokers that rely on TLS certificate verification may fail if clocks are not synchronized. Clock synchronization can be achieved using utilities like ntpd that implement the Network Time Protocol (NTP).