Confluent System Requirements¶
The following machine recommendations are for installing individual Confluent Platform components:
|Control Center||1||300 GB, preferably SSDs||32 GB RAM (JVM default 6 GB)||12 cores or more|
||64 GB RAM||Dual 12 core sockets|
|Connect||2||Only required for installation||0.5 - 4 GB heap size depending on connectors||Typically not CPU- bound. More cores is better than faster cores.|
|Replicator||Same as Connect for nodes, storage, memory, and CPU. (See note below regarding AWS.)|
|ksqlDB||2||Use SSD. Sizing depends on the number of concurrent queries and the aggregation performed.||20 GB RAM||4 cores|
|REST Proxy||2||Only required for installation||1 GB overhead plus 64 MB per producer and 16 MB per consumer||16 cores to handle HTTP requests in parallel and background threads for consumers and producers.|
|Schema Registry||2||Only required for installation||1 GB heap size||Typically not CPU- bound. More cores is better than faster cores.|
Each write to ZooKeeper must be persisted in the transaction log before the client gets an ack. Using SSD reduces the ZooKeeper write latency.
|4 GB RAM||2-4 cores|
In the above table, the recommended CPU resource is in CPU units which are the same in all platforms. For example, if we recommend 12 CPUs for non-Kubernetes environment, the recommendation for Kubernetes environment would be 12 CPU units, as well.
If deploying Confluent Platform on AWS VMs and running Replicator as a connector, be aware that VMs with burstable CPU types (T2, T3, T3a, and T4g) will not support high throughput streaming workloads. Replicator worker nodes running on these VMs experience throughput degradation due to credits expiring, making these VMs unsuitable for Confluent Platform nodes expected to run at elevated CPU levels for a sustained period of time, and supporting workloads that are above and beyond their baseline resource rates.
|RHEL/CentOS 6.x (deprecated)||no||no||no||yes||yes||yes||yes||yes||yes||yes||yes||yes|
|Ubuntu 14.04 LTS (deprecated)||no||no||no||yes||yes||yes||yes||yes||yes||yes||yes||yes|
|Ubuntu 16.04 LTS||yes||yes||yes||yes||yes||yes||yes||yes||yes||yes||yes||yes|
|Ubuntu 18.04 LTS||yes||yes||yes||yes||yes||yes||no||no||no||no||no||no|
Windows is not currently supported for Confluent Platform.
- macOS 10.14 and later is supported for testing and development purposes only.
- File descriptors
- For the file descriptor requirement for Kafka, see File Descriptors and mmap.
Control Center requires many open RocksDB files. Set the
ulimitfor the number of open files to a minimum value of 16384 using the
For the other Confluent Platform components, specifically Schema Registry and Replicator, you can leave the
ulimitas the OS default.
- RHEL with TLS
When installing Confluent Platform on RHEL8 with TLS encryption, you must add DH Key Size JVM Parameters for each component. For more information, see Strong crypto defaults in RHEL 8 and deprecation of weak crypto algorithms.
Set the following component-level environment variables to the argument:
- Control Center:
- Schema Registry:
- Kafka, ZooKeeper, and Connect:
- REST Proxy:
- Control Center:
Java 8 and Java 11 are supported in this version of Confluent Platform. From a security perspective, we recommend the latest released patch version as older freely available versions have disclosed security vulnerabilities.
Java 9 and 10 are not supported.
For more information, see Java supported versions.
You need to separately install the correct version of Java before you start the Confluent Platform installation process.
Control Center relies heavily on Kafka, so a fast and reliable network is important for performance. Modern datacenter networking speed of 1 GbE, 10 GbE should be sufficient.
The table below lists the network services and ports exposed as part of Confluent Platform.
All services listed below use the TCP protocol.
All ports listed below are the default ports, and in most cases you can configure each service to listen on a different port of your choice.
Ports that are indicated to be Internal Only need to be accessible by components within Confluent Platform, not by users or clients of the platform.
When deploying Confluent Platform, ensure that your networking rules allow for the required access to the various components and services.
|Component and Service||Default Port||Internal Only?|
|(Standalone) REST Proxy||8082||No|
|Confluent Control Center||9021||No|
[*] Reserve the Jolokia ports only when you deploy Confluent Platform using Ansible.
Clock synchronization on each Kafka broker is required to ensure the full system functions
correctly. For example, secure network communication with the brokers that rely on
TLS certificate verification may fail if clocks are not synchronized. Clock synchronization
can be achieved using utilities like
ntpd that implement the Network Time Protocol (NTP).