Important
You are viewing documentation for an older version of Confluent Platform. For the latest, click here.
System Requirements¶
Hardware¶
On-Premises¶
The following machine recommendations are for installing individual Confluent Platform components:
| Component | Nodes | Storage | Memory | CPU |
|---|---|---|---|---|
| Control Center | 1 | 300 GB, preferably SSDs | 32 GB RAM (JVM default 6 GB) | 12 cores or more |
| Broker | 3 |
|
64 GB RAM | Dual 12 core sockets |
| Connect | 2 | Only required for installation | 0.5 - 4 GB heap size depending on connectors | Typically not CPU- bound. More cores is better than faster cores. |
| ksqlDB | 2 | Use SSD. Sizing depends on the number of concurrent queries and the aggregation performed. | 20 GB RAM | 4 cores |
| REST Proxy | 2 | Only required for installation | 1 GB overhead plus 64 MB per producer and 16 MB per consumer | 16 cores to handle HTTP requests in parallel and background threads for consumers and producers. |
| Schema Registry | 2 | Only required for installation | 1 GB heap size | Typically not CPU- bound. More cores is better than faster cores. |
| ZooKeeper | 3-5 |
Each write to ZooKeeper must be persisted in the transaction log before the client gets an ack. Using SSD reduces the ZooKeeper write latency. |
4 GB RAM | 2-4 Cores |
Cloud¶
Software¶
Operating Systems¶
| Operating System | 5.5.x | 5.4.x | 5.3.x | 5.2.x | 5.1.x | 5.0.x | 4.1.x | 4.0.x | 3.3.x |
|---|---|---|---|---|---|---|---|---|---|
| RHEL/CentOS 6.x (deprecated) | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| RHEL/CentOS 7.x | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| Debian 8 | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| Debian 9 | yes | yes | yes | no | no | no | no | no | no |
| Ubuntu 14.04 LTS (deprecated) | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| Ubuntu 16.04 LTS | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| Ubuntu 18.04 LTS | yes | yes | yes | no | no | no | no | no | no |
- Windows is not currently supported for Confluent Platform. Windows users can download and use the ZIP and TAR archives, but must run the JAR files directly.
- Windows is supported by the C/C++ and .NET clients.
- macOS 10.13 and later is supported for testing and development purposes only.
- For the file descriptor requirement for Kafka, see File Descriptors and mmap.
- Control Center requires many open RocksDB files. Set the ulimit for the number of open files (
ulimit -n) to a minimum value of 16384. - For the other Confluent Platform components, specifically Schema Registry and Replicator, you can leave the ulimit as the OS default.
Java¶
Java 8 and Java 11 are supported in this version of Confluent Platform (Java 9 and 10 are not supported). For more information, see Java supported versions. You need to separately install the correct version of Java before you start the Confluent Platform installation process.
Docker¶
Optional: Docker version 1.11 or later running on a supported operating system. This is required if you are installing Confluent Platform by using the Docker images.
Network¶
Control Center relies heavily on Kafka, so a fast and reliable network is important for performance. Modern datacenter networking speed of 1 GbE, 10 GbE should be sufficient.
Ports¶
By default the Confluent Platform components use these ports to communicate. These ports must be open.
| Component | Port |
|---|---|
| Kafka brokers (plain text) | 9092 |
| Kafka inter-broker communication | 9091 |
| Confluent Control Center | 9021 |
| Kafka Connect REST API | 8083 |
| ksqlDB Server REST API | 8088 |
| Metadata Service (MDS) if enabled | 8090 |
| REST Proxy | 8082 |
| Schema Registry REST API | 8081 |
| ZooKeeper | 2181, 2888, 3888 |
Synchronize time¶
Clock synchronization on each Kafka broker is required to ensure the full system functions
correctly. For example, secure network communication with the brokers that rely on
TLS certificate verification may fail if clocks are not synchronized. Clock synchronization
can be achieved using utilities like ntpd that implement the Network Time Protocol (NTP).