Plan Capacity and Sizing
Plan capacity and sizing for Confluent Platform components deployed with Confluent for Kubernetes by reviewing the sizing guidelines and recommendations in this section before creating your Confluent Platform cluster.
Important
Starting with Confluent Platform version 8.0, ZooKeeper is no longer part of Confluent Platform.
The following table provides guidance on minimum cluster sizing:
Important
Starting with Confluent Platform version 8.0, Confluent Control Center (Legacy) is no longer supported with Confluent Platform. Use Control Center with Confluent Platform 8.0 and later.
Cluster type |
Production |
Minimum |
||||||
|---|---|---|---|---|---|---|---|---|
Resource |
Pods |
CPU |
Memory |
Disk |
Pods |
CPU |
Memory |
Disk |
ZooKeeper |
5 |
4 |
4 GB |
100 GB |
3 |
2 |
4 GB |
100 GB |
KRaft Controllers |
5 |
5 |
4 GB |
64 GB; use SSD |
3 |
3 |
4 GB |
64 GB; use SSD |
Kafka Brokers |
3 |
24 |
64 GB |
12 TB |
3 |
4 |
16 GB |
1 TB |
Connect Workers |
2 |
12 |
24 GB |
50 GB [1] |
2 |
4 |
16 GB |
50 GB [1] |
Schema Registry |
2 |
2 |
4 GB |
N/A [2] |
2 |
2 |
4 GB |
N/A [2] |
Control Center [3] |
1 |
4 |
8 GB |
200 GB; preferably SSD |
1 |
4 |
8 GB |
200 GB; preferably SSD |
Confluent Control Center (Legacy) |
1 |
12 |
32 GB |
300 GB; preferably SSD |
1 |
4 |
16 GB |
250 GB; preferably SSD |
ksqlDB |
2 |
4 |
32 GB |
100 GB; use SSD [4] |
2 |
4 |
20 GB |
100 GB; use SSD [4] |
Confluent REST Proxy |
2 |
16 |
1 GB+ [5] |
N/A [2] |
2 |
16 |
1 GB+ [5] |
N/A [2] |
[1] The disk storage requirement for Connect workers depends on how many Connect plugins you download per Connect cluster.
[2] Only required for installation.
[3] Set the resource requirement for Control Center at the pod level. It isn’t required to set container level resources, specifically for Prometheus and Alertmanager.
[4] The disk storage requirement for ksqlDB depends on the number of concurrent queries and the aggregation performed.
[5] 1 GB overhead plus 64 MB per producer and 16 MB per consumer.
Additional considerations for Confluent Platform
Keep the following additional considerations in mind when you size and deploy Confluent Platform:
Avoid placing multiple replicas of the same component, such as KRaft, Kafka, and ZooKeeper, on a single Kubernetes node.
A fully functioning Confluent Platform deployment requires at least three Kafka brokers. A one- or two-broker configuration isn’t supported for development, testing, or production deployments.
For more comprehensive, workload-based system requirements for Control Center, see Control Center System Requirements.
Considerations for Kubernetes worker nodes
The number of Kubernetes worker nodes that your cluster requires depends on whether you deploy a production cluster or a development testing cluster.
- Production cluster
Review the default capacity values in the Confluent Platform component custom resources (CRs). Determine how these values affect your production application and build out your nodes accordingly.
You can also use the on-premises System Requirements to determine what is required for your cloud production environment. The on-premises storage information provided doesn’t apply to cloud environments.
- Development testing cluster
Each node should typically have a minimum of two or four CPUs and 7 to 16 GB RAM. If you test a deployment of CFK and all Confluent Platform components, you can create a 10-node cluster with six nodes for Apache ZooKeeper™ and Apache Kafka® pods (three replicas each) and four nodes for all other components’ pods.
Confluent Platform component cluster sizing and resource requirements
In CFK, you specify resource requirements using the limits and
requests properties for custom resources (CRs). CFK configures the Confluent Platform
pods with the following default CPU and memory resources:
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 256Mi
The above default resource values are sufficient for most use cases. If you plan to manage a high number of day-2 application custom resources (CRs), such as 100 or more, refer to manage resources about increasing the number of application CRs and the corresponding memory and CPU.