Plan Capacity and Sizing for Confluent for Kubernetes

A minimum Confluent Platform deployment with Confluent for Kubernetes requires at least three Kafka brokers and three KRaft controllers. Each broker needs four CPUs, 16 GB of memory, and 1 TB of disk. For production, use the larger values in the following table. Review the sizing guidelines and recommendations in this section before you create your Confluent Platform cluster.

Important

Starting with Confluent Platform version 8.0, ZooKeeper is no longer part of Confluent Platform.

The following table provides guidance on minimum cluster sizing:

Important

Starting with Confluent Platform version 8.0, Confluent Control Center (Legacy) is no longer supported with Confluent Platform. Use Control Center with Confluent Platform 8.0 and later.

Cluster type	Production				Minimum
Resource	Pods	CPU	Memory	Disk	Pods	CPU	Memory	Disk
ZooKeeper	5	4	4 GB	100 GB	3	2	4 GB	100 GB
KRaft Controllers	5	5	4 GB	64 GB; use SSD	3	3	4 GB	64 GB; use SSD
Kafka Brokers	3	24	64 GB	12 TB	3	4	16 GB	1 TB
Connect Workers	2	12	24 GB	50 GB ^[1]	2	4	16 GB	50 GB ^[1]
Schema Registry	2	2	4 GB	N/A ^[2]	2	2	4 GB	N/A ^[2]
Control Center ^[3]	1	4	8 GB	200 GB; preferably SSD	1	4	8 GB	200 GB; preferably SSD
Confluent Control Center (Legacy)	1	12	32 GB	300 GB; preferably SSD	1	4	16 GB	250 GB; preferably SSD
ksqlDB	2	4	32 GB	100 GB; use SSD ^[4]	2	4	20 GB	100 GB; use SSD ^[4]
Confluent REST Proxy	2	16	1 GB+ ^[5]	N/A ^[2]	2	16	1 GB+ ^[5]	N/A ^[2]

^[1] The disk storage requirement for Connect workers depends on how many Connect plugins you download per Connect cluster.
^[2] Only required for installation.
^[3] Set the resource requirement for Control Center at the pod level. It isn’t required to set container level resources, specifically for Prometheus and Alertmanager.
^[4] The disk storage requirement for ksqlDB depends on the number of concurrent queries and the aggregation performed.
^[5] 1 GB overhead plus 64 MB per producer and 16 MB per consumer.

Additional considerations for Confluent Platform

Keep the following additional considerations in mind when you size and deploy Confluent Platform:

Avoid placing multiple replicas of the same component, such as KRaft, Kafka, and ZooKeeper, on a single Kubernetes node.
A fully functioning Confluent Platform deployment requires at least three Kafka brokers. A one- or two-broker configuration isn’t supported for development, testing, or production deployments.
For more comprehensive, workload-based system requirements for Control Center, see Control Center System Requirements.

If you plan to enable Confluent Private Cloud Multi-Tenancy on a Physical Kafka Cluster (PKC), CFK derives and enforces the following per-broker capacity quotas from broker heap size. Size each broker’s heap accordingly, in addition to the preceding guidance.

In the table, heap is the broker’s JVM heap in GiB, the value that -Xmx and -Xms are set to, not the raw pod memory request or limit. CFK sets the heap to half of the pod memory, up to 32 GiB:

Per-broker capacity	Formula
Producer byte rate	`heap (GiB) × 6.25 MiB` per second
Consumer byte rate	Three times the producer byte rate
Maximum connections	`heap (GiB) × 4096`
Maximum connections per Logical Kafka Cluster (LKC)	`heap (GiB) × 1024`

For more information about capacity planning, see Plan capacity.

Kubernetes worker node sizing and minimum cluster size

The number of Kubernetes worker nodes that your cluster requires depends on whether you deploy a production cluster or a development testing cluster.

Production cluster

Review the default capacity values in the Confluent Platform component custom resources (CRs). Determine how these values affect your production application and build out your nodes accordingly.

You can also use the on-premises System Requirements to determine what is required for your cloud production environment. The on-premises storage information provided doesn’t apply to cloud environments.

Development testing cluster

Each node should typically have a minimum of two or four CPUs and 7 to 16 GB RAM. If you test a deployment of CFK and all Confluent Platform components, you can create a 10-node cluster with six nodes for Apache ZooKeeper™ and Apache Kafka® pods (three replicas each) and four nodes for all other components’ pods.

Confluent Platform component cluster sizing and resource requirements

In CFK, you specify resource requirements using the limits and requests properties for custom resources (CRs). CFK configures the Confluent Platform pods with the following default CPU and memory resources:

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 256Mi

The above default resource values are sufficient for most use cases. If you plan to manage a high number of day-2 application custom resources (CRs), such as 100 or more, refer to manage resources about increasing the number of application CRs and the corresponding memory and CPU.