Configure CPU and Memory

Set proper CPU and memory requests and limits to maintain the performance of your Confluent Platform clusters and applications.

Before configuring CPU and memory resource requirements for Confluent Platform, review Plan Capacity and Sizing for Confluent for Kubernetes for resource allocation planning.

Specify CPU and memory requests and limits

Confluent for Kubernetes (CFK) allows you to define custom pod resource requirements for Confluent Platform components it deploys. You specify these requirements using the requests and limits properties for components in their custom resources (CR).

Requests and limits are dependent on your workload. As a best practice, follow the minimum requirements for Confluent Platform, and then benchmark and tune if needed to suit your environment.

For more information about CPU and memory resources in Kubernetes, see Resource Management for Pods and Containers.

spec:
  podTemplate:
    resources:
      limits:
        cpu:
        memory:
      requests:
        cpu:
        memory:

Field	Description
`resources`	Describes the compute resource requirements for this component custom resource (CR).
`limits`	Describes the maximum amount of compute resources allowed. If your Confluent Platform component tries to use more resources than the values set here, it throttles.
`requests`	Describes the minimum amount of compute resources required. If the `requests` section is omitted, it defaults to `limits` if that is explicitly specified in the same CR, and otherwise to the values defined for the Kubernetes cluster.
`cpu` (under `limits` and `requests`)	Limits and requests for CPU resources are measured in CPU units. 1 CPU unit is equivalent to 1 physical CPU core or 1 virtual core. Fractional CPU requests are allowed. For example, `resources.requests.cpu: 0.5` requests half as much CPU time. For CPU resource units, the quantity expression `0.1` is equivalent to the expression `100m`, which means “one hundred milli CPU”.
`memory` (under `limits` and `requests`)	Limits and requests for memory are measured in bytes.

The following example custom resource (CR) specifies requests for 0.25 CPU and 64 MiB of memory. The limits are set to 0.5 CPU and 128 MiB of memory.

spec:
  podTemplate:
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Resource definitions for CFK init container

CFK does not allow you to configure the resource requests or limits for the init container. For each Confluent Platform deployment, CFK sets the following for the init container:

resources:
  limits:
    cpu: 500m
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 512Mi

Define Java heap size

In addition to memory sizing, you can configure the Java Virtual Machine (JVM) heap size for Confluent components using the configuration override feature (spec.configOverrides.jvm).

For guidance on setting JVM heap size, see the Tuning defaults for containers article.

For the full list of JVM flags that CFK sets by default and how to override or remove them, see Default JVM settings.

Default JVM heap size

For Kafka brokers and KRaft controllers, CFK sets -Xms and -Xmx to the following values:

Kafka broker: min(50% of requests.memory, 32Gi)
KRaft controller: min(80% of requests.memory, 32Gi)

-Xms and -Xmx are set to the same value. If spec.podTemplate.resources.requests.memory is not set, CFK uses spec.podTemplate.resources.limits.memory instead.

All other Confluent Platform components continue to default to 100% of the pod memory request. If no request is set, CFK uses the memory limit instead.

An explicit -Xms/-Xmx value set through spec.configOverrides.jvm always takes precedence over this default.

Remove the default JVM settings

To auto scale the JVM heap size (UseContainerSupport) or to set the JVM heap size (MaxRAMPercentage), you must remove the existing JVM -Xms and -Xmx settings that CFK sets by default.

To remove them, add the following overrides to the component CR:

kind: <component>
spec:
  configOverrides:
    jvm:
      - "---Xmx"
      - "---Xms"

Specify the overrides exactly as shown, with no value. CFK removes the default setting only when the override is exactly ---Xmx or ---Xms. An override that includes a value, such as ---Xmx4G, does not remove the default.

Auto scale JVM heap size

To auto scale the JVM heap size, enable UseContainerSupport by adding the following to the component custom resource (CR):

kind: <component>
spec:
  configOverrides:
    jvm:
      - "-XX:+UseContainerSupport"

Set JVM heap size

To set the JVM heap size, in the component custom resource (CR), set MaxRAMPercentage to match the desired JVM heap size in relation to the total memory limit (spec.podTemplate.resources.limits.memory) you set in Specify CPU and memory requests and limits.

kind: <component>
spec:
  configOverrides:
    jvm:
      - "-XX:MaxRAMPercentage=<percentage value>"

The JVM calculates the heap as a percentage of the entire Kubernetes node memory, not the pod memory, when you set -XX:MaxRAMPercentage but do not set -Xms, -Xmx, pod memory requests (spec.podTemplate.resources.requests.memory), or pod memory limits (spec.podTemplate.resources.limits.memory). In this example, the Java heap is 50% of the node memory.

kind: <component>
spec:
  configOverrides:
    jvm:
      - "-XX:MaxRAMPercentage=50.0"

To make the heap a percentage of the pod memory instead of the node memory, you must set pod memory limits (spec.podTemplate.resources.limits.memory). In this example, the Java heap is 50% of the pod’s memory limit (4Gi), not 50% of the node memory.

kind: <component>
spec:
  podTemplate:
    resources:
      limits:
        memory: "4Gi"
  configOverrides:
    jvm:
      - "-XX:MaxRAMPercentage=50.0"

Examples for JVM heap size configuration

The following examples show the JVM MaxRAMPercentage set to 50% of the memory and auto-scaling enabled.

Example 1: In this example, only limits is set (4Gi) for the pod. To remove the -Xmx and -Xms, add the ---Xmx and ---Xms overrides.

apiVersion: platform.confluent.io/v1beta1
kind: Kafka
metadata:
  name: kafka
  namespace: confluent
spec:
  podTemplate:
    resources:
       limits:
         memory: "4Gi"

  configOverrides:
    jvm:
      - "---Xmx" # This removes -Xmx parameter
      - "---Xms" # This removes -Xms parameter
      - "-XX:+UseContainerSupport"
      - "-XX:MaxRAMPercentage=50.0"

Example 2: In this example, both limits and requests are set for the pod. To remove the -Xmx and -Xms, add the same ---Xmx and ---Xms overrides. The limits and requests values do not change the overrides.

apiVersion: platform.confluent.io/v1beta1
kind: Kafka
metadata:
  name: kafka
  namespace: confluent
spec:
  podTemplate:
    resources:
       limits:
         memory: "4Gi"
       requests:
         memory: "2Gi"

  configOverrides:
    jvm:
      - "---Xmx" # This removes -Xmx parameter
      - "---Xms" # This removes -Xms parameter
      - "-XX:+UseContainerSupport"
      - "-XX:MaxRAMPercentage=50.0"

For more details about the configuration overrides feature, see Configuration overrides.