CPU and Memory Resources for Confluent Platform in Confluent for Kubernetes¶
Setting proper requests and limits is important for the performance of the Confluent Platform clusters and their applications.
Before configuring CPU and memory resource requirements for Confluent Platform, review Cluster sizing for resource allocation planning.
Requests and limits are dependent on your workload. The best practice should follow the minimum requirements for Confluent Platform, and then benchmark and tune if needed to suit your environment.
For more information about CPU and memory resources in Kubernetes, see Resource Management for Pods and Containers.
Confluent for Kubernetes (CFK) allows you to define custom pod resource requirements for Confluent Platform
components it deploys. You specify these requirements using the
limits properties for components in their custom resources (CR).
spec: podTemplate: resources: ---  limits: ---  cpu: ---  memory: ---  requests: ---  cpu: ---  memory: --- 
resourcesdescribe the compute resource requirements for this component CR.
limitsdescribe the maximum amount of compute resources allowed. Your Confluent Platform component will throttle if it tries to use more resources than the values set here.
  Limits and requests for CPU resources are measured in CPU units. 1 CPU unit is equivalent to 1 physical CPU core or 1 virtual core.
Fractional CPU requests are allowed. For example,
resources.requests.cpu: 0.5requests half as much CPU time.
For CPU resource units, the quantity expression
0.1is equivalent to the expression
100m, which means “one hundred milli CPU”.
  Limits and requests for memory are measured in bytes.
requestsdescribe the minimum amount of compute resources required. If the
requestssection is omitted, it defaults to
limitsif that is explicitly specified in the same CR, and otherwise to the values defined for the Kubernetes cluster.
The following example CR specifies a request for 0.25 CPU and 64 MiB of memory. Its limits are set to 0.5 CPU and 128 MiB of memory.
spec: podTemplate: resources: requests: memory: "64Mi" cpu: "250m" limits: memory: "128Mi" cpu: "500m"
Define Java heap size¶
In addition to setting memory sizing, you need to define the the Java (JVM) heap size for the Confluent Platform components.
You define the Java heap size for Confluent Platform using the configuration override
spec.configOverrides.jvm) in the component CR.
Here’s an example where the JVM heap size is set to 6 GB:
spec: configOverrides: server: - auto.create.topics.enable=true - confluent.cluster.link.enable=true jvm: - "-Xmx6g" # Set to 6 GB - "-XX:MetaspaceSize=96m" - "-XX:+UseG1GC"
For more details about the configuration overrides feature, see Configuration overrides.
Resource definitions for CFK Init Container¶
CFK does not allow you to configure the resource requests or limits for the Init Container. For each Confluent Platform deployment, CFK sets the following for the Init Container:
resources: limits: cpu: 500m memory: 1Gi requests: cpu: 100m memory: 512Mi