Starting October 4, 2021, the Cluster Shrink and the Cluster Load metric features will incrementally release to Confluent Cloud customers using Dedicated clusters. Feature release is expected to complete by mid-November. For more information, see the Release Notes.
Cluster Load Metric for Dedicated Clusters in Confluent Cloud¶
The Cluster Load metric for Dedicated clusters helps provide visibility into the current load on a cluster.
Access the cluster load metric¶
You can access the Cluster load metric as both a current value and a time series graph that represents historical load values. Use the Confluent Cloud Metrics API or view the metric in the Confluent Cloud Console.
To view the Cluster load metric in the Cloud Console:
Navigate to the clusters page for your environment and choose a cluster.
View the Cluster load metric on the cluster Overview page.
Use the drop-down to choose the cluster load averaged over the Last hour, Last 6 hours, Last 24 hours, or Last 7 days.
Details about cluster load¶
The displayed Cluster load graphs percentage values between 0-100 over the time period selected, with 0% indicating no load, and 100% representing a fully-saturated cluster. This metric is similar to the Linux Load Average, in that it incorporates many underlying elements of system utilization to produce a singular metric specific to Kafka.
Cluster load includes metrics such as CPU utilization, memory pressure, I/O utilization, and specific Kafka metrics. Some CKU dimensions such as number of connections and throughput can influence the Cluster load metric, and other dimensions such as storage consumption do not. For more information on resource limits, see Kafka cluster quotas.
Evaluate cluster expansion with cluster load¶
Cluster expansion is a reasonable first step in troubleshooting performance issues in your Apache Kafka® applications. Use the cluster load metric to decide whether to expand your clusters, or if this does not resolve the performance issues, shrink them back to the original size.
Generally, expanding clusters provides more capacity for your workloads, and in many cases, will help improve the performance of your Kafka applications. However, there are scenarios in which cluster expansion does not adequately resolve application performance concerns. For more complicated scenarios, see Dedicated Cluster Performance and Expansion.