Resize a Dedicated Kafka Cluster

Basic, Standard, Enterprise, and Freight clusters are elastic and scale automatically based on load. Dedicated clusters don’t scale automatically, but you can resize one to meet your needs by changing its Confluent Unit for Kafka (CKU) count to expand or shrink capacity.

Choosing the right number of CKUs helps you reduce costs or scale to meet your streaming needs. Use the Confluent Cloud Console, Confluent CLI, or REST API to resize your cluster.

When you resize a Dedicated Kafka cluster, Confluent rebalances the cluster to help ensure a balanced load across the remaining brokers in the cluster. If you’re reducing the size of the cluster, Confluent removes unused brokers at the end of the rebalance operation.

Prerequisites:

An existing Dedicated Kafka cluster
Usage-based billing enabled

Considerations:

You cannot request another resize until the current resize completes.
To decide whether to expand, see Interpret a high cluster load value.
Confluent blocks shrink requests that would violate the per-CKU dimension limits for partitions or ingress/egress throughput for the requested cluster size. Multi-zone Dedicated Kafka clusters require a minimum capacity of 2 CKUs. For more information, see CKU limits per cluster and Kafka cluster service quotas.
If a shrink operation fails, the cluster expands to its original capacity, and you receive a notification when the expansion is complete.

Time estimates

A resize operation involves Confluent provisioning the added or removed capacity and rebalancing partitions across the cluster.

Provisioning time

Provisioning typically takes less than two hours. You receive an email when provisioning is complete.

Sometimes, due to cloud-provider-specific constraints, provisioning can take longer. Check the Confluent Cloud status page for any ongoing service disruptions. Contact Confluent Support if provisioning takes longer than 6 hours.

Provisioning time is excluded from the Confluent SLA.

Resizing time

Resizing typically takes about 30 to 60 minutes per CKU. You receive an email when the resize operation is complete.

When a cluster is under heavy load or has many partitions to move, resizing can take longer than expected. If you foresee a scheduled event that requires more CKU capacity, proactively scale up the cluster in advance. Doing so accounts for potential delays caused by resource constraints and the rebalancing process.

During a resize operation, your applications might detect leader elections, but performance remains stable. Supported Kafka clients handle these changes gracefully.

Resize a cluster

Choose a method to resize your cluster: Confluent Cloud Console, Confluent CLI, or REST API.

Cloud Console

Select your Confluent Cloud cluster.
Click Cluster Settings, then the Capacity tab. On the Adjustable limits pane, select Adjust capacity.
Use the slider to adjust the number of CKUs for the cluster. You see a preview of the adjusted capacity before it applies. When the changes look correct, click Apply changes.
1. To expand, increase the number of CKUs.
2. To shrink, reduce the number of CKUs to any number your cluster supports.
  
  Note
  
  In rare scenarios, such as when you have recently expanded your cluster, you might see the following warning in the Cloud Console. If this occurs, wait 1 to 2 hours and retry the shrink operation.
Confirm your changes and click Continue.
1. Expanding:
2. Shrinking:

Confluent CLI

To resize a cluster, run the following command in your terminal, specifying the number of CKUs you want with --cku. For full details, see confluent kafka cluster update.

confluent kafka cluster update <id> --cku <cku>

For example, the following command resizes the lkc-abc123 cluster to 4 CKUs:

confluent kafka cluster update lkc-abc123 --cku 4

Confluent Cloud APIs

To resize a cluster, make a PATCH call that specifies the number of CKUs you want with "cku":. For parameter descriptions and more details, see Update Kafka clusters.

For example, the following command resizes the lkc-abc123 cluster to 4 CKUs:

PATCH /cmk/v2/clusters/lkc-abc123?environment=env-test1 HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
         "config": {
               "kind": "Dedicated",
               "cku": 4
         },
         "environment": {
               "id": "env-test1"
         }
   }
}