Manage Self-Balancing Clusters

• Overview
  • How Self-Balancing simplifies Kafka operations
  • Self-Balancing vs. Auto Data Balancer
  • How it works
    • Architecture of a Self-Balancing cluster
  • Enable Self-Balancing Clusters
  • Frequently asked questions
    • What defines a “balanced” cluster and what triggers a rebalance?
    • What happens if the lead broker (controller) is removed or lost?
    • How do the brokers use Cruise Control?
    • What internal topics does the Self-Balancing Clusters feature create and use?
  • Limitations and known issues
  • Configuration and monitoring
    • Get status on the balancer
    • Use Control Center
    • Kafka server properties and commands
    • Metrics for monitoring a rebalance
  • Replica placement and rack configurations
    • Racks
    • Replica placement and multi-region clusters
    • Broker capacity limits
    • Replica and disk distribution
    • Debug rebalance failures
  • Security considerations
  • Troubleshooting
    • Self-Balancing options do not show up on Control Center
    • Broker metrics are not displayed on Control Center
    • Consumer lag reflected on Control Center
    • Broker removal attempt fails during Self-Balancing initialization
    • Broker removal cannot complete due to offline partitions
    • Too many excluded topics cause problems with Self-Balancing
    • The balancer status for a KRaft controller hangs
    • Rebalance fails with insufficient healthy cluster capacity
  • Related content
• Tutorial: Adding and Removing Brokers
  • About KRaft mode in this tutorial
  • Prerequisites
  • Environment variables
  • Configure Kafka brokers
    • Create broker-0 to use a template for the other brokers
    • Enable the Metrics Reporter for Control Center
    • Configure replication factors for Self-Balancing
    • Verify that Self-Balancing is enabled
    • Save the file
    • Create a basic configuration for a five-broker cluster
  • Start Confluent Platform and create topics
    • Start the controller and brokers
    • Create a topic and test the cluster
    • Next steps: Monitor and test Self-Balancing Clusters
  • (Optional) Install and configure Control Center
    • Download, extract, and configure Control Center
    • Configure Control Center with REST endpoints and advertised listeners
    • Start Prometheus and Control Center
    • Configure the controller and brokers to send metrics to Control Center with Prometheus
    • Restart the controller and brokers
  • Use the command line to test rebalancing
    • List topics and generate data to your test topic
    • Verify status using the command line
    • Remove a broker
    • Add a broker
  • Use Control Center to test rebalancing
    • Verify status using Control Center
    • Remove a broker
    • Add a broker
  • (Optional) Run Kafka Connect, ksqlDB, and Schema Registry
  • Shutdown and cleanup tasks
  • Related content
• Configure
  • Self-Balancing configuration
    • confluent.balancer.enable
    • confluent.balancer.heal.uneven.load.trigger
    • confluent.balancer.heal.broker.failure.threshold.ms
    • confluent.balancer.throttle.bytes.per.second
    • confluent.balancer.disk.max.load
    • confluent.balancer.max.replicas
    • confluent.balancer.exclude.topic.names
    • confluent.balancer.exclude.topic.prefixes
    • confluent.balancer.topic.replication.factor
  • Examples: Update broker configurations on the fly
    • Enable or disable Self-Balancing
    • Set trigger condition for rebalance
    • Set or remove a custom throttle
  • Self-Balancing internal topics
  • Required configurations for Control Center
    • Configure REST endpoints in the Control Center properties file
    • Configure authentication for REST endpoints on Kafka brokers (secure setup)
  • Monitoring the balancer with kafka-rebalance-cluster
    • Get the balancer status
    • Get the workload optimization status (AnyUnevenLoad)
  • kafka-remove-brokers
    • kafka-remove-brokers flags
    • kafka-remove-brokers examples
  • Broker removal phases
  • Self-Balancing initialization
  • Broker removal task priority
  • Related content
• Performance and Resource Usage
  • Add brokers to expand a small cluster with a high partition count
    • Test description: expand cluster
    • Cluster configuration: expand cluster
    • Performance results: expand cluster
  • Test scalability of a large cluster with many partitions
    • Test description: scalability test
    • Cluster configuration: scalability test
    • Performance results: scalability test
  • Repeatedly bounce the controller
    • Test description: controller failover
    • Cluster configuration: controller failover
    • Performance results: controller failover
  • Related content