Self-Balancing Clusters¶
- Self-Balancing Overview
- What are Self-Balancing Clusters?
- How Self-Balancing simplifies Kafka operations
- Self-Balancing vs. Auto Data Balancer
- How it works
- Architecture of a Self-Balancing cluster
- Enabling Self-Balancing Clusters
- What defines a “balanced” cluster and what triggers a rebalance?
- What happens if the lead broker (controller) is removed or lost?
- How do the brokers leverage Cruise Control?
- What internal topics does Self-Balancing create and use?
- Limitations
- Configuration and monitoring
- Replica placement and rack configurations
- Security considerations
- Troubleshooting
- Self-Balancing options do not show up on Control Center
- Broker metrics are not displayed on Control Center
- Consumer lag reflected on Control Center
- Broker removal attempt fails during Self-Balancing initialization
- Broker removal cannot complete due to offline partitions
- Too many excluded topics causes problems with Self-Balancing
- The balancer status for a KRaft controller hangs
- Related content
- Quick Start Demo (Docker)
- Tutorial: Adding and Removing Brokers
- Configuration Options and Commands
- Self-Balancing Configurations on the Brokers
- confluent.balancer.enable
- confluent.balancer.heal.uneven.load.trigger
- confluent.balancer.heal.broker.failure.threshold.ms
- confluent.balancer.throttle.bytes.per.second
- confluent.balancer.disk.max.load
- confluent.balancer.disk.max.replicas
- confluent.balancer.exclude.topic.names
- confluent.balancer.exclude.topic.prefixes
- confluent.balancer.topic.replication.factor
- Self-Balancing Internal Topics
- Required Configurations for Control Center
- Examples: Update broker configurations on the fly
- Monitoring the balancer with kafka-rebalance-cluster
- kafka-remove-brokers
- Suggested Reading
- Self-Balancing Configurations on the Brokers
- Performance and Resource Usage