Configuration Options for the rebalancer tool

The following configuration options are specific to the rebalancer tool. See Execute the rebalancer for an example. For any configuration options related to metrics reporting, see Confluent Metrics Reporter configuration.

The configuration file path is passed using the --config-file option. Additional connection options to the rebalancer tool include:

--zookeeper

This option is deprecated in Confluent Platform 5.5.0 and later. The connection string for the ZooKeeper connection in the form host:port. You can specify multiple URLs, separated by commas, to allow for failover should a ZooKeeper node go down.

  • Type: string
  • Default: empty string
  • Importance: high
--bootstrap-server

The connection string for the cluster’s broker(s) in the form host:port. You can specify multiple URLs, separated by commas, to allow for failover should a broker node go down. This is required.

  • Type: string
  • Default: empty string
  • Importance: high
--throttle

Specifies the maximum bandwidth, in bytes per second, allocated to moving replicas. This is required. See Execute the rebalancer for an example.

  • Type: int
  • Default: empty string
  • Importance: high
--command-config

Specifies a property file containing configurations to be passed to the Admin Client. This option is used only with the --bootstrap-server option. This is required in some use cases. For example, if the server is secured with SSL (security.protocol=SSL), you must specify this in the --command-config to the Admin Client and the --config-file for the metrics reporter.

  • Type: string
  • Default: empty string
  • Importance: low
--topics

Limits the rebalance plan to a specified topic or comma-separated list of topics.

  • Type: string
  • Default: empty string
  • Importance: medium
--replica-placement-only

Limits a rebalance plan to topics that have changed their replica placement constraints, and are not satisfying this constraint under their current location.

  • Type: boolean
  • Default: false
  • Importance: medium

To test the rebalancer tool, you must specify --bootstrap-server to connect to the Kafka cluster. Note that confluent.license is required after 30 days, however it’s still required that the license client is configured to communicate with the cluster. The following configurations are supported.

--verbose

Also output before and after rebalance per broker information on disk space usage, leader count, replica count, and topic partitions.

  • Type: boolean
  • Default: false
  • Importance: low

Tip

For complete license information for Confluent Platform, see Confluent Platform Licenses.

confluent.license

Confluent will issue a license key to each subscriber. The license key will be a short snippet of text that you can copy and paste. Without the license key, you can use the Confluent Rebalancer for a 30-day trial period. If you are a subscriber and don’t have a license key, contact Confluent Support at support@confluent.io. This must be specified in the --config-file.

  • Type: string
  • Valid Values: Confluent Platform license
  • Importance: high

You must configure the rebalancer client to register or validate the license with an Kafka cluster by prefixing the configuration names with confluent.license.. For example, to configure the license client to use SASL_SSL, specify the following:

confluent.license.security.protocol=SASL_SSL
confluent.license.ssl.truststore.location=/var/private/ssl/truststore.jks
confluent.license.ssl.truststore.password=<ssl-truststore-password>
confluent.license.ssl.keystore.location=/var/private/ssl/keystore.jks
confluent.license.ssl.keystore.password=<ssl-keystore-password>
confluent.license.ssl.key.password=<ssl-key-password>
confluent.license.sasl.mechanism=GSSAPI
confluent.license.sasl.kerberos.service.name=kafka
confluent.license.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
        useKeyTab=true \
        storeKey=true \
        keyTab="/path/to/kerberos/kafka.user.keytab" principal="kafka@KAFKA.SECURE";
confluent.rebalancer.metrics.collection.timeout.ms

The maximum amount of time the rebalancer will collect metrics for. If the collection does not succeed before the timeout elapses, then the rebalance command will fail.

  • Type: int
  • Default: 60000
  • Importance: low
confluent.rebalancer.metrics.topic

The topic in which the metrics reporter publishes its metrics.

  • Type: string
  • Default: “_confluent-metrics”
  • Importance: low
confluent.rebalancer.min.free.volume.space.percentage

The log.dir volume will have at least the specified percentage of free space during and after the rebalance. For example, if the total volume space is 100 GB and this configuration is defined as 20, then the rebalancer will use up to 80 GB during the rebalance. This is supported in Confluent 3.2 only (for both rebalancer and brokers), and only if every broker in the cluster has a single log.dir. In versions where this attribute supported, it is enabled by default; otherwise, it is disabled.

  • Type: double
  • Default: 20.0
  • Valid Values: [0,…,100]
  • Importance: low

You can configure the consumer used to retrieve metrics data by prefixing the consumer configuration name with confluent.rebalancer.metrics.. For example, to configure the consumer to use SASL_SSL, specify the following:

confluent.rebalancer.metrics.security.protocol=SASL_SSL
confluent.rebalancer.metrics.ssl.truststore.location=/var/private/ssl/truststore.jks
confluent.rebalancer.metrics.ssl.truststore.password=<ssl-truststore-password>
confluent.rebalancer.metrics.ssl.keystore.location=/var/private/ssl/keystore.jks
confluent.rebalancer.metrics.ssl.keystore.password=<ssl-keystore-password>
confluent.rebalancer.metrics.ssl.key.password=<ssl-key-password>
confluent.rebalancer.metrics.sasl.mechanism=GSSAPI
confluent.rebalancer.metrics.sasl.kerberos.service.name=kafka
confluent.rebalancer.metrics.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
        useKeyTab=true \
        storeKey=true \
        keyTab="/path/to/kerberos/kafka.user.keytab" principal="kafka@KAFKA.SECURE";