Confluent Metrics Reporter

Confluent Metrics Reporter collects various metrics from the Kafka cluster necessary for Confluent Control Center system health monitoring and Confluent Auto Data Balancer to operate.

This data is published to a configurable topic (_confluent-metrics by default) in a configurable Kafka cluster. You may choose to publish metrics to the cluster the broker is part of – as described in the following section – or to a different Kafka cluster (e.g. a dedicated metrics cluster).

Installation

To enable Confluent Metrics Reporter, install either the confluent-rebalancer package (installed automatically if either the confluent-control-center or the confluent package is installed) in each broker and add the following settings in each server.properties (updated to match your environment):

metric.reporters=io.confluent.metrics.reporter.ConfluentMetricsReporter
confluent.metrics.reporter.bootstrap.servers=localhost:9092
confluent.metrics.reporter.zookeeper.connect=localhost:2181

# Uncomment the following if the metrics cluster has < 3 brokers
#confluent.metrics.reporter.topic.replicas=1

For convenience, the server.properties shipped with the Confluent platform includes the above configs commented out under the Confluent Metrics Reporter section.

Note that you can choose to publish the metrics to the cluster the broker is part of (like in the config above) or to a different cluster (e.g. a dedicated metrics cluster).

A rolling bounce of the brokers is required for the config changes to be picked up.

In order for this to work correctly, we require the following:

  1. The broker’s principal must have permission to create the metrics topic in the configured Kafka cluster. This requires write access to the corresponding Zookeeper.
  2. The broker’s principal must have permission to produce to the metrics topic.
  3. The tool’s principal must have permission to consume from the metrics topic.

Configuration Options

These configs should be added to the server.properties of each broker that is part of the Kafka cluster to be rebalanced. Only the first two configs below are required although confluent.metrics.reporter.topic.replicas should be changed if there are less than 3 brokers in the Kafka metrics cluster. The other configs allow one to tune the publisher for additional performance and reliability.

confluent.metrics.reporter.bootstrap.servers

Bootstrap servers for the Kafka cluster where metrics will be published to.

  • Type: string
  • Importance: high
confluent.metrics.reporter.zookeeper.connect

The Zookeeper URL for the Kafka cluster used for publishing metrics. The metrics cluster may be different from the cluster(s) whose metrics are being collected. Several production Kafka clusters can publish to a single metrics cluster, for example.

  • Type: string
  • Importance: high
confluent.metrics.reporter.publish.ms

The metrics reporter will publish new metrics to the metrics topic in intervals defined by this setting. This means that control center system health data lags by this duration, or that rebalancer may compute a plan based on broker data that is stale by this duration. The default of 15 seconds is a reasonable value for production environments and it typically does not need to be changed.

  • Type: long
  • Default: 15000
  • Importance: medium
confluent.metrics.reporter.topic

Topic on which metrics data will be written.

  • Type: string
  • Default: “_confluent-metrics”
  • Importance: medium
confluent.metrics.reporter.topic.partitions

Number of partitions in the metric topic.

  • Type: int
  • Default: 12
  • Importance: medium
confluent.metrics.reporter.topic.replicas

Number of replicas in the metric topic. It should not be higher than the number of brokers in the Kafka cluster.

  • Type: int
  • Default: 3
  • Importance: medium
confluent.metrics.reporter.topic.retention.bytes

Retention bytes for the metrics topic.

  • Type: long
  • Default: -1
  • Importance: medium
confluent.metrics.reporter.topic.retention.ms

Retention time for the metrics topic.

  • Type: long
  • Default: 259200000 (3 days)
  • Importance: medium
confluent.metrics.reporter.topic.roll.ms

Log rolling time for the metrics topic.

  • Type: long
  • Default: 14400000 (4 hours)
  • Importance: medium
confluent.metrics.reporter.whitelist

Regex matching the yammer metric mbean name or Kafka metric name to be published to the metrics topic.

By default this includes all the metrics required by Confluent Control Center and Confluent Auto Data Balancer. This should typically never be modified unless requested by Confluent.

  • Type: string
  • Default: includes all the metrics necessary for Confluent Control Center and Confluent Auto Data Balancer
  • Importance: low