Brokers

The Brokers feature provides a succinct view of Apache Kafka® metrics, and other details, for brokers in a cluster.

Brokers overview page

The Brokers overview page enables you to easily assess the health of brokers (nodes) in a Kafka cluster. The page provides a summary of:

  • Throughput for production and consumption
  • Partitioning and replication status
  • Which broker is working as the active controller
  • Apache ZooKeeper™ status
  • Disk usage and distribution
  • Self-Balancing status
  • Tiered Storage status
  • System metrics for network and request pool usage

To access the page, select a cluster from the navigation bar and click Brokers from the menu in the left panel of the page.

Brokers overview page

Brokers overview

The clickable panels allow you to drill into detailed information on:

The panels show a red sidebar for any issues that require operator attention.

The Brokers table at the bottom of the page lists all brokers by ID.

Brokers table

Use this table to:

  • Search for a broker by its ID.
  • Sort a column by clicking in the column title cell.
  • View throughput (bytes in and out per second) and latency percentage metrics.

Metrics

All of the metrics for your broker are summarized on one page. To view metrics, click any of the following panels on the Brokers overview page:

Broker Metrics page

Tip

You can set alerts on many of these metrics, such as: production request latency, under-replicated partitions, out of sync replica count, ZooKeeper status, and more. Send alerts action notifications though email, Slack, or PagerDuty. For details, see Alerts.

Filter brokers

In a multiple broker environment, you can view the metrics charts for multiple brokers. Click Select all or Deselect all, or individually select each broker for which you want to view metrics. If all brokers are deselected, each panel displays a No Brokers Selected message.

Broker selection panel

Customize the dashboard for broker metrics

Drag and drop to rearrange the order of the brokers metrics dashboard panels.

Note

The settings only persist across the same browser and computer.

  1. Click Customize Dashboard on the Brokers Overview > Metrics page.

    Customize dashboard broker metrics
  2. Drag the panels into the order you want.

  3. Click Save.

Viewing latency by percentage

To view production or consumption request latency by a specific percentage, select a percentage from the menu over the Request latency graph.

Request latency percentage menu

Tip

Click a point on the graph in the Request latency panels to view details on Production or Consumption request latency.

Hover to view details

Hover on any point in a metrics chart to view details for a specific point in time.

Inspection cursor broker metrics

Production metrics panel

To view Production metrics, click the Production panel from the Brokers overview page.

The chart shows throughput, request latency, and any failed production requests.

Production metrics panel brokers metrics

Click a point on the graph in the Request latency panel to view details on Production request latency.

Production request latency

Production request latency

Consumption metrics panel

To view Consumption metrics, click the Consumption panel from the Brokers overview page.

The charts show throughput, request latency, and any failed consumption requests.

Consumption metrics panel brokers metrics

Click a point on the graph in the Request latency panel to view details on Consumption request latency.

Consumption request latency diagram

Consumption request latency

Partition and replication metrics

To view partition and replication metrics, click the Partitioning and replication panel.

The charts show the total number of partitions, leaders, under-replicated partitions, and under min in-sync replicas.

Partition replicas panel broker metrics

Broker uptime and ZooKeeper metrics

To access the broker uptime and ZooKeeper panel, click the ZooKeeper panel on the Brokers overview page.

The charts show uptime for available brokers and ZooKeeper.

Broker uptime metrics panel

Note

Broker uptime metrics are cluster-wide, and do not apply to individual brokers. The Filter broker controls are not applicable to the Broker uptime panel.

Disk usage panel

To view disk usage metrics, click the Disk panel from the Brokers overview page. This panel will show a red sidebar if disk usage is skewed and should be rebalanced.

The Disk panel shows maximum usage, minimum usage, and distribution.

Disk usage panel

System usage panel

To access the System panel, click the System summary card from the Brokers overview page.

The System panel shows network and request pool usage.

Network pool usage is the average network pool capacity usage across all brokers; that is, the percentage of time that the network processor threads are not idle.

Request pool usage is the average request handler capacity usage across all brokers; that is, the percentage of time that the request handler threads are not idle.

System usage panel network and request pool usage

Active Controller

The Active controller panel identifies which broker in your cluster is reporting as the Active controller. In a Kafka cluster, one of the brokers serves as the controller, which is responsible for managing the states of partitions and replicas and for performing administrative tasks like reassigning partitions. At any given time there is only one controller broker in your cluster. This identifier panel enables operators to know which broker is working as the controller.

Active controller panel

Self-Balancing

Starting with Confluent Platform 6.1, when Self-Balancing is enabled, clicking the Self-Balancing card displays broker task statuses.

self balancing broker task statuses

If Self-Balancing is disabled, the Self-balancing card takes you to Cluster Settings -> Self-balancing, which includes a tab where to edit settings and enable Self-Balancing.

The Self-Balancing panel on the Brokers page itself indicates the status of Self-Balancing.

self balancing panel

To learn more, see the following topics:

Tiered Storage

The Tiered storage panel takes you to Cluster Settings -> Storage, which includes a tab where you can configure Tiered Storage.

Tiered Storage panel disabled

Once Tiered storage is up and running, this panel shows storage data statistics.

Tiered Storage panel enabled

Click into these initial stats to view a metrics chart for Tiered Storage.

Tiered Storage metrics chart

To learn more, see the following topics: