Brokers¶
The Brokers feature provides a succinct view of Apache Kafka® metrics, and other details, for brokers in a cluster.
Brokers overview page¶
The Brokers overview page enables you to easily assess the health of brokers (nodes) in a Kafka cluster. The page provides a summary of:
- Throughput for production and consumption
- Partitioning and replication status
- Which broker is working as the active controller
- Apache ZooKeeper™ status
- Disk usage and distribution
- Self-Balancing status
- Tiered Storage status
- System metrics for network and request pool usage
To access the page, select a cluster from the navigation bar and click Brokers from the menu in the left panel of the page.
Brokers overview¶
The clickable panels allow you to drill into detailed information on:
- metrics: production, consumption, partitioning, system, and disc metrics
- the broker working as your active controller
- Self-Balancing cluster status
- Tiered storage status
The panels show a red sidebar for any issues that require operator attention.
The Brokers table at the bottom of the page lists all brokers by ID.
Use this table to:
- Search for a broker by its ID.
- Sort a column by clicking in the column title cell.
- View throughput (bytes in and out per second) and latency percentage metrics.
Metrics¶
All of the metrics for your broker are summarized on one page. To view metrics, click any of the following panels on the Brokers overview page:
Tip
You can set alerts on many of these metrics, such as: production request latency, under-replicated partitions, out of sync replica count, ZooKeeper status, and more. Send alerts action notifications though email, Slack, or PagerDuty. For details, see Alerts.
Filter brokers¶
In a multiple broker environment, you can view the metrics charts for multiple brokers. Click Select all or Deselect all, or individually select each broker for which you want to view metrics. If all brokers are deselected, each panel displays a No Brokers Selected message.
Customize the dashboard for broker metrics¶
Drag and drop to rearrange the order of the brokers metrics dashboard panels.
Note
The settings only persist across the same browser and computer.
Click Customize Dashboard on the Brokers Overview > Metrics page.
Drag the panels into the order you want.
Click Save.
Viewing latency by percentage¶
To view production or consumption request latency by a specific percentage, select a percentage from the menu over the Request latency graph.
Tip
Click a point on the graph in the Request latency panels to view details on Production or Consumption request latency.
Hover to view details¶
Hover on any point in a metrics chart to view details for a specific point in time.
Production metrics panel¶
To view Production metrics, click the Production panel from the Brokers overview page.
The chart shows throughput, request latency, and any failed production requests.
Click a point on the graph in the Request latency panel to view details on Production request latency.
Production request latency¶
Consumption metrics panel¶
To view Consumption metrics, click the Consumption panel from the Brokers overview page.
The charts show throughput, request latency, and any failed consumption requests.
Click a point on the graph in the Request latency panel to view details on Consumption request latency.
Consumption request latency¶
Partition and replication metrics¶
To view partition and replication metrics, click the Partitioning and replication panel.
The charts show the total number of partitions, leaders, under-replicated partitions, and under min in-sync replicas.
Broker uptime and ZooKeeper metrics¶
To access the broker uptime and ZooKeeper panel, click the ZooKeeper panel on the Brokers overview page.
The charts show uptime for available brokers and ZooKeeper.
Note
Broker uptime metrics are cluster-wide, and do not apply to individual brokers. The Filter broker controls are not applicable to the Broker uptime panel.
Disk usage panel¶
To view disk usage metrics, click the Disk panel from the Brokers overview page. This panel will show a red sidebar if disk usage is skewed and should be rebalanced.
The Disk panel shows maximum usage, minimum usage, and distribution.
System usage panel¶
To access the System panel, click the System summary card from the Brokers overview page.
The System panel shows network and request pool usage.
Network pool usage is the average network pool capacity usage across all brokers; that is, the percentage of time that the network processor threads are not idle.
Request pool usage is the average request handler capacity usage across all brokers; that is, the percentage of time that the request handler threads are not idle.
Active Controller¶
The Active controller panel identifies which broker in your cluster is reporting as the Active controller. In a Kafka cluster, one of the brokers serves as the controller, which is responsible for managing the states of partitions and replicas and for performing administrative tasks like reassigning partitions. At any given time there is only one controller broker in your cluster. This identifier panel enables operators to know which broker is working as the controller.
Self-Balancing¶
Starting with Confluent Platform 6.1, when Self-Balancing is enabled, clicking the Self-Balancing card displays broker task statuses.
If Self-Balancing is disabled, the Self-balancing card takes you to Cluster Settings -> Self-balancing, which includes a tab where to edit settings and enable Self-Balancing.
The Self-Balancing panel on the Brokers page itself indicates the status of Self-Balancing.
To learn more, see the following topics:
- Work with Self-Balancing Clusters in Control Center
- Self-Balancing Clusters in the Kafka Operations documentation
Tiered Storage¶
The Tiered storage panel takes you to Cluster Settings -> Storage, which includes a tab where you can configure Tiered Storage.
Once Tiered storage is up and running, this panel shows storage data statistics.
Click into these initial stats to view a metrics chart for Tiered Storage.
To learn more, see the following topics:
- Configure Tiered Storage in Control Center
- Tiered Storage in the Kafka Operations documentation