Monitor consumer lag for Confluent Platform clusters

You can monitor Apache Kafka® consumer lag for your Confluent Platform clusters registered with Unified Stream Manager (USM) in the Confluent Cloud Console. Consumer lag is the number of consumer offsets between the latest message produced in a partition and the last message consumed by a consumer. A large or quickly growing consumer lag indicates that consumers can’t read from a partition as fast as messages are produced. Common causes include a slow consumer, slow network, or slow broker.

View consumer lag for a Confluent Platform cluster

To monitor consumer lag in the Confluent Cloud Console:

  1. In the Confluent Cloud Console, navigate to the Environments page and select your environment.

    ../../_images/usm-env.png

  2. In the navigation menu, click Clusters and select your Confluent Platform cluster.

    ../../_images/usm-cluster.png

  3. In the cluster’s navigation menu, click Clients.

    The Consumer lag tab appears by default. The page shows a list of consumer groups with the Consumer group ID and Group lag for each group.

    ../../_images/usm-consumer-lag-overview.png

  4. Select a consumer group to view its lag details.

View consumer group lag details

After you select a consumer group, the detail view shows the following sections.

Summary metrics

../../_images/usm-consumer-lag-summary.png

  • Group lag (sum): The total number of messages the consumer group is behind across all partitions.

  • Total topics: The number of topics the consumer group is consuming from.

  • Total partitions: The number of partitions across all topics the consumer group is consuming from.

  • Total consumers: The number of active consumer instances in the consumer group.

Consumer lag over time chart

../../_images/usm-consumer-lag-chart.png

The Consumer lag over time chart shows how consumer lag changes for the selected group over time. The badge on each dropdown shows the number of items currently selected.

You can use these controls to filter the chart:

  • Topics: Select one or more topics to display in the chart. Each topic has a checkbox so you can compare lag across multiple topics.

  • Partition: Select one or more partitions to display in the chart. Each partition has a checkbox so you can isolate lag for specific partitions.

  • Time: Select a time range for the chart. Options include Last 10 minutes, Last hour, and Last 6 hours.

  • CSV: Export the chart data to a CSV file for offline analysis.

Topic and partition details

The details table provides a per-topic breakdown of consumer lag.

  • Topic: The name of the Kafka topic.

  • Partitions: The number of partitions for that topic.

  • Total Lag: The total consumer lag across all partitions for that topic.

To view per-partition details, click the expand arrow on a topic row.

../../_images/usm-consumer-lag-partition.png

The expanded row shows the following columns:

  • Partition: The partition number within the topic.

  • Consumer ID: The identifier of the consumer assigned to this partition.

  • Client ID: The client identifier for the consumer instance.

  • Lag: The consumer lag for this specific partition.