Manage Kafka Consumers Using Control Center for Confluent Platform

The Consumers menu provides a view of all consumer groups for all topics in a cluster. From the consumer pages, you can:

• View all consumer groups for a cluster in the All consumer groups page
• View consumer lag across all topics in a cluster
• View consumption metric for a consumer group
• Set up consumer group alerts

Tip

Consumer Lag for an individual topic can be accessed from the Topics menu.

Important

The Consumers feature is enabled by default using an Access Control setting in the Control Center properties files. See Enabling and disabling the Consumers feature. If RBAC is enabled for Control Center, the resources you can access are determined by your assigned role or roles and their corresponding privileges.

All consumer groups page

Use the All consumer groups page to:

• View all consumer groups for a cluster
• Search for a consumer group by its ID
• View the number of consumers and number of source topics for each consumer group
• View details such as Consumer Lag and Consumption metrics for a consumer group
• Set up a pre-populated alert for Consumer Lag

To access the All consumer groups page:

1. Select a cluster from the navigation bar and click the Consumers menu.

   

View consumer lag details for a consumer group

1. From the All consumer groups page, click the ID for your desired Consumer Group. Consumer lag details are displayed, including:

   • Total messages behind and the measured interval.
   • All consumers in the group.
   • A visualization of lag. The Max lag / consumer graph displays a pin point of each partition in the topic.
   

2. To view details for a particular point on the graph, click its pin. View the Consumer, Partition, Current offset, and Messages behind for that particular point.

   

View consumption details for a consumer group

View the percentage (%) of messages consumed and the end-to-end latency key metrics by a selected timeline.

Note

The consumption charts are only populated if there are Confluent Monitoring Interceptors that are set up correctly. For instructions, see Monitor Production and Consumption Using Control Center in Confluent Platform.

1. From the All consumer groups page, click the ID for your desired Consumer Group.

2. Click the Consumption tab. The streams monitoring metrics are displayed for the default timeframe (last 4 hours).

   

3. Select a timeframe from the menu and click Apply.

   

   Timeframe selector

   Use the time range selector to select the time range for data displayed in charts:

   • Static: A specific time range with constant start and end times.
   • Rolling: A time range where the end time is always equal to the current time and the extent of the time range is held constant.
   • Growing: A time range where the end time is always equal to the current time and the start time is held constant.

   The default selection is Last 4 hours.

4. Select the End-to-end latency metric you want to view:

   • average (ms)
   • minimum (ms)
   • maximum (ms)
   • total (ms)
   

   Tip

   You can set up alerts for average and maximum latency values.

Set up an alert for a consumer group

You can set alerts from the Consumer lag page. These options pre-populate most of the consumer group trigger fields.

On the Consumer lag page, click Set up an alert.

For more details, see Consumer lag trigger.

Enabling and disabling the Consumers feature

The Consumers feature for viewing details for consumer lag and consumption in consumer groups is enabled by default. The feature can be disabled by setting the confluent.controlcenter.consumers.view.enable option in the appropriate Control Center properties file to false. After disabling the feature, the Consumer submenu in the Cluster navigation is no longer visible in the Control Center UI.

Important

If RBAC is enabled for Control Center, it takes precedence over Access Control settings.

To disable the Consumers feature in Control Center:

1. Set the confluent.controlcenter.consumers.view.enable option in your control-center.properties file to false.

   ...
   confluent.controlcenter.consumers.view.enable=false
   ...
   

   Note

   Make the change in the appropriate Control Center properties files configured for your environments, including control-center-dev.properties or control-center-production.properties. The properties files are located in /path-to-confluent/etc/confluent-control-center/. For more information, see Control Center Configuration Examples for Confluent Platform.

2. Restart Control Center and pass in the properties file for the configuration to take effect.

   • Local (dev) environment:

     Important

     Do not use confluent local for production use.

     confluent local services control-center stop
     confluent local services control-center start --config ../etc/confluent-control-center/control-center.properties
     

   • Production environment:

     ./bin/control-center-stop
     ./bin/control-center-start ../etc/confluent-control-center/control-center.properties
     

To enable the feature again, set the option back to true and restart Control Center with the updated properties file.