Consumer Metrics

This topic describes JMX metrics available on Kafka consumer instances and consumer groups. These metrics are useful for monitoring the health and performance of your consumer applications.

For information about how to configure JMX, see Configure JMX for Monitoring.

Search for a metric

Consumer instance metrics

A consumer instance exposes the following metrics. These metrics are attributes on the listed MBean.

commit-sync-time-ns-total

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: commit-sync-time-ns-total: The cumulative sum of time in nanoseconds elapsed during calls to Consumer.commitSync. Kafka 3.1.1 and later.

committed-time-ns-total

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: committed-time-ns-total: The cumulative sum of time in nanoseconds elapsed during calls to Consumer.committed. Kafka 3.1.1 and later.

last-poll-seconds-ago

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: last-poll-seconds-ago: The number of seconds since the last poll() invocation. Kafka 2.4.0 and later.

poll-idle-ratio-avg

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: poll-idle-ratio-avg: The average fraction of time the consumer’s poll() is idle as opposed to waiting for the user code to process records. Kafka 2.4.0 and later.

time-between-poll-avg

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: time-between-poll-avg: The average delay between invocations of poll(). Kafka 2.4.0 and later.

time-between-poll-max

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: time-between-poll-max: The maximum delay between invocations of poll(). Kafka 2.4.0 and later.

Consumer fetch metrics

bytes-consumed-rate

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: bytes-consumed-rate: The average number of bytes consumed per second.

bytes-consumed-total

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: bytes-consumed-total: The total number of bytes consumed.

fetch-latency-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-latency-avg: The average time taken for a fetch request.

fetch-latency-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-latency-max: The maximum time taken for a fetch request.

fetch-rate

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-rate: The number of fetch requests per second.

fetch-size-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-size-avg: The average number of bytes fetched per request.

fetch-size-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-size-max: The maximum number of bytes fetched per request.

fetch-throttle-time-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-throttle-time-avg: The average throttle time in milliseconds. When quotas are enabled, the broker may delay fetch requests in order to throttle a consumer which has exceeded its limit. This metric indicates how throttling time has been added to fetch requests on average.

fetch-throttle-time-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-throttle-time-max: The maximum throttle time in milliseconds.

fetch-total

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: fetch-total: The total number of fetch requests.

records-consumed-rate

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: records-consumed-rate: The average number of records consumed per second.

records-consumed-total

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: records-consumed-total: The total number of records consumed.

records-lag-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: records-lag-max: The maximum lag in terms of number of records for any partition in this window. An increasing value over time is your best indication that the consumer group is not keeping up with the producers.

records-lead-min

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: records-lead-min: The minimum lead in terms of number of records for any partition in this window.

records-per-request-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId} Attribute: records-per-request-avg: The average number of records in each request.

Consumer per-topic fetch metrics

bytes-consumed-rate

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: bytes-consumed-rate: The average number of bytes consumed per second for a specific topic.

bytes-consumed-total

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: bytes-consumed-total: The total number of bytes consumed for a specific topic.

fetch-size-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: fetch-size-avg: The average number of bytes fetched per request for a specific topic.

fetch-size-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: fetch-size-max: The maximum number of bytes fetched per request for a specific topic.

records-consumed-rate

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: records-consumed-rate: The average number of records consumed per second for a specific topic.

records-consumed-total

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: records-consumed-total: The total number of records consumed for a specific topic.

records-per-request-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,client-id="{clientId}",topic="{topic}" Attribute: records-per-request-avg: The average number of records in each request for a specific topic.

Consumer per-partition fetch metrics

preferred-read-replica

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: preferred-read-replica: The current read replica for the partition, or -1 if reading from leader.

records-lag

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lag: The latest lag of the specified partition.

records-lag-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lag-avg: The average lag of the specified partition.

records-lag-max

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lag-max: The maximum lag of the specified partition.

records-lead

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lead: The latest lead of the specified partition.

records-lead-avg

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lead-avg: The average lead of the specified partition.

records-lead-min

MBean: kafka.consumer:type=consumer-fetch-manager-metrics,partition="{partition}",topic="{topic}",client-id="{clientId} Attribute: records-lead-min: The min lead of the specified partition.

Consumer global request metrics

incoming-byte-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: incoming-byte-rate: The average number of incoming bytes received per second from all servers. Global request metric.

outgoing-byte-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: outgoing-byte-rate: The average number of outgoing bytes sent per second to all servers. Global request metric.

request-latency-avg

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: request-latency-avg: The average request latency in milliseconds. Global request metric.

request-latency-max

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: request-latency-max: The maximum request latency in milliseconds. Global request metric.

request-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: request-rate: The average number of requests sent per second. Global request metric.

response-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: response-rate: The average number of responses received per second. Global request metric.

Consumer global connection metrics

connection-close-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: connection-close-rate: The number of connections closed per second in the window. Consumer global connection metric.

connection-count

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: connection-count: The current number of active connections. Consumer global connection metric.

connection-creation-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: connection-creation-rate: The number of new connections established per second in the window. Consumer global connection metric.

io-ratio

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: io-ratio: The fraction of time the I/O thread spent doing I/O. Consumer global connection metric.

io-time-ns-avg

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: io-time-ns-avg: The average length of time for I/O per select call in nanoseconds. Consumer global connection metric.

io-wait-ratio

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: io-wait-ratio: The fraction of time the I/O thread spent waiting. Consumer global connection metric.

io-wait-time-ns-avg

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: io-wait-time-ns-avg: The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds. Consumer global connection metric.

select-rate

MBean: kafka.consumer:type=consumer-metrics,client-id={clientId} Attribute: select-rate: The number of times the I/O layer checked for new I/O to perform per second. Consumer global connection metric.

Consumer per-broker metrics

incoming-byte-rate

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: incoming-byte-rate: The average number of bytes received per second from the specified broker.

outgoing-byte-rate

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: outgoing-byte-rate: The average number of bytes sent per second to the specified broker.

request-rate

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: request-rate: The average number of requests sent per second to the specified broker.

request-size-avg

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: request-size-avg: The average size of all requests in the window for a specified broker.

request-size-max

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: request-size-max: The maximum size of any request sent in the window for a specified broker.

response-rate

MBean: kafka.consumer:type=consumer-node-metrics,client-id={clientId},node-id=([0-9]+) Attribute: response-rate: The average number of responses received per second from the specified broker.

Consumer group metrics

A consumer group exposes the following metrics. These metrics are attributes on the listed MBean.

assigned-partitions

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: assigned-partitions: The number of partitions currently assigned to this consumer.

commit-latency-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: commit-latency-avg: The average time taken for a commit request.

commit-latency-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: commit-latency-max: The maximum time taken for a commit request.

commit-rate

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: commit-rate: The number of commit calls per second.

commit-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: commit-total: The total number of commit calls.

failed-rebalance-rate-per-hour

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: failed-rebalance-rate-per-hour: The number of failed group rebalance event per hour.

failed-rebalance-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: failed-rebalance-total: The total number of failed group rebalances.

heartbeat-rate

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: heartbeat-rate

The average number of heartbeats per second. After a rebalance, the consumer sends heartbeats to the coordinator to keep itself active in the group. You can control this using the heartbeat.interval.ms setting for the consumer. You may see a lower rate than configured if the processing loop is taking more time to handle message batches. Usually this is OK as long as you see no increase in the join rate.

The heartbeat.interval.ms setting is ignored when group.protocol =consumer, instead use the broker configuration group.consumer.heartbeat.interval.ms to control the heartbeat.

heartbeat-response-time-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: heartbeat-response-time-max: The maximum time taken to receive a response to a heartbeat request.

heartbeat-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: heartbeat-total: The total number of heartbeats.

join-rate

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: join-rate: The number of group joins per second. Group joining is the first phase of the rebalance protocol. A large value indicates that the consumer group is unstable and will likely be coupled with increased lag.

join-time-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: join-time-avg: The average time taken for a group rejoin. This value can get as high as the configured session timeout for the consumer, but should usually be lower.

join-time-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: join-time-max: The maximum time taken for a group rejoin. This value should not get much higher than the configured session timeout for the consumer.

join-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: join-total: The total number of group joins.

last-heartbeat-seconds-ago

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: last-heartbeat-seconds-ago: The number of seconds since the last controller heartbeat.

last-rebalance-seconds-ago

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: last-rebalance-seconds-ago: The number of seconds since the last rebalance event.

partitions-assigned-latency-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-assigned-latency-avg: The average time taken by the on-partitions-assigned rebalance listener callback.

partitions-assigned-latency-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-assigned-latency-max: The maximum time taken by the on-partitions-assigned rebalance listener callback.

partitions-lost-latency-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-lost-latency-avg: The average time taken by the on-partitions-lost rebalance listener callback.

partitions-lost-latency-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-lost-latency-max: The maximum time taken by the on-partitions-lost rebalance listener callback.

partitions-revoked-latency-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-revoked-latency-avg: The average time taken by the on-partitions-revoked rebalance listener callback.

partitions-revoked-latency-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: partitions-revoked-latency-max: The maximum time taken by the on-partitions-revoked rebalance listener callback.

rebalance-latency-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: rebalance-latency-avg: The average time taken for a group rebalance.

rebalance-latency-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: rebalance-latency-max: The maximum time taken for a group rebalance.

rebalance-latency-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: rebalance-latency-total: The total time taken for group rebalances so far.

rebalance-rate-per-hour

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: rebalance-rate-per-hour: The number of group rebalance participated per hour.

rebalance-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: rebalance-total: The total number of group rebalances participated.

sync-rate

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: sync-rate: The number of group syncs per second. Group synchronization is the second and last phase of the rebalance protocol. Similar to join-rate, a large value indicates group instability.

sync-time-avg

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: sync-time-avg: The average time taken for a group sync.

sync-time-max

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: sync-time-max: The maximum time taken for a group sync.

sync-total

MBean: kafka.consumer:type=consumer-coordinator-metrics,client-id={clientId} Attribute: sync-total: The total number of group syncs.