Share Consumer Metrics

Broker metrics

The following broker metrics are available for monitoring share consumer groups and share coordination.

Group coordinator metrics

These metrics track share group management at the group coordinator level.

group-count

MBean: kafka.server:type=group-coordinator-metrics,protocol=share,name=group-count

The total number of share groups managed by the group coordinator.

share-group-rebalance-count

MBean: kafka.server:type=group-coordinator-metrics,protocol=share,name=share-group-rebalance-count

The total number of share group rebalances.

share-group-rebalance-rate

MBean: kafka.server:type=group-coordinator-metrics,protocol=share,name=share-group-rebalance-rate

The rate of share group rebalances per second.

share-group-count

MBean: kafka.server:type=group-coordinator-metrics,state={Empty|Stable|Dead},name=share-group-count

The number of share groups in the respective state (Empty, Stable, or Dead).

Share coordinator metrics

These metrics track share partition coordination and state management.

partition-load-time-avg

MBean: kafka.server:type=share-coordinator-metrics,name=partition-load-time-avg

The average time taken in milliseconds to load the share-group state from the share-group state partitions.

partition-load-time-max

MBean: kafka.server:type=share-coordinator-metrics,name=partition-load-time-max

The maximum time taken in milliseconds to load the share-group state from the share-group state partitions.

thread-idle-ratio-min

MBean: kafka.server:type=share-coordinator-metrics,name=thread-idle-ratio-min

The minimum fraction of time the share coordinator thread is idle.

thread-idle-ratio-avg

MBean: kafka.server:type=share-coordinator-metrics,name=thread-idle-ratio-avg

The average fraction of time the share coordinator thread is idle.

write-rate

MBean: kafka.server:type=share-coordinator-metrics,name=write-rate

The number of share-group state write calls per second.

write-total

MBean: kafka.server:type=share-coordinator-metrics,name=write-total

The total number of share-group state write calls.

write-latency-avg

MBean: kafka.server:type=share-coordinator-metrics,name=write-latency-avg

The average time taken for a share-group state write call, including the time to write to the share-group state topic.

write-latency-max

MBean: kafka.server:type=share-coordinator-metrics,name=write-latency-max

The maximum time taken for a share-group state write call, including the time to write to the share-group state topic.

num-partitions

MBean: kafka.server:type=share-coordinator-metrics,state={loading|active|failed},name=num-partitions

The number of partitions in the share-group state topic in the respective state (loading, active, or failed).

last-pruned-offset

MBean: kafka.server:type=share-coordinator-metrics,topic={topic-name},partition={partition-index},name=last-pruned-offset

The offset at which the share-group state topic was last pruned for the specified topic and partition.

Broker topic metrics

These metrics track share fetch and acknowledgement requests at the broker topic level.

TotalShareFetchRequestsPerSec

MBean: kafka.server:type=BrokerTopicMetrics,name=TotalShareFetchRequestsPerSec,topic={topic}

The share fetch request rate per second. Similar to TotalFetchRequestsPerSec for consumer fetches.

FailedShareFetchRequestsPerSec

MBean: kafka.server:type=BrokerTopicMetrics,name=FailedShareFetchRequestsPerSec,topic={topic}

The share fetch request rate for requests that failed. Similar to existing fetch metric FailedFetchRequestsPerSec for Fetch, which record all topic cumulative and individual topic stats.

TotalShareAcknowledgementRequestsPerSec

MBean: kafka.server:type=BrokerTopicMetrics,name=TotalShareAcknowledgementRequestsPerSec,topic={topic}

The acknowledgement request rate per second. Similar to above defined total fetch metrics, this metric tracks the number of share acknowledgement requests stats.

FailedShareAcknowledgementRequestsPerSec

MBean: kafka.server:type=BrokerTopicMetrics,name=FailedShareAcknowledgementRequestsPerSec,topic={topic}

The share acknowledgement request rate for requests that failed. Similar to above defined failed fetch metrics, this metric tracks the number of failed share acknowledgement requests stats.

Share group metrics

These metrics track share group operations and partition management.

RecordAcknowledgementsPerSec

MBean: kafka.server:type=ShareGroupMetrics,name=RecordAcknowledgementsPerSec,ackType={Accept|Release|Reject}

The rate per second of records acknowledged per acknowledgement type. Tracks the rate of records acknowledged per acknowledgement type.

PartitionLoadTimeMs

MBean: kafka.server:type=ShareGroupMetrics,name=PartitionLoadTimeMs

The time taken to load the share partitions. Tracks the time to load the share partition. Elevated values may indicate prolonged initialization and warrant further investigation.

RequestTopicPartitionsFetchRatio

MBean: kafka.server:type=ShareGroupMetrics,name=RequestTopicPartitionsFetchRatio,group={group_id}

The ratio of topic-partitions acquired to the total number of topic-partitions in share fetch request.

TopicPartitionsAcquireTimeMs

MBean: kafka.server:type=ShareGroupMetrics,name=TopicPartitionsAcquireTimeMs,group={group_id}

The time elapsed (in millisecond) to acquire any topic partition for fetch. Tracks the time to acquire a topic partition for share fetches. High values suggest topic-partitions contention.

Share partition metrics

These metrics track share partition-level operations and resource usage.

AcquisitionLockTimeoutPerSec

MBean: kafka.server:type=SharePartitionMetrics,name=AcquisitionLockTimeoutPerSec,group={group_id},topic={topic_name},partition={partition}

The rate of acquisition locks for records which are not acknowledged within the timeout. Tracks the rate of acquisition locks for records which are not acknowledged within the timeout. Timeout is set by group.share.record.lock.duration.ms.

InFlightMessageCount

MBean: kafka.server:type=SharePartitionMetrics,name=InFlightMessageCount,group={group_id},topic={topic_name},partition={partition} The current number of in-flight messages for the share partition. A share partition cannot be acquired for additional fetches when this reaches the limit set by group.share.partition.max.record.locks.

InFlightBatchCount

MBean: kafka.server:type=SharePartitionMetrics,name=InFlightBatchCount,group={group_id},topic={topic_name},partition={partition}

The number of in-flight batches for the share partition. Specifies the number of batches currently being processed by a share partition. This metric provides insights into memory consumption and performance bottlenecks, aiding in optimization efforts.

InFlightBatchMessageCount

MBean: kafka.server:type=SharePartitionMetrics,name=InFlightBatchMessageCount,group={group_id},topic={topic_name},partition={partition}

The distribution of message counts across in-flight batches for the share partition. This histogram provides insights into batch size patterns for memory consumption analysis.

FetchLockTimeMs

MBean: kafka.server:type=SharePartitionMetrics,name=FetchLockTimeMs,group={group_id},topic={topic_name},partition={partition}

The time elapsed (in milliseconds) while a share partition is held under lock for fetching messages. Tracks the duration of fetch lock acquisitions.

FetchLockRatio

MBean: kafka.server:type=SharePartitionMetrics,name=FetchLockRatio,group={group_id},topic={topic_name},partition={partition}

The fraction of time share partition is held under lock. Measures the proportion of time a share partition spends under lock for fetching messages. A low ratio suggests that the partition is not actively involved in fetching data.

Share session cache metrics

These metrics track share session cache performance.

ShareSessionEvictionsPerSec

MBean: kafka.server:type=ShareSessionCache,name=ShareSessionEvictionsPerSec

The share session eviction rate per second. Similar to IncrementalFetchSessionEvictionsPerSec.

SharePartitionsCount

MBean: kafka.server:type=ShareSessionCache,name=SharePartitionsCount

The number of cached share partitions. Similar to NumIncrementalFetchPartitionsCached.

ShareSessionsCount

MBean: kafka.server:type=ShareSessionCache,name=ShareSessionsCount

The number of cached share sessions. Similar to NumIncrementalFetchSessions.

Delayed share fetch metrics

These metrics track delayed share fetch operations.

ExpiresPerSec

MBean: kafka.server:type=DelayedShareFetchMetrics,name=ExpiresPerSec

The expired delayed share fetch operation rate per second. Similar to ExpiresPerSec for other existing delayed operations i.e. DelayedFetch, etc.

Delayed operation purgatory metrics

These metrics track share fetch requests in the purgatory.

PurgatorySize

MBean: kafka.server:type=DelayedOperationPurgatory,delayedOperation=ShareFetch,name=PurgatorySize

The number of requests waiting in the share fetch purgatory. This is high if share consumers use a large value for fetch.wait.max.ms. Add ShareFetch to existing purgatory metrics.

NumDelayedOperations

MBean: kafka.server:type=DelayedOperationPurgatory,delayedOperation=ShareFetch,name=NumDelayedOperations

The number of delayed share fetch operations. Add ShareFetch to existing purgatory metrics.

Client metrics

The following client metrics are available for monitoring share consumer applications.

Share consumer metrics

These metrics track the overall performance and behavior of share consumer clients.

last-poll-seconds-ago

MBean: kafka.consumer:type=consumer-share-metrics,client-id=([-.w]+),name=last-poll-seconds-ago

The number of seconds since the last poll() invocation.

time-between-poll-avg

MBean: kafka.consumer:type=consumer-share-metrics,client-id=([-.w]+),name=time-between-poll-avg

The average delay between invocations of poll() in milliseconds.

time-between-poll-max

MBean: kafka.consumer:type=consumer-share-metrics,client-id=([-.w]+),name=time-between-poll-max

The maximum delay between invocations of poll() in milliseconds.

poll-idle-ratio-avg

MBean: kafka.consumer:type=consumer-share-metrics,client-id=([-.w]+),name=poll-idle-ratio-avg

The average fraction of time the consumer’s poll() is idle as opposed to waiting for user code to process records.

Share coordinator metrics

These metrics track share consumer coordination activities, including heartbeats and rebalancing.

heartbeat-response-time-max

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=heartbeat-response-time-max

The maximum time taken to receive a response to a heartbeat request in milliseconds.

heartbeat-rate

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=heartbeat-rate

The number of heartbeats per second.

heartbeat-total

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=heartbeat-total

The total number of heartbeats.

last-heartbeat-seconds-ago

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=last-heartbeat-seconds-ago

The number of seconds since the last coordinator heartbeat was sent.

rebalance-total

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=rebalance-total

The total number of rebalance events.

rebalance-rate-per-hour

MBean: kafka.consumer:type=consumer-share-coordinator-metrics,client-id=([-.w]+),name=rebalance-rate-per-hour

The number of rebalance events per hour.

Share fetch manager metrics

These metrics track fetch operations, record consumption, and acknowledgements for share consumers.

fetch-size-avg

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-size-avg

The average number of bytes fetched per request.

fetch-size-max

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-size-max

The maximum number of bytes fetched per request.

bytes-consumed-rate

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=bytes-consumed-rate

The average number of bytes fetched per second.

bytes-consumed-total

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=bytes-consumed-total

The total number of bytes fetched.

records-per-request-avg

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=records-per-request-avg

The average number of records in each request. When set to batch_optimized (default), this value may exceed max.poll.records to align with batch boundaries. When share.acquire.mode is set to record_limit, this value will not exceed max.poll.records.

records-per-request-max

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=records-per-request-max

The maximum number of records in a request. When set to batch_optimized (default), this value may exceed max.poll.records to align with batch boundaries. When share.acquire.mode is set to record_limit, this value will not exceed max.poll.records.

records-consumed-rate

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=records-consumed-rate

The average number of records fetched per second.

records-consumed-total

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=records-consumed-total

The total number of records fetched.

acknowledgements-send-rate

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=acknowledgements-send-rate

The average number of record acknowledgements sent per second.

acknowledgements-send-total

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=acknowledgements-send-total

The total number of record acknowledgements sent.

acknowledgements-error-rate

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=acknowledgements-error-rate

The average number of record acknowledgements that resulted in errors per second.

acknowledgements-error-total

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=acknowledgements-error-total

The total number of record acknowledgements that resulted in errors.

fetch-latency-avg

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-latency-avg

The average time taken for a fetch request in milliseconds.

fetch-latency-max

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-latency-max

The maximum time taken for any fetch request in milliseconds.

fetch-rate

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-rate

The number of fetch requests per second.

fetch-total

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-total

The total number of fetch requests.

fetch-throttle-time-avg

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-throttle-time-avg

The average throttle time in milliseconds.

fetch-throttle-time-max

MBean: kafka.consumer:type=consumer-share-fetch-manager-metrics,client-id=([-.w]+),name=fetch-throttle-time-max

The maximum throttle time in milliseconds.