# Kafka Streams Metrics in Confluent Platform Kafka Streams *metrics* are runtime measurements that a Kafka Streams application exposes through Java Management Extensions (JMX), organized in a four-level hierarchy that covers client, stream thread, task, processor node, state store, RocksDB, and record cache activity. You can also configure Kafka Streams applications to report these metrics through pluggable reporters such as Confluent Metrics Reporter. This hierarchy has the following levels: 1. At the top level, there are client-level metrics for each running Kafka Streams client. 2. Each client has *stream threads* with their own metrics. 3. Each stream thread has *tasks* with their own metrics. 4. Each task has a number of *processor nodes* with their own metrics. Also, each task has a number of *state stores* and *record caches*, all of which have their own metrics. The following metrics are available: - [Client metrics](#kafka-streams-metrics-client) - [Thread metrics](#kafka-streams-metrics-thread) - [Task metrics](#kafka-streams-metrics-task) - [Processor node metrics](#kafka-streams-metrics-processor-node) - [State store metrics](#kafka-streams-metrics-state-store) - [RocksDB metrics](#kafka-streams-metrics-rocksdb) - [Record cache metrics](#kafka-streams-metrics-record-cache) For more information, see [Monitor Kafka Streams Applications](monitoring.md#streams-monitoring). ## Client metrics All of the following metrics have a recording level of `info`. ### alive-stream-threads - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The current number of alive stream threads that are running or participating in rebalance. ### application-id - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The application ID of the Kafka Streams client. ### client-state - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+),process-id=([-.\w]+)` The state of the Kafka Streams client as a number, which is the `ordinal()` of the corresponding enum. ### commit-id - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The version control commit ID of the Kafka Streams client. ### failed-stream-threads - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The number of failed stream threads since the Kafka Streams client started. ### recording-level - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+),process-id=([-.\w]+)` The metric recording level, represented as a number. Possible values are: - 0 = INFO - 1 = DEBUG - 2 = TRACE ### state - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The state of the Kafka Streams client, represented as a string. ### topology-description - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The description of the topology executed in the Kafka Streams client. ### version - MBean name: `kafka.streams:type=stream-metrics,client-id=([-.\w]+)` The version of the Kafka Streams client. ## Thread metrics All of the following metrics have a recording level of `info`. ### blocked-time-ns-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total time the Kafka Streams thread has been blocked on Kafka brokers since it was started, in nanoseconds (ns). You can sample this metric periodically and use the difference between samples to measure the time blocked during an interval. This is useful for debugging Kafka Streams application performance, because it gives the proportion of time the application was blocked on Kafka, versus being blocked on processing messages. ### commit-latency-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average execution time for committing across all running tasks of the current thread, in milliseconds (ms). ### commit-latency-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum execution time for committing across all running tasks of the current thread, in milliseconds (ms). ### commit-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of commits per second across all running tasks of the current thread. ### commit-ratio - MBean name: `kafka.consumer:type=consumer-coordinator-metrics,client-id=([-.\w]+)` The fraction of time the thread has spent on committing all tasks. ### commit-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of commit calls across all running tasks of the current thread. ### poll-latency-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average execution time for consumer polling across all running tasks of the current thread, in milliseconds (ms). ### poll-latency-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum execution time for consumer polling across all running tasks of the current thread, in milliseconds (ms). ### poll-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of consumer poll calls per second across all running tasks of the current thread. ### poll-ratio - MBean name: `kafka.consumer:type=consumer-coordinator-metrics,client-id=([-.\w]+)` The fraction of time the thread has spent on polling records from consumer. ### poll-records-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of records polled from consumer within an iteration. ### poll-records-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum number of records polled from consumer within an iteration. ### poll-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of consumer poll calls across all tasks. ### process-latency-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average execution time for processing across all running tasks of the current thread, in milliseconds (ms). ### process-latency-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum execution time for processing across all running tasks of the current thread, in milliseconds (ms). ### process-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of processed records per second across all running tasks of the current thread. ### process-ratio - MBean name: `kafka.streams:type=stream-thread-metrics,client-id=([-.\w]+)` The fraction of time the thread has spent on processing active tasks. ### process-records-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of records processed within an iteration, computed as the total count of processed records over the total number of iterations. ### process-records-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum number of records processed within an iteration. ### process-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of processed records across all tasks. ### punctuate-latency-avg - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average execution time for punctuating across all running tasks of the current thread, in milliseconds (ms). ### punctuate-latency-max - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The maximum execution time for punctuating across all running tasks of the current thread, in milliseconds (ms). ### punctuate-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of punctuate calls per second across all running tasks of the current thread. ### punctuate-ratio - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The fraction of time the thread spends performing punctuating actions on active tasks. ### punctuate-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of punctuate calls across all tasks. ### state - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The state of the thread, represented as a string. ### task-closed-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of tasks closed per second. ### task-closed-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of tasks closed. ### task-created-rate - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The average number of newly-created tasks per second. ### task-created-total - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The total number of newly-created tasks. ### thread-start-time - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+)` The epoch time when the Kafka Streams thread was started, which is useful for computing the processing ratio during the first interval after the thread starts. ### thread-state - MBean name: `kafka.streams:type=stream-thread-metrics,thread-id=([-.\w]+),process-id=([-.\w]+)` The state of the thread as a number, which is the `ordinal()` of the corresponding enum. ## Task metrics All of the following metrics have a recording level of `debug`, except for the `dropped-records-*` and `active-process-ratio` metrics, which have a recording level of `info`. All latency metrics are reported in nanoseconds (ns). ### active-buffer-count - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The count of buffered records that are polled from consumer and not yet processed for the current active task. ### active-process-ratio - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The fraction of time the thread spent on processing this active task among all assigned active tasks. ### cache-size-bytes-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The cache size in bytes accumulated by the current task. ### dropped-records-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of records dropped per second within the current task. ### dropped-records-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of records dropped within the current task. ### enforced-processing-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of enforced processings per second for the current task. ### enforced-processing-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of enforced processings for the current task. ### input-buffer-bytes-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of bytes accumulated by the current task. ### process-latency-avg - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average execution time for processing, in nanoseconds (ns). ### process-latency-max - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The maximum execution time for processing, in nanoseconds (ns). ### process-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of processed records per second across all source processor nodes of the current task. ### process-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of processed records across all source processor nodes of the current task. ### punctuate-latency-avg - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average amount of time taken to execute periodic tasks per call to `punctuate()`, in nanoseconds (ns). ### punctuate-latency-max - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The maximum amount of time taken for any single call to `punctuate()` to complete, in nanoseconds (ns). ### punctuate-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of calls to `punctuate()` per second. ### punctuate-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of times the `punctuate()` method was called to trigger periodic actions during task processing. ### record-lateness-avg - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average observed lateness ([stream time](/platform/current/streams/concepts.html#time) - record timestamp) for the current task, in milliseconds (ms). For more information on out-of-order records, see [Out-of-order handling](/platform/current/streams/concepts.html#out-of-order-handling). ### record-lateness-max - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The maximum observed lateness ([stream time](/platform/current/streams/concepts.html#time) - record timestamp) for the current task, in milliseconds (ms). For more information on out-of-order records, see [Out-of-order handling](/platform/current/streams/concepts.html#out-of-order-handling). ### record-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of records restored per second. ### record-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of records restored. ### update-rate - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The average number of records updated per second. ### update-total - MBean name: `kafka.streams:type=stream-task-metrics,thread-id=([-.\w]+),task-id=([-.\w]+)` The total number of records updated. ## Processor node metrics The following metrics are available only on certain types of nodes. - The `emit-final-*` metrics are available only for windowed aggregations nodes. - The `process-rate` and `process-total` metrics are available only for source processor nodes. - The `suppression-emit-rate` and `suppression-emit-total` metrics are available only for suppression operation nodes. - The `record-e2e-latency-*` metrics are available only for source processor nodes and terminal nodes, which are nodes without successor nodes. All of the metrics have a recording level of `debug`, except for the `record-e2e-latency-*` metrics, which have a recording level of `info`. ### bytes-consumed-total - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The total number of bytes consumed by a source processor node. ### bytes-produced-total - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The total number of bytes produced by a sink processor node. ### emit-final-latency-avg - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The avg latency to emit final records when a record could be emitted, in milliseconds (ms). ### emit-final-latency-max - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The max latency to emit final records when a record could be emitted, in milliseconds (ms). ### emit-final-records-rate - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The rate of records emitted per second when records could be emitted. ### emit-final-records-total - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The total number of records emitted. ### Idempotent-update-skip-rate - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The average number of skipped idempotent updates per second. ### Idempotent-update-skip-total - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The total number of skipped idempotent updates. ### process-rate - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The average number of records processed per second by a source node. ### process-total - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The total number of records processed by a source node. ### record-e2e-latency-avg - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The average end-to-end latency of a record in milliseconds (ms), measured by comparing the record timestamp with the system time when it has been fully processed by the node. ### record-e2e-latency-max - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The maximum end-to-end latency of a record in milliseconds (ms), measured by comparing the record timestamp with the system time when it has been fully processed by the node. ### record-e2e-latency-min - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The minimum end-to-end latency of a record in milliseconds (ms), measured by comparing the record timestamp with the system time when it has been fully processed by the node. ### records-consumed-total - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The total number of records consumed by a source processor node. ### records-produced-total - MBean name: `kafka.streams:type=stream-topic-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+),topic=([-.\w]+)` The total number of records produced by a sink processor node. ### suppression-emit-rate - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The rate of records emitted per second that have been emitted downstream from suppression operation nodes. Compare with the `process-rate` metric to determine how many updates are being suppressed. ### suppression-emit-total - MBean name: `kafka.streams:type=stream-processor-node-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)` The total number of records that have been emitted downstream from suppression operation nodes. Compare with the `process-total` metric to determine how many updates are being suppressed. ## State store metrics All of the following metrics have a recording level of `debug`, except for the `record-e2e-latency-*` metrics, which have a recording level of `trace`, and `num-open-iterators`, which has a recording level of `info`. The `store-scope` value is specified in `StoreSupplier#metricsScope()` for your customized state stores. For built-in state stores, currently Kafka Streams supports the following metrics: * `in-memory-state` * `in-memory-lru-state` * `in-memory-window-state` * `in-memory-suppression`: for suppression buffers * `rocksdb-state`: for RocksDB backed key-value store * `rocksdb-window-state`: for RocksDB backed window store * `rocksdb-session-state`: for RocksDB backed session store Metrics `suppression-buffer-size-avg`, `suppression-buffer-size-max`, `suppression-buffer-count-avg`, and `suppression-buffer-count-max` are available only for suppression buffers. All other metrics are not available for suppression buffers. All latency metrics are reported in nanoseconds (ns). ### all-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time for the all operation, from iterator create to close, in nanoseconds (ns). ### all-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time for the all operation, from iterator create to close, in nanoseconds (ns). ### all-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average all operation rate per second for the current store. ### delete-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time of delete operations, in nanoseconds (ns). ### delete-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time of a delete operation, in nanoseconds (ns). ### delete-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of delete operations per second for the current store. ### flush-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time for flush operations in nanoseconds (ns). ### flush-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time for flush operations in nanoseconds (ns). ### flush-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of flush operations per second for the current store. ### get-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time of get operations, in nanoseconds (ns). ### get-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time of a get operation, in nanoseconds (ns). ### get-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of get operations per second for the current store. ### iterator-duration-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average time spent between creating and closing an iterator, in nanoseconds. ### iterator-duration-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum time spent between creating and closing an iterator, in nanoseconds. ### num-open-iterators - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The current number of iterators on the store that have been created but not yet closed. ### oldest-iterator-open-since-ms - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The UNIX timestamp since the oldest still-open iterator was created, in milliseconds. ### prefix-scan-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time for the prefix-scan operation, from iterator create to close, in nanoseconds (ns). ### prefix-scan-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time for the prefix-scan operation, from iterator create to close, in nanoseconds (ns). ### prefix-scan-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of prefix scan operations per second for the current store. ### put-all-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time of put-all operations, in nanoseconds (ns). ### put-all-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time of a put-all operation, in nanoseconds (ns). ### put-all-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of put-all operations per second for the current store. ### put-if-absent-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time of a put-if-absent operation, in nanoseconds (ns). ### put-if-absent-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time of a put-if-absent operation, in nanoseconds (ns). ### put-if-absent-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of put-if-absent operations per second for the current store. ### put-if-absent-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of put-if-absent operations for the current store. ### put-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time of a put operation, in nanoseconds (ns). ### put-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time of a put operation, in nanoseconds (ns). ### put-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of put operations per second for the current store. ### range-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time for the range operation, from iterator create to close, in nanoseconds (ns). ### range-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time for the range operation, from iterator create to close, in nanoseconds (ns). ### range-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of range operations per second for the current store. ### record-e2e-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average end-to-end latency of a record, in nanoseconds (ns), measured by comparing its timestamp with the system time when fully processed by the store. ### record-e2e-latency-min - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The minimum end-to-end latency of a record, in nanoseconds (ns), measured by comparing its timestamp with the system time when fully processed by the store. ### record-e2e-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum end-to-end latency of a record, in nanoseconds (ns), measured by comparing its timestamp with the system time when fully processed by the store. ### restore-latency-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average execution time for restore operations in nanoseconds (ns). ### restore-latency-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum execution time for restore operations in nanoseconds (ns). ### restore-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of restore operations per second for the current store. ### suppression-buffer-count-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),in-memory-suppression-id=([-.\w]+)` The average number of records buffered over the sampling window. This metric may be useful for choosing a value for `BufferConfig.maxRecords(...)`. ### suppression-buffer-count-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),in-memory-suppression-id=([-.\w]+)` The maximum number of records buffered over the sampling window. This metric may be useful for choosing a value for `BufferConfig.maxRecords(...)`. ### suppression-buffer-size-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),in-memory-suppression-id=([-.\w]+)` The average total size of the buffered data over the sampling window, in bytes. This metric may be useful for choosing a value for `BufferConfig.maxBytes(...)`. ### suppression-buffer-size-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),in-memory-suppression-id=([-.\w]+)` The maximum total size of the buffered data over the sampling window, in bytes. This metric may be useful for choosing a value for `BufferConfig.maxBytes(...)`. #### State store metrics changes in Confluent Platform 8.3 The following state store metrics are new in Confluent Platform 8.3 (Kafka Streams 4.3): - `commit-rate`: The average number of commit operations per second. Replaces the deprecated `flush-rate` metric. - `commit-latency-avg`: The average execution time for commit operations in nanoseconds. Replaces the deprecated `flush-latency-avg` metric. - `commit-latency-max`: The maximum execution time for commit operations in nanoseconds. Replaces the deprecated `flush-latency-max` metric. - `num-keys`: The number of keys in an in-memory state store. This metric provides an exact count, unlike `estimate-num-keys` for RocksDB stores which provides an approximation. Available only for non-persistent (in-memory) state stores. The MBean is registered under the store-type tag that matches the in-memory variant: | Store variant | MBean `store-type` tag | |---------------------------|--------------------------| | In-memory key-value store | `in-memory` | | In-memory window store | `in-memory-window` | | In-memory session store | `in-memory-session` | Example MBean name for an in-memory window store: `kafka.streams:type=stream-state-metrics,thread-id=,task-id=,in-memory-window-state-id=`. The `flush-rate`, `flush-latency-avg`, and `flush-latency-max` metrics are deprecated. Use the corresponding `commit-*` metrics instead. For more information, see [KIP-1035](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1035%3A+StateStore+Managed+Changelog+Offsets) and [KIP-1250](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1250%3A+Add+metric+to+track+size+of+in-memory+state+stores). ## RocksDB metrics RocksDB metrics are grouped into statistics-based metrics and properties-based metrics. - Statistics-based metrics are recorded from statistics that a RocksDB state store collects. Statistics collected by RocksDB provide cumulative measurements over time, for example, bytes written to the state store. - Properties-based metrics are recorded from properties that RocksDB exposes. Properties exposed by RocksDB provide current measurements, for example, the amount of memory currently used. The built-in RocksDB state stores have these values for `store-scope`: * `rocksdb-state`: for RocksDB-backed key-value stores * `rocksdb-window-state`: for RocksDB-backed window stores * `rocksdb-session-state`: for RocksDB-backed session stores **RocksDB Statistics-based Metrics:** All of the following metrics have a recording level of `debug`, because collecting statistics in [RocksDB may have an impact on performance](https://github.com/facebook/rocksdb/wiki/Statistics#stats-level-and-performance-costs). Statistics-based metrics are collected every minute from the RocksDB state stores. If a state store consists of multiple RocksDB instances, which is the case for WindowStores and SessionStores, each metric reports an aggregation over the RocksDB instances of the state store. ### block-cache-data-hit-ratio - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The ratio of block cache hits for data blocks, relative to all lookups for data blocks to the block cache. ### block-cache-index-hit-ratio - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The ratio of block cache hits for index blocks, relative to all lookups for index blocks to the block cache. ### block-cache-filter-hit-ratio - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The ratio of block cache hits for filter blocks, relative to all lookups for filter blocks to the block cache. ### bytes-written-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of bytes written per second to the RocksDB state store. ### bytes-written-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of bytes written to the RocksDB state store. ### bytes-read-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of bytes read per second from the RocksDB state store. ### bytes-read-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of bytes read from the RocksDB state store. ### bytes-read-compaction-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of bytes read per second during compaction. ### bytes-written-compaction-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of bytes written per second during compaction. ### compaction-time-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average duration of disk compactions, in milliseconds (ms). ### compaction-time-min - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The minimum duration of disk compactions, in milliseconds (ms). ### compaction-time-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum duration of disk compactions, in milliseconds (ms). ### memtable-bytes-flushed-rate - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average number of bytes flushed per second from the memtable to disk. ### memtable-bytes-flushed-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of bytes flushed from the memtable to disk. ### memtable-hit-ratio - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The ratio of memtable hits, relative to all lookups to the memtable. ### memtable-flush-time-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average duration of memtable flushes to disk, in milliseconds (ms). ### memtable-flush-time-min - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The minimum duration of memtable flushes to disk, in milliseconds (ms). ### memtable-flush-time-max - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The maximum duration of memtable flushes to disk, in milliseconds (ms). ### number-open-files - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of current open files. This metric returns constant -1, because the RocksDB counter NO_FILE_CLOSES was removed in RocksDB version 9.7.3. ### number-file-errors-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of file errors that occurred. ### write-stall-duration-avg - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The average duration of write stalls, in milliseconds (ms). ### write-stall-duration-total - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total duration of write stalls, in milliseconds (ms). **RocksDB Properties-based Metrics:** All of the following properties-based metrics have a recording level of `info` and are recorded when the metrics are accessed. If a state store consists of multiple RocksDB instances, which is the case for WindowStores and SessionStores, each metric reports the sum over all the RocksDB instances of the state store, except for the block cache metrics, named `block-cache-*`. The block cache metrics report the sum over all RocksDB instances if each instance uses its own block cache, and they report the recorded value from only one instance if a single block cache is shared among all instances. ### background-errors - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total number of background errors. ### block-cache-capacity - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The capacity of the block cache, in bytes. ### block-cache-pinned-usage - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The memory size for the entries being pinned in the block cache, in bytes. ### block-cache-usage - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The memory size of the entries residing in block cache, in bytes. ### compaction-pending - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` This metric reports `1` if at least one compaction is pending, otherwise it reports `0`. ### cur-size-active-mem-table - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The approximate size of the active memtable, in bytes. ### cur-size-all-mem-tables - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The approximate size of active and unflushed immutable memtables, in bytes. ### estimate-num-keys - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The estimated number of keys in the active and unflushed immutable memtables and storage. ### estimate-pending-compaction-bytes - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The estimated total number of bytes a compaction needs to rewrite on disk to get all levels down to under target size. Valid only for level compaction. ### estimate-table-readers-mem - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The estimated memory, in bytes, used for reading Sorted Sequence Tables (SSTs), excluding memory used in block cache. ### live-sst-files-size - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total size in bytes of all Sorted Sequence Table (SST) files that belong to the latest log-structured merge (LSM) tree. ### mem-table-flush-pending - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` This metric reports `1` if a memtable flush is pending, otherwise it reports `0`. ### num-deletes-active-mem-table - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of delete entries in the active memtable. ### num-deletes-imm-mem-tables - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of delete entries in the unflushed immutable memtables. ### num-entries-active-mem-table - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of entries in the active memtable. ### num-entries-imm-mem-tables - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of entries in the unflushed immutable memtables. ### num-immutable-mem-table - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of immutable memtables that have not yet been flushed. ### num-live-versions - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` Number of live versions of the log-structured merge (LSM) tree. ### num-running-compactions - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of currently running compactions. ### num-running-flushes - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The number of currently running flushes. ### size-all-mem-tables - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The approximate size of active, unflushed immutable, and pinned immutable memtables, in bytes. ### total-sst-files-size - MBean name: `kafka.streams:type=stream-state-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)` The total size in bytes of all Sorted Sequence Table (SST) files. ## Record cache metrics All of the following metrics have a recording level of `debug`. ### hit-ratio-avg - MBean name: `kafka.streams:type=stream-record-cache-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),record-cache-id=([-.\w]+)` The average cache hit ratio, defined as the ratio of cache read hits over the total cache read requests. ### hit-ratio-max - MBean name: `kafka.streams:type=stream-record-cache-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),record-cache-id=([-.\w]+)` The maximum cache hit ratio, defined as the ratio of cache read hits over the total cache read requests. ### hit-ratio-min - MBean name: `kafka.streams:type=stream-record-cache-metrics,thread-id=([-.\w]+),task-id=([-.\w]+),record-cache-id=([-.\w]+)` The minimum cache hit ratio, defined as the ratio of cache read hits over the total cache read requests. ## Related content - [Monitor Kafka Streams Applications](monitoring.md#streams-monitoring) #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).