# Log and Network Metrics This topic describes JMX metrics for Kafka logs and network components. These metrics are useful for monitoring storage and network performance in your Kafka cluster. For information about how to configure JMX, see [Configure JMX for Monitoring](configure-jmx.md#configure-jmx). ## Search for a metric

## Log metrics The following metrics are available for monitoring logs, log cleaners, and log cleaner managers. A log directory is a directory on disk that contains one or more Kafka log segments. When a Kafka broker starts up, it registers all of the log directories that it finds on its local disk with the log manager. The log manager is responsible for managing the creation, deletion, and cleaning of log segments across all registered log directories. ### cleaner-recopy-percent MBean: `kafka.log:type=LogCleaner,name=cleaner-recopy-percent` : A metric to track the recopy rate of each thread’s last cleaning ### compacted-partition-bytes MBean: `kafka.log:type=LogCleanerManager,name=compacted-partition-bytes` : A gauge metric that provides compacted data in each partition. ### compacted-partition-local-bytes MBean: `kafka.log:type=LogCleanerManager,name=compacted-partition-local-bytes` : A gauge metric that provides local compacted data in each partition. Confluent Server only. ### compacted-partition-tiered-bytes MBean: `kafka.log:type=LogCleanerManager,name=compacted-partition-tiered-bytes` : A gauge metric that provides compacted data in each partition when using tiered storage. Confluent Server only. ### DeadThreadCount MBean: `kafka.log:type=LogCleaner,name=DeadThreadCount` : Provides the number of dead threads for the process. ### LogDirectoryOffline MBean: `kafka.log:logDirectory="{/log-directory}":type=LogManager,name=LogDirectoryOffline` : A metric that indicates if a log directory is offline (1) or online (0). ### LogEndOffset MBean: `kafka.log:type=Log,name=LogEndOffset` : The offset of the last message in a partition. Use with `LogStartOffset` to calculate the current message count for a topic. ### LogFlushRateAndTimeMs MBean: `kafka.log:type=LogFlushStats,name=LogFlushRateAndTimeMs` : Log flush rate and time in milliseconds. ### LogStartOffset MBean: `kafka.log:type=Log,name=LogStartOffset` : The offset of the first message in a partition. Use with `LogEndOffset` to calculate the current message count for a topic. ### max-buffer-utilization-percent MBean: `kafka.log:type=LogCleaner,name=max-buffer-utilization-percent` : A metric to track the maximum utilization of any thread’s buffer in the last cleaning. ### max-clean-time-secs MBean: `kafka.log:type=LogCleaner,name=max-clean-time-secs` : A metric to track the maximum cleaning time for the last cleaning from each thread. ### max-compaction-delay-secs MBean: `kafka.log:type=LogCleaner,name=max-compaction-delay-secs` : A metric that provides the maximum delay, in seconds, that the log cleaner will wait before compacting a log segment. ### max-dirty-percent MBean: `kafka.log:type=LogCleanerManager,name=max-dirty-percent` : A gauge metric that provides the maximum percentage of the log segment that can contain dirty messages before the log cleaner will begin cleaning it. ### NumLogSegments MBean: `kafka.log:type=Log:name=NumLogSegments` : The number of log segments that currently exist for a given partition. ### OfflineLogDirectoryCount MBean: `kafka.log:type=LogManager,name=OfflineLogDirectoryCount` : A metric that describes the number of log directories that are registered with the log manager, but are currently offline. ### OffsetIndexAppendTimeMs MBean: `kafka.log:type=SegmentStats,name=OffsetIndexAppendTimeMs` : The time in milliseconds to append an entry to the log segment offset index. The offset index maps from logical offsets to physical file positions. Available on Confluent Server only. ### remainingLogsToRecover MBean: `kafka.log:type=LogManager,name=remainingLogsToRecover` : The number of remaining logs for each log directory to be recovered. This metric provides an overview of the recovery progress for a given log directory. ### remainingSegmentsToRecover MBean: `kafka.log:type=LogManager,name=remainingSegmentsToRecover` : The number of remaining segments assigned to the currently active recovery thread. ### SegmentAppendTimeMs MBean: `kafka.log:type=SegmentStats,name=SegmentAppendTimeMs` : The time in milliseconds to append a record to the log segment. Available on Confluent Server only. ### Size MBean: `kafka.log:type=Log,name=Size` : A metric for the total size in bytes of all log segments that belong to a given partition. ### time-since-last-run-ms MBean: `kafka.log:type=LogCleanerManager,name=time-since-last-run-ms` : A gauge metric that provides the time, in milliseconds, since the log cleaner last ran. ### TimestampIndexAppendTimeMs MBean: `kafka.log:type=SegmentStats,name=TimestampIndexAppendTimeMs` : The time in milliseconds to append an entry to the log segment timestamp index. Available on Confluent Server only. ### uncleanable-bytes MBean: `kafka.log:logDirectory="{/log-directory}":type=LogCleanerManager:name=uncleanable-bytes` : A metric that provides information about the total number of bytes in log segments for a specified log directory that cannot be cleaned by the log cleaner due to retention policy or other configuration settings. ### uncleanable-partitions-count MBean: `kafka.log:logDirectory="{/log-directory}":type=LogCleanerManager:name=uncleanable-partitions-count` : A gauge metric that tracks the number of partitions marked as uncleanable for each log directory. ## Network metrics Kafka brokers can generate a lot of network traffic because they collect and distribute data for processing. You can use the following metrics to help evaluate whether your Kafka deployment is communicating efficiently. ### DeprecatedRequestsPerSec

MBean: kafka.network:type=RequestMetrics,name=DeprecatedRequestsPerSec,request={apiName},
version={apiVersion},clientSoftwareName={clientSoftwareName},
clientSoftwareVersion={clientSoftwareVersion}

The request rate for deprecated clients in milliseconds. Provides the request API, request API version, client software name and client software version. This is an important metric to track because support for deprecated clients will be removed in Confluent Platform 8.0. Note that this metric is only exposed if a client has used a deprecated API since the server’s last restart. ### ErrorsPerSec

MBean: kafka.network:type=RequestMetrics,name=ErrorsPerSec,request={requestType},
error={error-code}

Indicates the number of errors in responses counted per-request-type, per-error-code. If a response contains multiple errors, all are counted. error=NONE indicates successful responses. Available for request types: `InitProducerId`, `Metadata`, `ApiVersions`, `DescribeCluster`, `Fetch`, `DescribeConfigs`, `FindCoordinator`, `ListGroups`, `CreateTopics`, `UpdateMetadata`, `ListOffsets`, `LeaderAndIsr`, `Produce`, `AllocateProducerIds`. ### ExpiredConnectionsKilledCount MBean: `kafka.network:type=SocketServer,name=ExpiredConnectionsKilledCount` : The total number of connections disconnected, across all processors, due to a client not re-authenticating and then using the connection beyond its expiration time for anything other than re-authentication. Ideally 0 when re-authentication is enabled, implying there are no longer any older, pre-2.2.0 clients connecting to this broker ### LocalTimeMs MBean: `kafka.network:type=RequestMetrics,name=LocalTimeMs,request={Produce|FetchConsumer|FetchFollower}` : Time in milliseconds the request is processed at the leader. ### MessageConversionsTimeMs MBean: `kafka.network:type=RequestMetrics,name=MessageConversionsTimeMs,request={Produce|Fetch}` : Time in milliseconds spent on message format conversions. ### NetworkProcessorAvgIdlePercent MBean: `kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent` : The average fraction of time the network processor threads are idle. Values are between `0` (all resources are used) and `1` (all resources are available). Ideally this is greater than 0.4. ### RemoteTimeMs MBean: `kafka.network:type=RequestMetrics,name=RemoteTimeMs,request={requestType}` : Time in milliseconds the request waits for the follower. This is non-zero for produce requests when `ack=-1` or `acks=all`. Available for: `Produce`, `FetchConsumer`, `FetchFollower`, `TxnOffsetCommit`. ### RequestBytes MBean: `kafka.network:type=RequestMetrics,name=RequestBytes,request={requestType}` : Request size in bytes. Available for request type: `OffsetCommit`. ### RequestQueueSize MBean: `kafka.network:type=RequestChannel,name=RequestQueueSize` : Size of the request queue. A congested request queue will not be able to process incoming or outgoing requests. ### RequestQueueTimeMs MBean: `kafka.network:type=RequestMetrics,name=RequestQueueTimeMs,request={Produce|FetchConsumer|FetchFollower}` : Time in milliseconds that the request waits in the request queue. ### RequestsPerSec

MBean: kafka.network:type=RequestMetrics,name=RequestsPerSec,
request={requestType},version=([0-9]+)

The request rate. `version` refers to the API version of the request type. To get the total count for a specific request type, make sure that JMX monitoring tools aggregate across different versions. Available for request types: `Produce`, `FetchConsumer`, `FetchFollower`, `DescribeCluster`, `ListOffsets`, `Metadata`, `UpdateMetadata`, `LeaderAndIsr`, `Fetch`. ### ResponseQueueSize MBean: `kafka.network:type=RequestChannel,name=ResponseQueueSize` : Size of the response queue. The response queue is unbounded. A congested response queue can result in delayed response times and memory pressure on the broker. ### ResponseSendTimeMs

MBean: kafka.network:type=RequestMetrics,name=ResponseSendTimeMs,
request={Produce|FetchConsumer|FetchFollower}

Time to send the response. ### TemporaryMemoryBytes MBean: `kafka.network:type=RequestMetrics,name=TemporaryMemoryBytes,request={RequestType}` : Temporary memory, in bytes, used for message format conversions and decompression. Available for request types: `Produce`, `Fetch`. ### ThrottleTimeMs MBean: `kafka.network:type=RequestMetrics,name=ThrottleTimeMs,request={RequestType}` The time in milliseconds that requests are throttled due to quota enforcement. Available for request types: `Produce`, `Fetch`, `RequestQuotas`, `OffsetFetch`. ### TotalTimeMs

MBean: kafka.network:type=RequestMetrics,name=TotalTimeMs,
request={requestType}

The total time in milliseconds to serve the specified request. Available for request types: `DescribeCluster`, `JoinGroup`, `LeaveGroup`, `ListOffsets`, `Metadata`, `OffsetCommit`, `AddOffsetsToTxn`, `AddPartitionsToTxn`, `DescribeTransactions`, `EndTxn`, `ListTransactions`, `WriteTxnMarkers`, `UpdateFeatures`, `LeaderAndIsr`. ## Related content - [Monitor Kafka with JMX in Confluent Platform](monitoring.md#kafka-monitoring) - [Configure JMX for Monitoring](configure-jmx.md#configure-jmx) - [Broker and Controller Metrics](broker-metrics.md#broker-metrics) - [Group Coordinator Metrics](group-coordinator-metrics.md#group-coordinator-metrics) - [Producer Metrics](producer-metrics.md#producer-metrics) - [Consumer Metrics](consumer-metrics.md#consumer-metrics) - [Security Metrics](security-metrics.md#security-metrics)