Monitor Schema Registry in Confluent Platform

Schema Registry reports a variety of metrics through Java Management Extensions (JMX). You can also configure it to report statistics through additional pluggable reporters by using the metric.reporters configuration option. The easiest way to view the available metrics is to browse the JMX MBeans in JConsole.

Schema Registry has two types of metrics. Global metrics help you monitor the overall health of the service. Per-endpoint metrics monitor each API endpoint request method and are prefixed with the endpoint name, for example subjects.get-schema. Per-endpoint metrics help you understand how the proxy is used and track down specific performance problems.

Tip

Kafka disables authentication for JMX by default. To override these defaults, use environment variables. For each platform component, the options to override the JMX defaults take the form <component-name>_JMX_OPTS. For Schema Registry, use SCHEMA_REGISTRY_JMX_OPTS.

Global Metrics

MBean: kafka.schema.registry:type=jetty-metrics

connections-active

Total number of active TCP connections.

connections-accepted-rate (deprecated since 2.0)

In 1.x: The average rate per second of accepted TCP connections.
In 2.x: Same as connections-opened-rate.

connections-opened-rate

The average rate per second of opened TCP connections.

connections-closed-rate

The average rate per second of closed TCP connections.

busy_thread_count

The number of threads currently working on a task (busy threads).

thread_pool_usage

Percentage of total thread pool in use (total/threads in use).

request_queue_size

Number of threads required for requests in the queue.

MBean: kafka.schema.registry:type=master-slave-role

master-slave-role: The current role of this Schema Registry instance. A value of 1 indicates that this instance is the primary, and 0 indicates that it’s a secondary.

MBean: kafka.schema.registry:type=exporter

Schema Registry reports the following metrics for the schema exporters you create with Schema Linking. Each metric carries tags on its object name that identify the exporter, such as exporter_name and tenant. To browse the full set in a JMX client such as JConsole, filter on kafka.schema.registry:type=exporter,*.

exporter_tasks: The number of exporter tasks in each state (UNASSIGNED, STARTING, RUNNING, or PAUSED). Tagged with exporter_name, exporter_state, has_error, and is_retriable.
exporter_starting_progress: The progress of an exporter that is starting. Tagged with exporter_name.
schema_transfer_success_total: The total number of successful schema register calls made by the exporter. This can exceed the number of unique schemas, because a schema with multiple versions registers each version separately. Tagged with exporter_name.
exporter_objects_sync_total: The total number of objects synced successfully, including schemas, Key Encryption Keys (KEKs), Data Encryption Keys (DEKs), and deletes. Unlike schema_transfer_success_total, this metric also counts non-schema objects. Tagged with exporter_name, object_type, and operation.
exporter_lifecycle_latency: The wall-clock duration of the exporter’s most recent lifecycle phase (for example, STARTING or RUNNING), in milliseconds. This measures time spent in that phase, not the latency of an individual schema export. Tagged with exporter_name and exporter_state.
exporters_errored_total: The number of times an exporter moved to the PAUSED state because of an exception or error. Tagged with exporter_name, error_type, and operation.
exporter_service_start_failure_total: The number of schema exporter service failures. This metric applies to the service as a whole, so it has no exporter tags.

Starting with Confluent Platform 8.0, Schema Registry also reports the following metrics for the executor pools that run exporter tasks. These metrics describe the health of the executor pools themselves, not any individual exporter. They carry the pool_type tag (starting or running) and have no exporter_name tag.

exporter_executor_active_tasks: The number of tasks currently executing. Tagged with pool_type.
exporter_executor_stripe_count: The number of per-tenant stripes the executor is tracking. Tagged with pool_type.
exporter_executor_total_threads: The total thread count in the executor pool. Tagged with pool_type.
exporter_executor_utilization_ratio: The ratio of active tasks to total threads in the executor pool. Tagged with pool_type.
exporter_executor_task_duration_total: The cumulative task execution duration, in milliseconds. Tagged with pool_type.
exporter_executor_task_rejected_total: The number of tasks rejected by the executor pool. Tagged with pool_type and reason.
exporter_tenant_queue_depth: The number of pending tasks queued for a tenant. Tagged with tenant and pool_type.
tenant_task_queue_wait_ms: The time, in milliseconds, the most recent task for a tenant spent waiting in the queue before executing. Tagged with tenant and pool_type.
tenant_task_execution_ms: The time, in milliseconds, the most recent task for a tenant spent executing. Tagged with tenant and pool_type.
tenant_task_total_latency_ms: The total time, in milliseconds, for the most recent task for a tenant, from being queued through completion. Tagged with tenant and pool_type.
exporter_tenant_tasks_enqueued_total: The total number of tasks submitted for a tenant. Tagged with tenant and pool_type.
exporter_tenant_tasks_completed_total: The total number of tasks completed for a tenant. Tagged with tenant and pool_type.

Per-Endpoint Metrics

Each endpoint request method has the following metrics. Schema Registry also aggregates the metrics for all requests into a global instance for each metric. These aggregate instances have no prefix in their name.

MBean: kafka.schema.registry:type=jersey-metrics

<endpoint>.request-byte-rate: Bytes per second of incoming requests
<endpoint>.request-error-rate: The average number of requests per second that resulted in HTTP error responses
<endpoint>.request-latency-avg: The average request latency in ms
<endpoint>.request-latency-max: The maximum request latency in ms
<endpoint>.request-rate: The average number of HTTP requests per second.
<endpoint>.request-size-avg: The average request size in bytes
<endpoint>.request-size-max: The maximum request size in bytes
<endpoint>.response-byte-rate: Bytes per second of outgoing responses
<endpoint>.response-rate: The average number of HTTP responses per second.
<endpoint>.response-size-avg: The average response size in bytes
<endpoint>.response-size-max: The maximum response size in bytes

Endpoints

The following table lists all the API endpoint methods. The names map intuitively to the API operations. To create a full metric name, prefix a per-endpoint metric name with one of these values. For example, to find the average latency of GET /subjects API calls, combine the endpoint name subjects.list with the metric name request-latency-avg to get subjects.list.request-latency-avg.

`compatibility.subjects.versions.verify`	`POST /compatibility/subjects/{subject}/versions/{version}`
`schemas.ids.get-schema`	`GET /schemas/ids/{id}`
`subjects.get-schema`	`POST /subjects/{subject}`
`subjects.list`	`GET /subjects`
`subjects.versions.get-schema`	`GET /subjects/{subject}/versions/{version}`
`subjects.versions.get-schema.only`	`GET /subjects/{subject}/versions/{version}/schema`
`subjects.versions.list`	`GET /subjects/{subject}/versions`
`subjects.versions.register`	`POST /subjects/{subject}/versions`