Monitor Schema Registry in Confluent Platform

Schema Registry reports a variety of metrics through Java Management Extensions (JMX). It can also be configured to report stats using additional pluggable stats reporters using the metric.reporters configuration option. The easiest way to view the available metrics is to use JConsole to browse JMX MBeans.

Schema Registry has two types of metrics. Global metrics help you monitor the overall health of the service. Per-endpoint metrics monitor each API endpoint request method and are prefixed by a name of the endpoint (e.g. subjects.get-schema). These help you understand how the proxy is being used and track down specific performance problems.

Tip

Authentication is disabled for JMX by default in Kafka. You must use environment variables to override these defaults. For each platform component, the options to override the JMX defaults take the form <component-name>_JMX_OPTS. Use the following environment variable to override default JMX options for Schema Registry: SCHEMA_REGISTRY_JMX_OPTS.

Global Metrics

MBean: kafka.schema.registry:type=jetty-metrics

connections-active

Total number of active TCP connections.

connections-accepted-rate (deprecated since 2.0)

In 1.x: The average rate per second of accepted TCP connections.
In 2.x: Same as connections-opened-rate.

connections-opened-rate

The average rate per second of opened TCP connections.

connections-closed-rate

The average rate per second of closed TCP connections.

busy_thread_count

The number of threads currently working on a task (busy threads).

thread_pool_usage

Percentage of total thread pool in use (total/threads in use).

request_queue_size

Number of threads required for requests in the queue.

MBean: kafka.schema.registry:type=master-slave-role

master-slave-role: The current role of this Schema Registry instance. A value of 1 indicates this instance is the primary, 0 indicates it is a secondary.

MBean: kafka.schema.registry:type=exporter

Schema Registry reports the following metrics for the schema exporters you create with Schema Linking. Each metric carries tags on its object name that identify the exporter, such as exporter_name and tenant. To browse the full set in a JMX client such as JConsole, filter on kafka.schema.registry:type=exporter,*.

exporter_tasks: The number of exporter tasks in each state (UNASSIGNED, STARTING, RUNNING, or PAUSED). Tagged with exporter_name, exporter_state, has_error, and is_retriable.
exporter_starting_progress: The progress of an exporter that is starting. Tagged with exporter_name.
schema_transfer_success_total: The total number of successful schema register calls made by the exporter. This can exceed the number of unique schemas, since a schema with multiple versions registers each version separately. Tagged with exporter_name.
exporter_objects_sync_total: The total number of objects synced successfully, including schemas, Key Encryption Keys (KEKs), Data Encryption Keys (DEKs), and deletes. Unlike schema_transfer_success_total, this metric also counts non-schema objects. Tagged with exporter_name, object_type, and operation.
exporter_lifecycle_latency: The wall-clock duration of the exporter’s most recent lifecycle phase (for example, STARTING or RUNNING), in milliseconds. This measures time spent in that phase, not the latency of an individual schema export. Tagged with exporter_name and exporter_state.
exporters_errored_total: The number of times an exporter moved to the PAUSED state because of an exception or error. Tagged with exporter_name, error_type, and operation.
exporter_service_start_failure_total: The number of schema exporter service failures. This metric applies to the service as a whole, so it has no exporter tags.

Starting with Confluent Platform 8.0, Schema Registry also reports the following metrics for the executor pools that run exporter tasks. These metrics describe the health of the executor pools themselves, not any individual exporter, so they are tagged with pool_type (starting or running) instead of exporter_name.

exporter_executor_active_tasks: The number of tasks currently executing. Tagged with pool_type.
exporter_executor_stripe_count: The number of per-tenant stripes the executor is tracking. Tagged with pool_type.
exporter_executor_total_threads: The total thread count in the executor pool. Tagged with pool_type.
exporter_executor_utilization_ratio: The ratio of active tasks to total threads in the executor pool. Tagged with pool_type.
exporter_executor_task_duration_total: The cumulative task execution duration, in milliseconds. Tagged with pool_type.
exporter_executor_task_rejected_total: The number of tasks rejected by the executor pool. Tagged with pool_type and reason.
exporter_tenant_queue_depth: The number of pending tasks queued for a tenant. Tagged with tenant and pool_type.
tenant_task_queue_wait_ms: The time, in milliseconds, the most recent task for a tenant spent waiting in the queue before executing. Tagged with tenant and pool_type.
tenant_task_execution_ms: The time, in milliseconds, the most recent task for a tenant spent executing. Tagged with tenant and pool_type.
tenant_task_total_latency_ms: The total time, in milliseconds, for the most recent task for a tenant, from being queued through completion. Tagged with tenant and pool_type.
exporter_tenant_tasks_enqueued_total: The total number of tasks submitted for a tenant. Tagged with tenant and pool_type.
exporter_tenant_tasks_completed_total: The total number of tasks completed for a tenant. Tagged with tenant and pool_type.

Per-Endpoint Metrics

The following are the metrics available for each endpoint request method. Metrics for all requests are also aggregated into a global instance for each one. These aggregate instances have no prefix in their name.

MBean: kafka.schema.registry:type=jersey-metrics

<endpoint>.request-byte-rate: Bytes/second of incoming requests
<endpoint>.request-error-rate: The average number of requests per second that resulted in HTTP error responses
<endpoint>.request-latency-avg: The average request latency in ms
<endpoint>.request-latency-max: The maximum request latency in ms
<endpoint>.request-rate: The average number of HTTP requests per second.
<endpoint>.request-size-avg: The average request size in bytes
<endpoint>.request-size-max: The maximum request size in bytes
<endpoint>.response-byte-rate: Bytes/second of outgoing responses
<endpoint>.response-rate: The average number of HTTP responses per second.
<endpoint>.response-size-avg: The average response size in bytes
<endpoint>.response-size-max: The maximum response size in bytes

Endpoints

The following is a list of all the API endpoint methods. The naming should map intuitively to each of the API operations. To create a full metric name, prefix a per-endpoint metric name with one of these values. For example, to find the average latency of GET /subjects API calls, you would combine the endpoint name subjects.list with the metric name request-latency-avg to get subjects.list.request-latency-avg.

`compatibility.subjects.versions.verify`	`POST /compatibility/subjects/{subject}/versions/{version}`
`schemas.ids.get-schema`	`GET /schemas/ids/{id}`
`subjects.get-schema`	`POST /subjects/{subject}`
`subjects.list`	`GET /subjects`
`subjects.versions.get-schema`	`GET /subjects/{subject}/versions/{version}`
`subjects.versions.get-schema.only`	`GET /subjects/{subject}/versions/{version}/schema`
`subjects.versions.list`	`GET /subjects/{subject}/versions`
`subjects.versions.register`	`POST /subjects/{subject}/versions`