# Monitor Confluent Cloud Gateway Metrics Confluent Cloud Gateway (Confluent Gateway) exposes Prometheus-format metrics and a liveness check through a single administrative service. For information on how to enable the metrics endpoint, see [Administration and metrics configuration](gateway-deploy.md#gateway-config-admin-metrics-docker). Metric names and labels can change between Confluent Gateway releases. Query the administrative service’s `/metrics` endpoint (`http://{bindAddress}:{port}/metrics`) directly on your Confluent Gateway instance to verify the exact metric names exposed by your deployed version. To view pre-built Grafana dashboards and a sample Prometheus stack you can run locally, see the [gateway monitoring examples](https://github.com/confluentinc/confluent-kubernetes-examples/tree/master/gateway/monitoring) in the `confluent-kubernetes-examples` repository. ## Monitor failed connections Use these metrics to detect downstream request failures and upstream broker connections that fail to establish. #### Failed connection metrics | Metric | Description | Alert guidance | |-----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------| | `kroxylicious_client_to_proxy_errors_total` | Total count of downstream client-facing errors. Measures failures
occurring on the proxy before requests reach upstream Kafka brokers. | Alert on a sustained non-zero rate over five minutes. | | `kroxylicious_client_to_proxy_connections_total` and
`kroxylicious_proxy_to_server_connections_total` | Measures cumulative connection attempts from clients to the proxy with
connection attempts from the proxy to upstream brokers. Divergence
indicates the proxy accepts client connections but fails to
establish broker connections. | Alert if the gap remains non-zero for more than five
minutes. | | `kroxylicious_client_to_proxy_disconnects_total{cause}` | Measures total client disconnections partitioned by cause, such as graceful
shutdowns (`client_closed`), session timeouts (`idle_timeout`),
or active node drains (`drain_timeout`). | Alert on spikes in `drain_timeout` or any cause other than
`client_closed`. | | `kroxylicious_virtual_cluster_state{state}` | Identifies current lifecycle state of virtual cluster routes, returning `1`
for the active state and `0` for inactive states. Valid states
include `serving`, `failed`, `draining`, `initializing`,
and `stopped`. | Alert if any route reports `state="failed"` with a value of `1`. | Example Prometheus Query Language (PromQL) query for the downstream error rate: ```promql rate(kroxylicious_client_to_proxy_errors_total[5m]) ``` ## Track request and response completion Use these metrics to identify requests that do not complete and schema validation failures on enforced routes. #### Request and response completion metrics | Metric | Description | Alert guidance | |-----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------| | `kroxylicious_client_to_proxy_request_total` compared to
`kroxylicious_proxy_to_client_response_total` or
`kroxylicious_server_to_proxy_response_total` | Requests that never receive a response. | Alert if the ratio of responses to requests drifts from
`1` for more than five minutes. | | `gateway_schema_validation_requests_total{outcome}` | Schema validation success and failure counts per route and topic.
Confluent Gateway emits this metric only for routes that have schema
validation enforcement enabled. For more information, see
[Enforce Data Governance Centrally with Confluent Gateway](governance/gateway-governance.md#gateway-governance). | Alert on a sustained `outcome="failure"` rate. | Example PromQL queries: ```promql # Request rate rate(kroxylicious_client_to_proxy_request_total[5m]) # Response rate rate(kroxylicious_proxy_to_client_response_total[5m]) # Schema validation failure rate by route and topic sum by (route, topic) (rate(gateway_schema_validation_requests_total{outcome="failure"}[5m])) ``` ## Measure throughput and latency Use these metrics for capacity planning and to track authentication swapping (AuthSwap) latency against your service-level objectives (SLOs). #### Throughput and latency metrics | Metric | Description | Alert guidance | |----------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|---------------------------------------------------------| | `kroxylicious_*_request_size_bytes_*` and
`kroxylicious_*_response_size_bytes_*` | Traffic volume and payload sizes for client-to-proxy and
proxy-to-broker traffic. | Track for capacity planning. No default alert required. | | `gateway_authswap_latency_seconds{quantile}` | Client-observed end-to-end AuthSwap latency percentiles. | Alert if the `quantile="0.99"` series exceeds your SLO. | | `gateway_authswap_secret_store_latency_seconds{quantile,type}` | Secret store lookup latency during AuthSwap. | Alert if the `quantile="0.99"` series exceeds your SLO. | | `gateway_authswap_client_auth_total{result}` and
`gateway_authswap_cluster_auth_total{result}` | Client-side and cluster-side AuthSwap authentication success and
failure counts. | Alert on a sustained `result="failure"` rate. | #### NOTE Confluent Gateway emits `gateway_authswap_*` metrics only for routes configured with AuthSwap authentication. Routes using passthrough authentication do not emit these series. For details, see [Authentication swapping](gateway-security.md#gateway-authn-swap-docker). Example PromQL queries: ```promql # Average request size by virtual cluster rate(kroxylicious_client_to_proxy_request_size_bytes_sum[5m]) / rate(kroxylicious_client_to_proxy_request_size_bytes_count[5m]) # AuthSwap client authentication failure rate by route rate(gateway_authswap_client_auth_total{result="failure"}[5m]) # 99th-percentile AuthSwap latency by route gateway_authswap_latency_seconds{quantile="0.99"} ``` ## Check process health Use standard Java Virtual Machine (JVM) and process metrics to monitor process health. #### Process health metrics | Metric | Description | Alert guidance | |-------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------| | `jvm_memory_used_bytes` and `jvm_memory_max_bytes` | Heap and non-heap memory usage compared to configured limits. | Alert if usage exceeds 85% of maximum memory. | | `jvm_gc_pause_seconds` | Garbage collection pause time. | Alert on a rising `rate(..._sum[5m])`. | | `process_cpu_usage` and `system_cpu_usage` | Process CPU usage compared to total host CPU usage. | Alert if usage exceeds your configured threshold. | | `process_files_open_files` and `process_files_max_files` | Open file descriptors compared to the maximum limit. Requires
`FileDescriptorMetrics` in the `admin.jvmMetrics` configuration.
For more information, see
[Administration and metrics configuration](gateway-deploy.md#gateway-config-admin-metrics-docker). | Alert if open files exceed 80% of the maximum limit. | | `kroxylicious_client_to_proxy_active_connections` and
`kroxylicious_proxy_to_server_active_connections` | Active client-to-proxy and proxy-to-broker connection counts. | Track against expected client and broker counts. | | `netty_eventexecutor_tasks_pending` | Pending tasks per Netty event-loop thread. | Alert on a sustained non-zero value to detect
event-loop saturation. | Example PromQL query for heap usage: ```promql jvm_memory_used_bytes{area="heap"} / jvm_memory_max_bytes{area="heap"} ```