Monitor Confluent Cloud Gateway Metrics

Confluent Cloud Gateway (Confluent Gateway) exposes Prometheus-format metrics and a liveness check through a single administrative service. For information on how to enable the metrics endpoint, see Administration and metrics configuration.

Metric names and labels can change between Confluent Gateway releases. Query the administrative service’s /metrics endpoint (http://{bindAddress}:{port}/metrics) directly on your Confluent Gateway instance to verify the exact metric names exposed by your deployed version.

To view pre-built Grafana dashboards and a sample Prometheus stack you can run locally, see the gateway monitoring examples in the confluent-kubernetes-examples repository.

Monitor failed connections

Use these metrics to detect downstream request failures and upstream broker connections that fail to establish.

Failed connection metrics

Metric	Description	Alert guidance
kroxylicious_client_to_proxy_errors_total	Total count of downstream client-facing errors. Measures failures occurring on the proxy before requests reach upstream Kafka brokers.	Alert on a sustained non-zero rate over five minutes.
kroxylicious_client_to_proxy_connections_total and kroxylicious_proxy_to_server_connections_total	Measures cumulative connection attempts from clients to the proxy with connection attempts from the proxy to upstream brokers. Divergence indicates the proxy accepts client connections but fails to establish broker connections.	Alert if the gap remains non-zero for more than five minutes.
kroxylicious_client_to_proxy_disconnects_total{cause}	Measures total client disconnections partitioned by cause, such as graceful shutdowns (client_closed), session timeouts (idle_timeout), or active node drains (drain_timeout).	Alert on spikes in drain_timeout or any cause other than client_closed.
kroxylicious_virtual_cluster_state{state}	Identifies current lifecycle state of virtual cluster routes, returning 1 for the active state and 0 for inactive states. Valid states include serving, failed, draining, initializing, and stopped.	Alert if any route reports state="failed" with a value of 1.

Example Prometheus Query Language (PromQL) query for the downstream error rate:

rate(kroxylicious_client_to_proxy_errors_total[5m])

Track request and response completion

Use these metrics to identify requests that do not complete and schema validation failures on enforced routes.

Request and response completion metrics

Metric	Description	Alert guidance
kroxylicious_client_to_proxy_request_total compared to kroxylicious_proxy_to_client_response_total or kroxylicious_server_to_proxy_response_total	Requests that never receive a response.	Alert if the ratio of responses to requests drifts from 1 for more than five minutes.
gateway_schema_validation_requests_total{outcome}	Schema validation success and failure counts per route and topic. Confluent Gateway emits this metric only for routes that have schema validation enforcement enabled. For more information, see Enforce Data Governance Centrally with Confluent Gateway.	Alert on a sustained outcome="failure" rate.

Example PromQL queries:

# Request rate
rate(kroxylicious_client_to_proxy_request_total[5m])

# Response rate
rate(kroxylicious_proxy_to_client_response_total[5m])

# Schema validation failure rate by route and topic
sum by (route, topic) (rate(gateway_schema_validation_requests_total{outcome="failure"}[5m]))

Measure throughput and latency

Use these metrics for capacity planning and to track authentication swapping (AuthSwap) latency against your service-level objectives (SLOs).

Throughput and latency metrics

Metric	Description	Alert guidance
kroxylicious_*_request_size_bytes_* and kroxylicious_*_response_size_bytes_*	Traffic volume and payload sizes for client-to-proxy and proxy-to-broker traffic.	Track for capacity planning. No default alert required.
gateway_authswap_latency_seconds{quantile}	Client-observed end-to-end AuthSwap latency percentiles.	Alert if the quantile="0.99" series exceeds your SLO.
gateway_authswap_secret_store_latency_seconds{quantile,type}	Secret store lookup latency during AuthSwap.	Alert if the quantile="0.99" series exceeds your SLO.
gateway_authswap_client_auth_total{result} and gateway_authswap_cluster_auth_total{result}	Client-side and cluster-side AuthSwap authentication success and failure counts.	Alert on a sustained result="failure" rate.

Note

Confluent Gateway emits gateway_authswap_* metrics only for routes configured with AuthSwap authentication. Routes using passthrough authentication do not emit these series. For details, see Authentication swapping.

Example PromQL queries:

# Average request size by virtual cluster
rate(kroxylicious_client_to_proxy_request_size_bytes_sum[5m])
  / rate(kroxylicious_client_to_proxy_request_size_bytes_count[5m])

# AuthSwap client authentication failure rate by route
rate(gateway_authswap_client_auth_total{result="failure"}[5m])

# 99th-percentile AuthSwap latency by route
gateway_authswap_latency_seconds{quantile="0.99"}

Check process health

Use standard Java Virtual Machine (JVM) and process metrics to monitor process health.

Process health metrics

Metric	Description	Alert guidance
jvm_memory_used_bytes and jvm_memory_max_bytes	Heap and non-heap memory usage compared to configured limits.	Alert if usage exceeds 85% of maximum memory.
jvm_gc_pause_seconds	Garbage collection pause time.	Alert on a rising rate(..._sum[5m]).
process_cpu_usage and system_cpu_usage	Process CPU usage compared to total host CPU usage.	Alert if usage exceeds your configured threshold.
process_files_open_files and process_files_max_files	Open file descriptors compared to the maximum limit. Requires FileDescriptorMetrics in the admin.jvmMetrics configuration. For more information, see Administration and metrics configuration.	Alert if open files exceed 80% of the maximum limit.
kroxylicious_client_to_proxy_active_connections and kroxylicious_proxy_to_server_active_connections	Active client-to-proxy and proxy-to-broker connection counts.	Track against expected client and broker counts.
netty_eventexecutor_tasks_pending	Pending tasks per Netty event-loop thread.	Alert on a sustained non-zero value to detect event-loop saturation.

Example PromQL query for heap usage:

jvm_memory_used_bytes{area="heap"} / jvm_memory_max_bytes{area="heap"}