Get Started Free
  • Get Started Free
  • Courses
    • Apache Kafka 101
    • Kafka Connect 101
    • Kafka Streams 101
    • Data Mesh 101
    • ksqlDB 101
    • Inside ksqlDB
    • Spring Framework and Apache Kafka
    • Building Data Pipelines with Apache Kafka and Confluent
    • Event Sourcing and Event Storage with Apache Kafka
    • Kafka Internals
  • Learn
    • Articles
    • Patterns
    • FAQs
    • 100 Days of Code
    • Blog
    • Podcast
    • Confluent Developer Live
  • Build
    • Language Guides
    • Stream Processing Cookbook
    • Demos
  • Community
    • Kafka Summit Conference
    • Meetups & Events
    • Ask the Community
    • Community Catalysts
  • Docs
    • Docs Home
    • Confluent Cloud
    • Confluent Platform
    • Confluent Connectors
    • Tools
    • Clients
Courses
  • Apache Kafka 101
  • Kafka Connect 101
  • Kafka Streams 101
  • Data Mesh 101
  • ksqlDB 101
  • Inside ksqlDB
  • Spring Framework and Apache Kafka
  • Building Data Pipelines with Apache Kafka and Confluent
  • Event Sourcing and Event Storage with Apache Kafka
  • Kafka Internals
Learn
  • Articles
  • Patterns
  • FAQs
  • 100 Days of Code
  • Blog
  • Podcast
  • Confluent Developer Live
Build
  • Language Guides
  • Stream Processing Cookbook
  • Demos
Community
  • Kafka Summit Conference
  • Meetups & Events
  • Ask the Community
  • Community Catalysts
Docs
  • Docs Home
  • Confluent Cloud
  • Confluent Platform
  • Confluent Connectors
  • Tools
  • Clients
Get Started Free
Confluent Documentation
  • Site Filter

CONFLUENT PLATFORM

  • Overview
  • Get Started
    • What is Confluent Platform?
    • Apache Kafka Quick Start
      • Overview
      • Quick Start for Apache Kafka using Confluent Platform (Local)
      • Quick Start for Apache Kafka using Confluent Platform (Docker)
      • Quick Start for Apache Kafka using Confluent Platform Community Components (Local)
      • Quick Start for Apache Kafka using Confluent Platform Community Components (Docker)
    • Kafka Basics on Confluent Platform
    • Introduction to Kafka
    • Kafka Design
    • Videos, Demos, and Reading Material
      • Overview
      • Confluent Platform Demo (cp-demo)
      • Tutorial: Introduction to Streaming Application Development
      • Confluent Cloud Examples
        • ccloud-stack Utility for Confluent Cloud
        • Tutorial: Confluent Cloud CLI
        • Code Examples
          • Apache Kafka Commands
          • C
          • Clojure
          • Confluent CLI
          • Confluent Cloud CLI
          • .NET
          • Go
          • Groovy
          • Java
          • Java Spring Boot
          • Kafka Connect Datagen
          • kafkacat
          • Kotlin
          • KSQL Datagen
          • Node.js
          • Python
          • REST Proxy
          • Ruby
          • Rust
          • Scala
        • On-Prem Kafka to Cloud
        • Cloud ETL Example
        • Tutorial: Introduction to Streaming Application Development
        • Google Kubernetes Engine to Confluent Cloud with Confluent Replicator
        • Confluent Replicator to Confluent Cloud Configurations
      • Confluent Platform on Google Kubernetes Engine
      • Confluent Platform on Azure Kubernetes Service
      • Azure Kubernetes Service to Confluent Cloud with Confluent Replicator
      • Clickstream Data Analysis Pipeline Using ksqlDB
      • RBAC Example
      • Replicator Schema Translation Example
      • Build Your Own Demos
  • Install and Upgrade
    • On-Premises Deployments
      • Overview
      • ZIP and TAR
      • Ubuntu and Debian
      • RHEL and CentOS
      • Docker
        • Install using Docker
        • Docker Configuration Parameters
        • Docker Image Reference
        • Docker Security
    • System Requirements
    • Confluent Platform Licenses
    • Upgrade
    • Supported Versions and Interoperability
    • Using Confluent Platform systemd Service Unit Files
    • Confluent Platform Packages
    • Migrate to Confluent Platform
    • Migrate an Existing Kafka Deployment
    • Migrate to Confluent Server
  • Build Applications
    • Command Line Tools
      • CLI Tools for Confluent Platform
      • Confluent CLI
      • Confluent Cloud CLI
    • Kafka Clients
      • Overview
      • Kafka Consumer
      • Kafka Producer
      • JMS Client
        • Overview
        • JMS Client Installation
        • JMS Client Development Guide
    • API and Javadocs
      • Streams Javadocs
      • Connect Javadocs
      • Connect REST Interface
      • Clients API reference
      • Schema Registry API Reference
      • ksqlDB syntax reference
      • Confluent REST Proxy API Reference
      • Confluent Metadata API Reference
    • Application Development
      • Overview
      • kcat (formerly kafkacat) Utility
    • Docker Developer Guide
    • Streams Developer Guide
      • Writing a Streams Application
      • Testing Streams Code
      • Configuring a Streams Application
      • Streams DSL
      • Naming Kafka Streams DSL Topologies
      • Optimizing Kafka Streams Topologies
      • Processor API
      • Data Types and Serialization
      • Interactive Queries
      • Memory Management
      • Running Streams Applications
      • Managing Streams Application Topics
      • Streams Security
      • Application Reset Tool
    • Connector Developer Guide
    • MQTT Proxy
      • Introduction
      • Communication Security Settings
      • MQTT Proxy Configuration Options
    • ksqlDB Developer Guide
    • Pipelining with Kafka Connect and Kafka Streams
  • Confluent REST APIs
    • Overview
    • Quick Start
    • API Reference
    • Production Deployment
      • Confluent Server
        • Configuration
        • Security
      • REST Proxy
        • Overview
        • Configuration
        • Monitoring
        • Security
    • Connect to Confluent Cloud
    • Changelog
    • REST Proxy Tutorial
  • ksqlDB and Kafka Streams
    • ksqlDB
      • ksqlDB Overview
      • ksqlDB Quickstart
      • Install ksqlDB
      • Operate ksqlDB
      • Upgrade ksqlDB
      • Develop applications for ksqlDB
      • Run ksqlDB in Confluent Cloud
      • Connect ksqlDB to Confluent Cloud
      • Pull queries preview with Confluent Cloud ksqlDB
      • Migrate Confluent Cloud ksqlDB applications
      • Run ksqlDB in Confluent Control Center
      • Connect ksqlDB to Confluent Control Center
      • Secure ksqlDB with RBAC
      • Frequently Asked Questions
      • Troubleshoot ksqlDB issues
      • Tutorials and Examples
        • Examples Overview
        • ksqlDB quick start
        • Write streaming queries using ksqlDB (local)
        • Write streaming queries using ksqlDB and Confluent Control Center
        • Materialized view/cache
        • Streaming ETL pipeline
        • Event-driven microservice
        • ksqlDB with Embedded Connect
        • Clickstream Data Analysis Pipeline Using ksqlDB
        • Integrate with PostgreSQL
        • Kafka Tutorials using ksqlDB
        • Stream Processing Cookbook
        • Examples
    • Kafka Streams
      • Kafka Streams Overview
      • Introduction
      • Kafka Streams Quick Start
      • Tutorial: Introduction to Streaming Application Development
      • Kafka Streams Demo Application
      • Connect Confluent Platform Components to Confluent Cloud
      • Streams Concepts
      • Streams Architecture
      • Streams Code Examples
      • Streams Developer Guide
        • Writing a Streams Application
        • Testing Streams Code
        • Configuring a Streams Application
        • Streams DSL
        • Naming Kafka Streams DSL Topologies
        • Optimizing Kafka Streams Topologies
        • Processor API
        • Data Types and Serialization
        • Interactive Queries
        • Memory Management
        • Running Streams Applications
        • Managing Streams Application Topics
        • Streams Security
        • Application Reset Tool
      • Pipelining with Kafka Connect and Kafka Streams
      • Streams Operations
        • Capacity planning and sizing
        • Monitoring Kafka Streams Applications
      • Streams Upgrade Guide
      • Streams FAQ
      • Streams Javadocs
  • Integrate External Systems to Kafka
    • Overview
    • Getting Started with Kafka Connect
    • Connect to Confluent Cloud
    • Connect to External Systems
    • Connector Developer Guide
    • Tutorial: Moving Data In and Out of Kafka
    • Kafka Connect Concepts
    • Kafka Connect Reference
      • Connect Javadocs
      • Connect REST Interface
      • Worker Configuration Properties
      • Connector Configuration Properties
    • Kafka Connect Licensing
    • Kafka Connect Monitoring
    • Kafka Connect Logging
    • Kafka Connect Transformations
      • Single Message Transforms for Confluent Platform
      • Cast
      • Drop
      • DropHeaders
      • ExtractField
      • ExtractTopic
      • Filter (Apache Kafka)
      • Filter (Confluent)
      • Flatten
      • GzipDecompress
      • HeaderFrom
      • HoistField
      • InsertField
      • InsertHeader
      • MaskField
      • MessageTimestampRouter
      • RegexRouter
      • ReplaceField
      • SetSchemaMetadata
      • TimestampConverter
      • TimestampRouter
      • TombstoneHandler
      • TopicRegexRouter
      • ValueToKey
      • Custom transformations
    • Kafka Connect Security
      • Kafka Connect Security Basics
      • Kafka Connect and RBAC
        • Getting started with RBAC and Kafka Connect
        • Configuring RBAC for a Connect cluster
        • Configuring RBAC for a Connect worker
        • Configuring RBAC for connectors
        • Connect Secret Registry
        • Example role-binding sequence
    • Kafka Connect Architecture
    • Kafka Connect FAQ
  • Schema Management
    • Overview
    • Installing and Configuring
      • Overview
      • Configuration Options
      • Run in Production
      • Connect to Confluent Cloud
      • Migrate Schemas
    • Schema Registry Tutorials
      • Tutorials Overview
      • On-Premises
      • Confluent Cloud
    • Schema Validation on Confluent Server
    • Monitoring
    • Single and Multi-Datacenter Setup
    • Schema Evolution and Compatibility
    • Schemas in Control Center
    • Schemas on Confluent Cloud
    • Migrate Schemas
    • Deleting Schemas
    • Security
      • Schema Registry Security Overview
      • Role-Based Access Control
      • Schema Registry Security Plugin
        • Overview
        • Install and Configure
        • Schema Registry Authorization
          • Supported Operations and Resources
          • Role-Based Access Control
          • Schema Registry ACL Authorizer
          • Topic ACL Authorizer
    • Developer Guide
      • Overview
      • Maven Plugin
      • API Reference
      • API Usage Examples
    • Schema Formats
      • Formats, Serializers, and Deserializers
      • Avro
      • Protobuf
      • JSON Schema
    • Integrate Schemas from Connectors
  • Security
    • Overview
    • Security Tutorial
    • Encryption with SSL
    • Secrets
      • Secrets Management
      • Tutorial: Secret Protection
    • Encryption and Authentication with SSL
    • Authentication with SASL
      • Authentication Methods Overview
      • Authentication with SASL using JAAS
      • Configuring GSSAPI
      • Configuring OAUTHBEARER
      • Configuring PLAIN
      • Configuring SCRAM
      • Authentication using Delegation Tokens
      • Configuring Client Authentication with LDAP
    • HTTP Basic Auth
    • Metadata Service (MDS)
      • Configure Metadata Service (MDS)
      • Metadata Service Configuration Options
      • Configure LDAP Authentication
      • Configure LDAP Group-Based Authorization for MDS
      • Configure Kerberos Authentication for Brokers Running MDS
      • Configure MDS to Manage Centralized Audit Logs
      • Configure mTLS Authentication and RBAC for Kafka Brokers
    • Authorization using Role-Based Access Control
      • RBAC Overview
      • Quick Start
      • Predefined Roles
      • Confluent Server Authorizer
        • Overview
        • Configuring the Confluent Server Authorizer
      • Enable RBAC in a Running Cluster
      • Discover Identifiers for Clusters
      • Configuring Token Authentication
      • Confluent Metadata API Reference
      • RBAC Example
    • ACLs
      • Authorization using centralized ACLs
      • Authorization using ACLs
      • Confluent LDAP Authorizer Overview
        • Confluent LDAP Authorizer
        • Using the Confluent LDAP Authorizer
        • Configuring the LDAP Authorizer
        • Configure LDAP Authentication
    • Configure RBAC using the REST API
    • Audit Logs
      • Audit Log Concepts
      • Configuring Audit Logs using the CLI
      • Configure MDS to Manage Centralized Audit Logs
      • MDS API Audit Log Configuration
      • Configuring Audit Logs using the Properties File
    • Cluster Registry
    • Streams Security
    • Kafka Connect Security
      • Kafka Connect Security Basics
      • Kafka Connect and RBAC
        • Getting started with RBAC and Kafka Connect
        • Configuring RBAC for a Connect cluster
        • Configuring RBAC for a Connect worker
        • Configuring RBAC for connectors
        • Connect Secret Registry
        • Example role-binding sequence
    • ksqlDB RBAC
    • Confluent Control Center Security
      • Confluent Control Center Security
      • Configuring SSL
      • Control Center Configuring SASL
      • Control Center UI Authentication
      • Configuring Control Center to work with Kafka ACLs
      • Configuring Control Center with LDAP authentication
      • Configure RBAC for Control Center
      • Manage and view RBAC roles in Control Center
        • Log in to Control Center when RBAC enabled
        • Manage RBAC roles with Control Center
        • View your RBAC roles in Control Center
    • REST Proxy Security
    • Schema Registry Security
    • Confluent Security Plugins
      • Overview
      • REST Proxy Security Plugins
      • Schema Registry Security Plugin
    • Docker Security
    • Prefixes for Configuring Security
    • Security Compliance
    • ZooKeeper Security
    • Adding security to a running cluster
  • Multi-DC Deployment Architectures
    • Overview
    • Architecture Patterns and Use Cases
    • Replicator for Multi-Datacenter Replication
      • Replicator Overview
      • Tutorial: Replicating Data Across Clusters
      • Demo (Docker)
      • Download and Install
      • Configure and Run
      • Tuning Replicator
      • Monitoring Replicator
      • Replicator and Cross-Cluster Failover
      • Configuration Options
      • Migrate from MirrorMaker to Replicator
      • Replicator Verifier
      • Confluent Replicator to Confluent Cloud Configurations
    • Multi-Region Clusters
      • Overview
      • Tutorial: Multi-Region Clusters
    • Cluster Linking Preview
      • Cluster Linking
      • Demo (Docker)
      • Tutorial
      • Commands
      • Configuration Options
      • Metrics and Monitoring
      • Security
  • Administer
    • Configuration Reference
      • Broker Configurations
      • Topic Configurations
      • Consumer Configurations
      • Producer Configurations
      • Connect Configurations
        • Overview
        • Sink Configuration Properties
        • Source Configuration Properties
      • AdminClient Configurations
      • License Configurations
      • Streams Configurations
      • Docker Configuration Parameters
      • Control Center Configurations
      • Streams Configurations
      • ZooKeeper Configurations
      • ksqlDB Server Configurations
    • CLI Tools for Confluent Platform
      • CLI Tools for Confluent Platform
      • Confluent CLI
      • Confluent Cloud CLI
    • Dynamic Configurations
    • Configure Multi-Node Environment
    • Confluent Metrics Reporter
    • Confluent Telemetry Reporter
    • Confluent REST APIs
      • Overview
      • Quick Start
      • API Reference
      • Production Deployment
        • Confluent Server
          • Configuration
          • Security
        • REST Proxy
          • Overview
          • Configuration
          • Monitoring
          • Security
      • Connect to Confluent Cloud
      • Changelog
      • REST Proxy Tutorial
    • Kafka Operations
      • Overview
      • Running Kafka in Production
      • Dynamic Configurations
      • Post Kafka Deployment
      • Self-Balancing Clusters
        • Self-Balancing Overview
        • Quick Start Demo (Docker)
        • Tutorial: Adding and Removing Brokers
        • Configuration Options and Commands
        • Performance and Resource Usage
      • Auto Data Balancing
        • Quick Start
        • Tutorial (Docker)
        • Command and Configuration Options
      • Monitoring Kafka
      • Confluent Metrics Reporter
      • kcat (formerly kafkacat) Utility
      • Tiered Storage
    • Docker Operations
      • Overview
      • Kafka Monitoring and Metrics Using JMX
      • Configure Docker Logging
      • Mounting Docker External Volumes
    • Post Kafka Deployment
    • ZooKeeper Operations
      • Overview
      • Running ZooKeeper in Production
    • Kafka Streams Operations
      • Capacity planning and sizing
      • Monitoring Kafka Streams Applications
    • ksqlDB Operations
    • Kafka DevOps with Kubernetes and GitOps
  • Monitor and Manage
    • Monitor and Manage Confluent Platform
    • Proactive Support
    • Confluent Telemetry Reporter
    • Telemetry Reporter Metrics
    • Confluent Metrics Reporter
    • Monitor Kafka
    • Confluent Control Center
      • Control Center Overview
      • Installing and configuring Control Center
        • Configuring Control Center
        • Control Center Configuration Reference
        • Check Control Center Version and Enable Auto-Update
        • Control Center Usage Data Collection
        • Control Center Properties Files
        • Connecting Control Center to Confluent Cloud
        • Confluent Monitoring Interceptors
        • Installing Control Center on Kafka
        • Managing Confluent Platform Licenses
        • Troubleshooting Control Center
        • Upgrading Control Center
      • Control Center User Guide
        • User Guide Overview
        • Clusters
        • Brokers
        • Topics
          • Topics Overview
          • Add topics
          • View topic metrics
          • Message Browser
          • Manage Schemas for Topics
          • Edit the configuration settings for topics
          • Delete a topic
        • Connect
        • ksqlDB
        • Consumers
        • Replicators
        • Cluster settings
        • Alerts
          • Concepts
          • Configure alerts properties
          • Configure PagerDuty email integration with Control Center alerts
          • Alerts navigation
          • Trigger management
          • Actions management
          • Example triggers and actions
          • REST API for alerts history
          • Troubleshooting alerts
        • System Health (deprecated view)
        • Data streams monitoring (deprecated view)
      • Confluent Control Center Security
      • Changelog
        • Control Center Changelog
        • Auto-Updating User Interface Changelog
    • Configure Docker Logging
    • Monitor Streams Applications
  • Resources
  • Release Notes
    • Overview
    • Component Changelogs
  • API and Javadocs
    • Streams Javadocs
    • Connect Javadocs
    • Connect REST Interface
    • Clients API reference
    • Schema Registry API Reference
    • ksqlDB syntax reference
    • Confluent REST Proxy API Reference
    • Confluent Metadata API Reference
  1. Home
  2. Multi-DC Deployment Architectures
  3. Cluster Linking Preview

Monitoring Cluster Metrics and Optimizing Links¶

Looking for Confluent Cloud Cluster Linking docs? You are currently viewing Confluent Platform documentation. If you are looking for Confluent Cloud docs, check out Cluster Linking on Confluent Cloud.

Important

This feature is available as a preview feature. A preview feature is a component of Confluent Platform that is being introduced to gain early feedback from developers. This feature can be used for evaluation and non-production testing purposes or to provide feedback to Confluent.

Confluent Platform exposes several metrics through Java Management Extensions (JMX) that are useful for monitoring Cluster Linking. Some of these are derived from extending existing interfaces, and others are new for Cluster Linking.

To monitor Cluster Linking, set the JMX_PORT environment variable before starting the cluster, then collect the reported metrics using your usual monitoring tools. JMXTrans, Graphite, and Grafana are a popular combination for collecting and reporting JMX metrics from Kafka. Datadog is another popular monitoring solution.

Wherever possible, metrics are categorized below per associated link names.

Also, you can set quotas to limit resources and bandwidth used by Cluster Linking to help optimize the system.

Cluster Link Fetcher Metrics¶

kafka.server.link:type=ClusterLinkFetcherManager,name=MaxLag,clientId=ClusterLink,link-name={linkName}
Maximum lag in messages between the replicas on the destination cluster and the leader replica on the source cluster. Provides an indication of overall cluster link health and a relative measure of whether the destination cluster is lagging behind.
kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=ClusterLinkFetcherThread-{destBrokerId}-{linkName}-{sourceBrokerId},topic={topic},partition={partition},link-name={linkName}
Drill-down of MaxLag, which provides lag in number of messages per replica on the destination cluster from the leader replica on the source cluster. Useful to debug any specific lagging topic partitions/replicas on the destination cluster.
kafka.server.link:type=ClusterLinkFetcherManager,name=DeadThreadCount,clientId=ClusterLink,link-name={linkName}
Number of dead threads of type ClusterLinkThread, where a dead thread is indicative of an issue with fetching from the source cluster for a partition.
kafka.server:type=FetcherStats,name=BytesPerSec,clientId=ClusterLinkFetcherThread-{destBrokerId}-{linkName}-{sourceBrokerId},brokerHost={host},brokerPort={port},link-name={linkName}
Rate at which data is fetched from the source cluster. Indicates amount of throughput in bytes per second on the cluster link.
kafka.server:type=FetcherStats,name=RequestsPerSec,clientId=ClusterLinkFetcherThread-{destBrokerId}-{linkName}-{sourceBrokerId}, brokerHost={host},brokerPort={port},link-name={linkName}
Rate at which requests are issued to the source cluster over the cluster link.

Throttle Time Metrics for Cluster Link Fetchers¶

kafka.server:type=cluster-link,link-name={linkName}
Throttle time metrics may indicate that a cluster link connection is being throttled, which is useful for understanding why lag may be increasing over a cluster link.
Name Description
fetch-throttle-time-avg, fetch-throttle-time-max Gives throttle times for the cluster linking fetchers. May indicate increases in lag on the cluster link due to throttling/quotas being enforced.

Cluster Link Network Client Metrics¶

kafka.server:type=cluster-link-metadata-metrics,link-name={linkName}
Metrics pertaining to the metadata refresh client in the cluster link.
kafka.server:type=cluster-link-fetcher-metrics,link-name={linkName},broker-id={id},fetcher-id={id}
Metrics pertaining to data fetcher requests in the cluster link.

Metrics seen through cluster-link-fetcher-metrics are shown in the following table.

Name Description
connection-count Number of connections for a cluster link
connection-creation-rate, connection-creation-total Rate per second and total for connections created for the cluster link. If the rate is high, the source cluster may be overloaded with connections.
connection-close-rate, connection-close-total Rate per second and total for connections closed for the cluster link. Can be compared to connection creation to understand the balance of creating/closing connections.
incoming-byte-rate, incoming-byte-total Number of bytes per second and total received from the source cluster.
outgoing-byte-rate, outgoing-byte-total Number of bytes per second and total sent to the source cluster.
network-io-rate, network-io-total Indicates rate and total network input/output (IO).
request-rate, request-total Rate per second and total requests issued to the source cluster over the cluster link.
response-rate, response-total Rate per second and total number of responses received from the source cluster over the cluster link.
request-size-avg, request-size-max Average and maximum size in bytes of requests issued to the source cluster over the cluster link.
io-ratio, io-wait-time-ns-avg, io-waittime-total, iotime-total Statistics of the destination cluster’s network IO for the requests over the cluster link.
successful-authentication-rate, successful-authentication-total Rate per second and total number of cluster link clients authenticating to the source cluster over the cluster link.
successful-reauthentication-rate, successful-reauthentication-total Rate per second and total re-authentication to the source cluster over the cluster link.
failed-reauthentication-rate, failed-reauthentication-total Rate per second and total failed re-authentications to the source cluster over the cluster link. If failures are present, it could indicate misconfigured or stale cluster link credentials.
reauthentication-latency-avg Average re-authentication latency. Helps to assess whether clients are taking too long to authenticate to the clusters.

Request Metrics for Cluster Link APIs¶

kafka.network:type=RequestMetrics,name={ LocalTimeMs| RemoteTimeMs | RequestBytes | RequestQueueTimeMs| ResponseQueueTimeMs | ResponseSendIoTimeMs | ResponseSendTimeMs | ThrottleTimeMs| TotalTimeMs },request={CreateClusterLinks| DeleteClusterLinks| ListClusterLinks}
Depending on the request name, provides statistics on requests on the cluster link, including requests and response times, time requests wait in the queue, size of requests (bytes), and so forth.
kafka.server:type=DelayedOperationPurgatory,name=PurgatorySize,delayedOperation=ClusterLink
Provides metrics on delayed operations on the cluster link.

Metrics on the Destination Cluster¶

Starting with Confluent Platform 6.0.0 for the Cluster Linking preview, the following broker metrics, specific to Cluster Linking, are available on the brokers in the destination cluster.

kafka.server:type=cluster-link-metrics,name=mirror-partition-count,link-name={linkName}
Number of actively mirrored partitions for which the broker is leader
kafka.server:type=cluster-link-metrics,name=failed-mirror-partition-count,link-name={linkName}
Number of failed mirror partitions for which the broker is leader. A partition may be in a failed state if the source partition’s epoch has gone backwards; for example, if the topic was recreated.
kafka.server:type=cluster-link-metrics,name=(linked-leader-epoch-change-rate, linked-leader-epoch-change-total), link-name={linkName}

Rate per second and total number of times leader election was triggered on this broker due to source leader changes.

  • Frequent triggers for leader election might indicate issues on the source cluster. This can be a useful metric during on-premises to Confluent Cloud migrations to identify if there are issues on the source cluster.
kafka.server:type=cluster-link-metrics,name=(linked-topic-partition-addition-rate, linked-topic-partition-addition-total),link-name={linkName}

Rate per second and total number of times at which the partition count was updated due to source changes.

  • A high volume or rate of partition count updates might indicate issues on the source cluster.
kafka.server:type=cluster-link-metrics,name=(consumer-offset-committed-rate, consumer-offset-committed-total),link-name={linkName}

Rate per second and total number of consumer offsets committed for a cluster link.

  • Helps verify if offset migration is happening correctly, a critical metric for failover readiness scenarios.
kafka.server:type=cluster-link-metrics,name=(topic-config-update-rate, topic-config-update-total),link-name={linkName}

Rate per second and total number of topic configurations updates due to source topic configuration changes.

  • Can help troubleshooting and debugging issues where configurations are not propagated.
kafka.server:type=cluster-link-metrics,name=(acls-added-rate, acls-added-total),link-name={linkName}
Rate per second and total number of access control lists (ACLs) that are being added for each link.
kafka.server:type=cluster-link-metrics,name=(acls-deleted-rate, acls-deleted-total),link-name={linkName
Rate per second and total number of access control lists (ACLs) that are being deleted for each link.
kafka.server:type=cluster-link-metrics,name=link-count
Total number of cluster links in the cluster.
kafka.server:type=cluster-link-metrics,name=global-active-mirror-topic-count
Total number of active (healthy) mirror topics.
kafka.server:type=cluster-link-metrics,name=global-stopped-mirror-topic-count
Total number of mirror topics whose mirroring has been temporarily stopped.
kafka.server:type=cluster-link-metrics,name=global-failed-mirror-topic-count
Total number of failed mirror topics; that is, mirroring cannot proceed.

Metrics on the Source Cluster¶

The source cluster is unaware of cluster links, but can monitor cluster link related usage by allocating link-specific credentials. Quota metrics for each credential is available in both Kafka and Confluent Platform when user quotas are enabled.

kafka.server:type=Fetch,user={linkPrincipal},client-id={optionalLinkClientId}
  • byte-rate
  • throttle-time
kafka.server:type=Request,user={linkPrincipal},client-id={optionalLinkClientId}
  • request-time
  • throttle-time

Setting Quotas on Resources and Bandwidth Usage¶

You can set various types of quotas to place caps on resources and bandwidth used for Cluster Linking on the source and destination clusters.

Destination Cluster Quotas¶

You can limit total usage for cluster links on each broker in the destination cluster by setting the broker config confluent.cluster.link.io.max.bytes.per.second.

Source Cluster Quotas¶

The source cluster is unaware of cluster links, but can limit usage for cluster links by allocating link-specific credentials and assigning quotas for the link principal.

  • Fetch byte-rate (replication throughput) for each cluster link principal
  • Request rate quota (CPU/thread usage) for each cluster link principal. This also includes CPU usage for metadata and configuration sync as well as ACL and consumer offset migration.

Suggested Reading¶

  • See Monitoring Kafka for a full list of metrics for Confluent Platform.
  • See the Apache Kafka® documentation on metrics reporting and monitoring through JMX endpoints.

Confluent Cloud is a fully-managed Apache Kafka service available on all three major clouds. Try it free today.

Get Started Free
  • Confluent
  • About
  • Careers
  • Contact
  • Professional Services
  • Product
  • Confluent Cloud
  • ksqlDB
  • Developer
  • Free Courses
  • Tutorials
  • Event Streaming Patterns
  • Documentation
  • Blog
  • Podcast
  • Community
  • Forum
  • Meetups
  • Kafka Summit
  • Catalysts
Terms & Conditions Privacy Policy Do Not Sell My Information Modern Slavery Policy Cookie Settings

Copyright © Confluent, Inc. 2014- . Apache, Apache Kafka, Kafka, and associated open source project names are trademarks of the Apache Software Foundation

On this page: