.. _controlcenter_userguide_replicators: Replicators =========== |crep-full| replicates |ak| topics from one cluster to another (source to destination), copying the messages, and creating replica topics that preserve the topic configuration in the source cluster. From the **Replicators** pages on |c3-short|, you can: - Monitor tasks, message throughput, and |kconnect| workers running as replicators. - Monitor metrics on source topics on the origin cluster. - Monitor metrics on replicated topics on the destination cluster. - Drill down on source and replicated topics to monitor and configure them through the |c3-short| **Topics** pages (see :ref:`controlcenter_userguide_topics`). The metrics provided are: - Throughput - the number of messages replicated per second. - Message Lag - the number of messages that have been produced to the origin cluster that have not yet been replicated to the destination. - Latency - the average time period between message production to the origin cluster and message production to the destination cluster. The metrics are broken down by connector, task and topic/partition. Enable |crep| monitoring ------------------------ If you are just getting started with |crep|, you can use the :ref:`replicator_quickstart` as a guide to bring up origin and destination clusters with topic replication, then follow the steps in the last section, :ref:`rep-quickstart-monitoring`, to configure monitoring. All Replicators page -------------------- To access the Replicators overview page, select a cluster from the navigation bar and click **Replicators** from the cluster submenu. .. figure:: ../images/c3-replicators-all.png :width: 600px The Replicators overview page lists all replicators in the selected cluster, and shows status at a glance of tasks for each replicator, including: - Status of replicators (running, failed, and paused) - Number of source topics processed - Number of messages replicated per second - Message lag - Latency .. _rep-rename-C3-displayed-cluster-names: ------------------------------- Origin and Destination clusters ------------------------------- If your clusters and replication are configured properly, a minimum of two clusters, origin and destination, will be displayed on the |c3-short|. .. tip:: If you do not see fully populated origin and destination clusters on |c3-short|, review the configuration details in :ref:`replicator_quickstart` and particularly the subtopic in that guide, :ref:`rep-quickstart-monitoring`. The clusters are rendered on |c3-short| with auto-generated names. The destination cluster defaults to |c3| because |c3| runs on the destination. To edit the cluster names: #. Select a cluster. #. Click **Cluster settings** on the navigation menu. #. On the |ak| tab, type a cluster name, and click **Save**. This is the renamed destination cluster. .. figure:: ../images/c3-replicators-cluster-name.png :width: 600px This is the renamed source cluster. .. figure:: ../images/c3-replicators-cluster-name-origin.png :width: 600px ---------------------------- Source and Replicated topics ---------------------------- Source topics are located on the origin cluster. There are several ways to navigate to these topics. For example: - With either cluster selected, click **Replicators** on the navigation menu, click a replicator, then choose **Source Topics** tab -> . - With either cluster selected, click **Replicators** on the navigation menu, click a replicator, then click the **Throughput** metrics card -> -> . - Select the origin cluster, click **Topics**, and drill down on a topic that way. Replicated topics live on the destination cluster. To navigate to these topics: #. Select the destination cluster. #. Select **Topics** from the navigation menu. .. figure:: ../images/c3-replicator-topics-overview.png :width: 600px #. Click a replicated topic. .. figure:: ../images/c3-replicated-topic-drilldown-overview.png :width: 600px The following sections describe the available replication monitoring, metrics, and configuration views in more detail. Metrics and topics for a replicator ----------------------------------- To view details on a specific replicator, click a replicator in the list on the All Replicators page. .. figure:: ../images/c3-replicators-drilldown.png :width: 600px The |crep| status page is displayed, showing: - Throughput metrics on message replication, including messages replicated per second, number of source topics, lag, and latency - Task overview, including total tasks, and number of running, failed, and paused tasks - List of |kconnect| workers showing status of each worker, along with message performance metrics per worker. You can scroll and search the list of workers. .. figure:: ../images/c3-replicators-status.png :width: 600px ---------------------------------- Tasks associated with a replicator ---------------------------------- To view a list of tasks associated with a replicator, click the **Throughput** card for the selected replicator. .. figure:: ../images/c3-replicators-throughput.png :width: 600px The list of tasks is displayed. .. figure:: ../images/c3-replicators-metrics.png :width: 600px -------------------------- Message metrics for a task -------------------------- To view metrics for a specific task, click one of the tasks associated with the selected replicator. .. figure:: ../images/c3-replicators-task-drilldown.png :width: 600px The task metrics are displayed at the top of the page. A list of source topics and associated metrics per topic is displayed on the lower half of the page. You can search and scroll through this list. .. figure:: ../images/c3-replicators-task-metrics-and-source-topics.png :width: 600px -------------- Source topics -------------- To view details on a source topic (a topic on the origin cluster), select the **Source Topics** tab for the |crep|, then click a topic in the list. .. figure:: ../images/c3-replicators-source-topics-select.png :width: 600px You can also view details on source topic by clicking a topic on the task metrics page or, with the origin cluster selected, click **Topics** from the navigation bar, then click a topic in that list. An overview of metrics for the source topic is displayed. .. figure:: ../images/c3-replicators-source-topic-drilldown-overview.png :width: 600px You have the same monitoring and configuration options on both source and replicated topics. ----------------- Replicated topics ----------------- To view details on a replicated topic (on the destination cluster), select the Destination cluster, then click **Topics** on the |c3-short| menu to show all topics in the cluster. .. figure:: ../images/c3-replicator-topics-overview.png :width: 600px Click one of the replicas in the topic list to bring up the overview page for that topic. .. figure:: ../images/c3-replicated-topic-drilldown-overview.png :width: 600px Click the metrics cards on the **Overview** tab to drill down on the monitoring data for that topic. For example, click into **Production** to show production producer metrics. .. figure:: ../images/c3-replicated-topic-metrics-production.png :width: 600px Click into **Consumption** to show various consumer metrics. .. figure:: ../images/c3-replicated-topic-metrics-consumer-lag.png :width: 600px Navigate using the other tabs to view and manage details on the topic. For example, this page shows **Messages** associated with the replicated topic. .. figure:: ../images/c3-replicator-topic-drilldown-messages.png :width: 600px This page shows the topic configuration and provides options to re-configure it. (Click **Show full config** to get to advanced configuration options.) .. figure:: ../images/c3-replicator-topic-drilldown-config.png :scale: 80% .. figure:: ../images/c3-replicator-topic-drilldown-config-edit.png :width: 600px You have the same monitoring and configuration options on both source and replicated topics. To learn more about working with topics in |c3-short|, see :ref:`controlcenter_userguide_topics`. Suggested Reading ----------------- - :ref:`replicator_quickstart` - :ref:`rep-quickstart-monitoring` - :ref:`rep-tutorial-troubleshoot` - :ref:`Replicator Monitoring Setup and Reference ` - :ref:`controlcenter_troubleshooting` - :ref:`metrics_reporter` - :ref:`controlcenter_clients` - :ref:`replicator`