.. _controlcenter_userguide_replicators: Manage Replicator Using |c3-short| for |cp| =========================================== |crep-full| replicates |ak| topics from one cluster to another (source to destination), copying the messages, and creating replica topics that preserve the topic configuration in the source cluster. From the **Replicators** pages in |c3-short|, you can: - Monitor tasks, message throughput, and |kconnect| workers running as replicators. - Monitor metrics on source topics on the origin cluster. - Monitor metrics on replicated topics on the destination cluster. - Drill down on source and replicated topics to monitor and configure them through the |c3-short| **Topics** pages (see :ref:`controlcenter_userguide_topics`). The metrics provided are: - Throughput - the number of messages replicated per second. - Message Lag - the number of messages that have been produced to the origin cluster that have not yet been replicated to the destination. - Latency - the average time period between message production to the origin cluster and message production to the destination cluster. The metrics are broken down by connector, task and topic/partition. Enable |crep| monitoring ------------------------ If you are just getting started with |crep|, you can use the :ref:`replicator_quickstart` as a guide to bring up origin and destination clusters with topic replication, then follow the steps in the last section, :ref:`rep-quickstart-monitoring`, to configure monitoring. All Replicators page -------------------- To access the Replicators overview page, select a cluster from the navigation bar and click **Replicators** from the cluster submenu. .. figure:: ../images/c3-replicators-all.png :scale: 80% The Replicators overview page lists all replicators in the selected cluster, and shows status at a glance of tasks for each replicator, including: - Status of replicators (running, failed, and paused) - Number of source topics processed - Number of messages replicated per second - Message lag - Latency .. _rep-rename-C3-displayed-cluster-names: ------------------------------- Origin and Destination clusters ------------------------------- If your clusters and replication are configured properly, a minimum of two clusters, for example origin and destination, will be displayed in the Environment overview. .. figure:: ../images/c3-replicators-multi-cluster.png :scale: 70% .. tip:: If you do not see fully populated origin and destination clusters on |c3-short|, review the configuration details in :ref:`replicator_quickstart` and particularly the subtopic in that guide, :ref:`rep-quickstart-monitoring`. The clusters are rendered on |c3-short| with auto-generated names, based on your configuration. Optionally, you can edit the cluster names: #. Select a cluster. #. Click **Cluster settings** on the navigation menu. #. On the **General** tab, type a cluster name, and click **Save**. Here, you can also configure other :ref:`cluster wide settings ` such as broker defaults, |sbc|, and storage. In the associated example from the :ref:`Replicator Tutorial `, |crep| is set up to replicate topic data from origin to destination. The destination cluster settings are shown here. .. figure:: ../images/c3-replicators-cluster-name.png :scale: 65% The origin cluster settings are shown here. .. figure:: ../images/c3-replicators-cluster-name-origin.png :scale: 65% ---------------------------- Source and Replicated topics ---------------------------- Source topics are located on the origin cluster. There are several ways to navigate to these topics. For example: - With either cluster selected, click **Replicators** on the navigation menu, click a replicator, then choose **Source Topics** tab -> . - With either cluster selected, click **Replicators** on the navigation menu, click a replicator, then click the **Throughput** metrics card -> -> . - Select the origin cluster, click **Topics**, and drill down on a topic that way. .. tip:: Not all topics on the origin cluster will necessarily be replicated; only topics :ref:`configured for replication ` will be replicated. Also, some internal topics live on the origin cluster and are not replicated, such as ``__consumer_timestamps`` which the |crep| source consumer reads from to perform :ref:`consumer offset translation `. Replicated topics live on the destination cluster. To navigate to these topics: #. Select the destination cluster. #. Select **Topics** from the navigation menu. .. figure:: ../images/c3-replicator-topics-overview.png :scale: 60% #. Click a replicated topic. .. figure:: ../images/c3-replicated-topic-drilldown-overview.png :scale: 60% The following sections describe the available replication monitoring, metrics, and configuration views in more detail. Metrics and topics for a replicator ----------------------------------- To view details on a specific replicator, click a replicator in the list on the All Replicators page. .. figure:: ../images/c3-replicators-drilldown.png :scale: 60% The |crep| status page is displayed, showing: - Throughput metrics on message replication, including messages replicated per second, number of source topics, lag, and latency - Task overview, including total tasks, and number of running, failed, and paused tasks - List of |kconnect| workers showing status of each worker, along with message performance metrics per worker. You can scroll and search the list of workers. .. figure:: ../images/c3-replicators-status.png :scale: 60% ---------------------------------- Tasks associated with a replicator ---------------------------------- To view a list of tasks associated with a replicator, click the **Throughput** card for the selected replicator. .. figure:: ../images/c3-replicators-throughput.png :scale: 70% The list of tasks is displayed. .. figure:: ../images/c3-replicators-metrics.png :scale: 70% -------------------------- Message metrics for a task -------------------------- To view metrics for a specific task, click one of the tasks associated with the selected replicator. .. figure:: ../images/c3-replicators-task-drilldown.png :scale: 70% The task metrics are displayed at the top of the page. A list of source topics and associated metrics per topic is displayed on the lower half of the page. You can search and scroll through this list. .. figure:: ../images/c3-replicators-task-metrics-and-source-topics.png :scale: 70% -------------- Source topics -------------- To view details on a source topic (a topic on the origin cluster), select the **Source Topics** tab for the |crep|, then click a topic in the list. .. figure:: ../images/c3-replicators-source-topics-select.png :scale: 70% You can also view details on source topic by clicking a topic on the task metrics page or, with the origin cluster selected, click **Topics** from the navigation bar, then click a topic in that list. An overview of metrics for the source topic is displayed. .. figure:: ../images/c3-replicators-source-topic-drilldown-overview.png :scale: 70% You have the same monitoring and configuration options on both source and replicated topics. ----------------- Replicated topics ----------------- To view details on a replicated topic (on the destination cluster), select the Destination cluster, then click **Topics** on the |c3-short| menu to show all topics in the cluster. .. figure:: ../images/c3-replicator-topics-overview.png :scale: 70% Click one of the replicas in the topic list to bring up the overview page for that topic. .. figure:: ../images/c3-replicated-topic-drilldown-overview.png :scale: 70% Click the metrics cards on the **Overview** tab to drill down on the monitoring data for that topic. For example, click into **Production** to show producer metrics. .. figure:: ../images/c3-replicated-topic-metrics-production.png :scale: 60% Click **Consumption** to show consumer metrics. .. figure:: ../images/c3-replicated-topic-metrics-consumer-lag.png :scale: 60% Navigate using the other tabs to view and manage details on the topic. For example, this page shows **Messages** associated with the replicated topic. .. figure:: ../images/c3-replicator-topic-drilldown-messages.png :scale: 75% This page shows the topic configuration and provides options to re-configure it. (Click **Show full config** to get to advanced configuration options.) .. figure:: ../images/c3-replicator-topic-drilldown-config.png :scale: 80% Click **Edit settings** to modify basic configurations on the topic. Click **Switch to expert mode** for more options. .. figure:: ../images/c3-replicator-topic-drilldown-config-edit.png :scale: 70% You have the same monitoring and configuration options on both source and replicated topics, and can configure these independently. For example, you might choose to reduce the retention time or size on replicated topic but keep these the same on the source topic. To learn more about working with topics in |c3-short|, see :ref:`controlcenter_userguide_topics`. Suggested Reading ----------------- - :ref:`replicator_quickstart` - :ref:`rep-quickstart-monitoring` - :ref:`rep-tutorial-troubleshoot` - :ref:`Replicator Monitoring Setup and Reference ` - :ref:`controlcenter_troubleshooting` - :ref:`metrics_reporter` - :ref:`controlcenter_clients` - :ref:`replicator`