.. _replicator_verifier: Verify |crep| Configuration for |cp| ==================================== The |crep| verifier is a command line tool for identifying issues with a |crep| configuration and recommending remedial actions. This tool accepts a connector or an executable configuration and runs a number of checks against the clusters referenced. This can be useful for: * Preparing new deployments - the verifier tool provides confidence in a new |crep| flow configuration without actually deploying the |crep| * Diagnosing |crep| issues in existing flows - the verifier tool can quickly identify the area of the issue without redeploying the flow .. tip:: The |crep| verifier is available as part of a :ref:`Confluent Platform installation `, included in the ``$CONFLUENT_HOME/bin`` directory. Once you have |cp| installed and `up-and-running `__, type **replicator-verifier** at the command line to get a read-out on the command and its options. -------------------- Running the verifier -------------------- Run the |crep| verifier using the configuration from three sources: .. note:: |crep| verifier should be run from the same host that will run |crep| * From |crep| executable:: replicator-verifier \ --consumer.config ./consumer.properties \ --producer.config ./producer.properties \ --replication.config ./replication.properties * From a connector json:: replicator-verifier \ --replicator-json ./replicator.json * From a deployed connector:: replicator-verifier \ --connect-url localhost:8083 \ --connector-name replicator On completion the |crep| verifier will produce a summary of checks performed and their results: .. code:: bash [2020-02-28 12:39:02,252] INFO All checks completed. (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,252] INFO Source cluster describe configs check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Connectivity check Destination : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO topic.whitelist/topic.blacklist check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO License manager check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Consumer offset commit check source : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Consumer offset commit check destination : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Destination topic ACL check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Topic regex check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Source topic ACL check : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Connectivity check Source : PASSED (io.confluent.connect.replicator.Verifier) [2020-02-28 12:39:02,253] INFO Verifier run complete. (io.confluent.connect.replicator.Verifier) Any failed checks are accompanied by guidance explaining the relevant parameters to resolve the issue: .. code:: bash [2020-02-28 12:41:46,846] INFO License manager check : FAILED By default, Replicator will assume the license is stored in the _confluent-command topic on the destination cluster. Ensure that the properties prefixed "dest.kafka." refer to your license cluster, topic and principal. Alternatively, these can be overridden using the "confluent.topic." prefix (e.g. confluent.topic.bootstrap.servers). (io.confluent.connect.replicator.Verifier) .. important:: |crep| verifier does not currently support monitoring interceptors. Configurations like `*.interceptor.classes` should be removed from |crep| configurations before running the verifier. -------------------------------------------- Command Line Parameters of |crep| Verifier -------------------------------------------- The available command line parameters are: =================================== ================================ ========================================================= Command line parameter Value Description =================================== ================================ ========================================================= ``--connect-url`` URL for the Connect instance running connectors ``--connector-name`` Name for the replicator connector ``--replicator-json`` JSON file containing configuration for replicator running as a connector ``--producer.config`` Producer config file for replicator running in executable mode ``--consumer.config`` Consumer config file for replicator running in executable mode ``--replication.config`` Replicator config file for replicator running in executable mode ``--destination-test-topic`` Name of destination topic, which is an override for the randomized topic name used for dry run checks at the destination cluster (this topic must already exist) =================================== ================================ ========================================================= ---------------- Checks performed ---------------- |crep| verifier performs the following checks to determine that the configuration provided is valid. Non-intrusive checks -------------------- * Source cluster connectivity check - Confirms |crep| can connect to the source cluster. * Destination cluster connectivity check - Confirms |crep| can connect to the destination cluster. * Topic whitelist/blacklist check - Confirms the list of topics to be whitelisted/blacklisted are valid and not conflicting. * Topic regex check - Confirms any regex topic matching pattern is valid and displays they set of topics matched. * License check - Confirms that a valid license to use |crep| is present. * Source topic read checks - Confirms that |crep| can read the configured source topics. * Source topic describe check - Confirms that |crep| can describe the configured source topics. * Consumer timestamps test - Confirms |crep| can read/write timestamps to the internal timestamps topic for offset translation. Intrusive checks ---------------- .. note:: Intrusive checks will temporarily change the state of the cluster they are performed on. These changes take the form of topic creation/manipulation and consumer offset commits. |crep| verifier will prompt for approval before running these checks and as such should be run in interactive mode. * Destination topic check - Confirms that |crep| can perform the following required actions on a sample destination topic: 1. Create 2. Describe 3. Describe Configs 4. Alter 5. Produce * Source cluster offset commit - Confirms that |crep| can commit consumer offsets to the source cluster * Destination cluster offset commit - Confirms that |crep| can commit consumer offsets to the destination cluster