Register a Confluent Platform Connect cluster with USM

To monitor and govern your Confluent Platform Connect clusters from Confluent Cloud, you must first register them with Unified Stream Manager (USM). You can register a Connect cluster using either the Confluent Cloud Console or the Confluent CLI.

Warning

Before you register your Kafka Connect clusters, ensure that sensitive values, such as secrets and credentials, for all connectors on that cluster are configured only in properties of type PASSWORD. For USM, Confluent Platform masks configuration values of type PASSWORD before it sends them as part of the connector metadata to Confluent Cloud. To find which configuration options are of type PASSWORD, see Self-Managed Connectors for Confluent Platform for specific connectors.

Before you begin

Register a Connect cluster

To register a Connect cluster:

  1. Log in to your Confluent Platform cluster by using the Confluent Platform CLI.

  2. Get the Connect cluster ID by running the following command to list the available Kafka Connect clusters:

    confluent connect cluster list

  3. From the output, copy the ID of the Kafka Connect cluster that you want to register.

  4. In the Confluent Cloud Console, navigate to Environments and select your environment.

  5. In the navigation menu, click Clusters and select a Confluent Platform cluster.

  6. In the navigation menu, click Connectors.

  7. Click Register Connect cluster.

  8. Paste the Kafka Connect cluster ID that you copied.

  9. Click Finish.

Data from your connectors might take a few minutes to appear in the Confluent Cloud.

Alternatively, If connectors are already running but the cluster isn’t registered, a Register Connect cluster button is also available at the top right of the Connectors page.

Register an additional Connect cluster

To register an additional Connect cluster, do the following:

  1. Navigate to the Connectors page for your registered Confluent Platform cluster.

  2. Click Connector cluster list.

  3. Click Register Connect cluster.

  4. In your Confluent Platform environment, get the ID of the additional Connect cluster by running the following command:

    confluent connect cluster list

  5. Enter the Confluent Platform Connect cluster ID in the Confluent Cloud Console.

  6. Click Finish.

Configure Connect workers for metadata sync

After registering your cluster, the final step is to ensure your Connect workers can securely send metadata to Confluent Cloud. The required actions depend on your Confluent Platform deployment method.

For Confluent for Kubernetes or Ansible Playbooks for Confluent Platform deployments

If you deployed Confluent Platform using Confluent for Kubernetes or Ansible Playbooks, no additional configuration is needed. These tools automatically manage the settings required for metadata synchronization. After the registration is complete, your connector metadata begins to appear in Confluent Cloud.

For manual or other deployments

If you installed or manage Confluent Platform manually, you must update the properties file for each Kafka Connect worker in the registered cluster.

  1. Open the properties file for each Kafka Connect worker in the cluster that you registered.

  2. Confirm that the bootstrap.servers property correctly points to the Kafka cluster that the Connect workers use to store connector metadata.

  3. Add the following metadata properties to the worker configuration file to enable the metadata events service.

    confluent.connect.metadata.events.enabled=true
confluent.connect.metadata.events.client.base.url=https://usm-agent.connect.svc.cluster.local:10000
confluent.connect.metadata.events.ssl.key.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword}
confluent.connect.metadata.events.ssl.keystore.location=/mnt/sslcerts/tls-connect/keystore.p12
confluent.connect.metadata.events.ssl.keystore.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword}
confluent.connect.metadata.events.ssl.protocol=TLSv1.2
confluent.connect.metadata.events.ssl.truststore.location=/mnt/sslcerts/tls-connect/truststore.p12
confluent.connect.metadata.events.ssl.truststore.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword}

  4. Save your changes and perform a rolling restart of your Connect workers for the new configuration to take effect. After they restart, the workers began to send metadata to Confluent Cloud.

Deregister a Connect cluster

You can deregister a Connect cluster from USM using the Confluent CLI. Deregistering the Connect cluster using the Confluent Cloud Console is not supported.

  1. Log in to the Confluent CLI.

    confluent login

  2. Get the Environment ID using the following steps:

    1. In the Confluent Cloud Console, navigate to the Environments page and select your environment.
    2. Click the Details tab and copy the ID from the About section.

  3. Run the list command to see all USM registered Connect cluster clusters in your environment. This allows you to find the specific USM_CONNECT_CLUSTER_ID required for deregistration.

    confluent unified-stream-manager connect list --environment <YOUR_ENVIRONMENT_ID>

    From the output, identify the Connect cluster you want to remove and copy its ID. Note that a USM_CONNECT_CLUSTER_ID always starts with the prefix usmcc-.

  4. Use the confluent unified-stream-manager connect deregister command with the usmcc- ID you copied.

    confluent unified-stream-manager connect deregister <USM_CONNECT_CLUSTER_ID> --environment <YOUR_ENVIRONMENT_ID>

    The following example deregisters a Connect cluster with the ID usmcc-abc123 in the environment env-12345.

    confluent unified-stream-manager connect deregister usmcc-abc123 --environment env-12345

  5. The CLI prompts you for confirmation. Type y and press Enter to proceed.

    Are you sure you want to deregister USM Connect cluster "usmcc-abc123"? (y/n): y
Deregistered USM Connect cluster "usmcc-abc123".

Upon successful deregistration, the corresponding Connect cluster card is removed from the Confluent Cloud Console.

For a complete reference of related commands, see the confluent unified stream manager CLI documentation.

Related content