# Confluent Platform Kafka as the Metadata Cluster This page covers registering a self-managed Connect cluster that produces to and consumes from a Confluent Platform Kafka cluster, which serves as its metadata cluster, using Unified Stream Manager (USM). For the Confluent Cloud Kafka case instead, see [Confluent Cloud Kafka as the Metadata Cluster](cloud-kafka.md#register-connect-cloud). ## Before you begin * Ensure that you have [registered a Confluent Platform cluster to Confluent Cloud using USM](../register/overview.md#cloud-usm-register). * Get the Kafka cluster details that the Connect worker uses as the metadata cluster. ## Register a Connect cluster against Confluent Platform Kafka To register a self-managed Connect cluster: ### Confluent Cloud Console 1. Log in to your Confluent Platform cluster by using the Confluent Platform CLI. 2. Get the Connect cluster ID by running the following command to list the available Kafka Connect clusters: ```bash confluent connect cluster list ``` 3. From the output, copy the ID of the Kafka Connect cluster that you want to register. 4. In the Confluent Cloud Console, navigate to **Environments** and select your environment. ![image](images/usm-env.png) 5. In the navigation menu, click **Clusters** and select a Confluent Platform cluster. ![image](images/usm-cluster.png) 6. In the navigation menu, click **Connectors**. ![image](images/usm-connect-register.png) 7. Click **Register Connect cluster**. ![image](images/usm-connect-registerselect.png) 8. Paste the Kafka Connect cluster ID that you copied. ![image](images/usm-connector.png) 9. Click **Finish**. Data from your connectors might take a few minutes to appear in the Confluent Cloud. Alternatively, if connectors are already running but the cluster isn’t registered, a **Register Connect cluster** button is also available on the **Connectors** page. ### Confluent CLI To register a Confluent Platform Connect cluster, use the `confluent usm connect-cluster register` command. ```bash confluent usm connect-cluster register --kafka-cluster-id --cloud --region ``` For `--cloud`, specify `aws` or `azure`, and for `--region`, specify a region from [Configure AWS Private Networking for USM Agents](../register/usm-network.md#usm-network) (AWS) or [Configure Azure Private Networking for USM Agents](../register/usm-network-azure.md#usm-network-azure) (Azure). For example, to register a Kafka cluster with ID `4k0R9d1GTS5tI9f4Y2xZ0Q` and connector ID `connect-group-xyz123` in AWS region `us-east-1`: ```bash confluent usm connect-cluster register connect-group-xyz123 --kafka-cluster-id 4k0R9d1GTS5tI9f4Y2xZ0Q --cloud aws --region us-east-1 ``` The following is an example of the output: ```text +--------------------------------+--------------------------+ | ID | usmcc-global-ghijkl67890 | | Confluent Platform Connect | connect-group-xyz123 | | Cluster | | | USM Kafka Cluster ID | usmkc-global-abcdef12356 | | Confluent Platform Kafka | 4k0R9d1GTS5tI9f4Y2xZ0Q | | Cluster ID | | | Cloud | AWS | | Region | us-east-1 | | Environment | env-stgcqqd737 | +--------------------------------+--------------------------+ ``` To register a Connect cluster in Azure region `centralus`: ```bash confluent usm connect-cluster register connect-group-xyz123 --kafka-cluster-id 9f4Y2xZ0Q4k0R9d1GTS5tI --cloud azure --region centralus ``` The following is an example of the output: ```text +--------------------------------+--------------------------+ | ID | usmcc-global-lkjihg09876 | | Confluent Platform Connect | connect-group-xyz123 | | Cluster | | | USM Kafka Cluster ID | usmkc-global-fedcba54321 | | Confluent Platform Kafka | 9f4Y2xZ0Q4k0R9d1GTS5tI | | Cluster ID | | | Cloud | AZURE | | Region | centralus | | Environment | env-stgcqqd737 | +--------------------------------+--------------------------+ ``` After registration, the **Connectors** page populates with all connectors that are associated with that cluster. For more information about monitoring these connectors, see [Monitor Confluent Platform Connectors with USM](../monitor/connectors.md#platform-connectors). For a complete reference of related commands, see the [confluent usm CLI documentation](https://docs.confluent.io/confluent-cli/current/command-reference/usm/index.html#confluent-usm). ## Register an additional Connect cluster To register an additional Connect cluster, do the following: 1. Navigate to **Connectors** for your registered Confluent Platform cluster. 2. Click **Connector cluster list**. 3. Click **Register Connect cluster**. 4. In your Confluent Platform environment, get the ID of the additional Connect cluster by running the following command: ```bash confluent connect cluster list ``` 5. Enter the Confluent Platform Connect cluster ID in the Confluent Cloud Console. 6. Click **Finish**. ## Configure Connect workers for metadata sync After registering your cluster, you must ensure your Connect workers can securely send metadata to Confluent Cloud. The required actions depend on your Confluent Platform deployment method. ### For Confluent for Kubernetes or Ansible Playbooks for Confluent Platform deployments If you deployed Confluent Platform using [Confluent for Kubernetes](https://docs.confluent.io/operator/current/overview.html) or [Ansible Playbooks](https://docs.confluent.io/ansible/current/overview.html), no additional configuration is needed. These tools automatically manage the settings required for metadata synchronization. After the registration is complete, your connector metadata begins to appear in Confluent Cloud. #### NOTE Confluent Platform Connect versions `7.9.6`, `7.9.7`, `7.9.8`, `8.1.2`, `8.1.3`, `8.1.4`, `8.2.0`, `8.2.1`, `8.2.2`, and `8.3.0` have a known issue that prevents Confluent for Kubernetes and Ansible Playbooks for Confluent Platform deployments from emitting the metrics required for USM. If connector metrics do not appear in Confluent Cloud after registration, see [Issue: Connect metrics are missing from USM](../troubleshoot.md#troubleshoot-connect-metrics-missing) to apply the workaround. ### For manual or other deployments If you installed or manage Confluent Platform manually, you must update the properties file for each Kafka Connect worker in the registered cluster. 1. Open the properties file for each Kafka Connect worker in the cluster that you registered. 2. Confirm that the `bootstrap.servers` property correctly points to the Kafka cluster that the Connect workers use to store connector metadata. 3. Add the following metadata properties to the worker configuration file to enable the metadata events service. ```properties confluent.connect.metadata.events.enabled=true confluent.connect.metadata.events.client.base.url=https://usm-agent.connect.svc.cluster.local:10000 confluent.connect.metadata.events.ssl.key.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword} confluent.connect.metadata.events.ssl.keystore.location=/mnt/sslcerts/tls-connect/keystore.p12 confluent.connect.metadata.events.ssl.keystore.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword} confluent.connect.metadata.events.ssl.protocol=TLSv1.2 confluent.connect.metadata.events.ssl.truststore.location=/mnt/sslcerts/tls-connect/truststore.p12 confluent.connect.metadata.events.ssl.truststore.password=${file:/mnt/sslcerts/tls-connect/jksPassword.txt:jksPassword} ``` 4. Save your changes. 5. Perform a rolling restart of your Connect workers. After they restart, the workers send metadata to Confluent Cloud. ## Related content * [Unified Stream Manager in Confluent Cloud](../overview.md#cloud-usm-overview) * [Configure AWS Private Networking for USM Agents](../register/usm-network.md#usm-network) * [Configure Azure Private Networking for USM Agents](../register/usm-network-azure.md#usm-network-azure) * [Configure a Service Account for the USM Agent](../register/usm-service-account.md#usm-sa) * [Deploy the USM Agent](../register/deploy-agent.md#usm-agent) * [Monitor Confluent Platform Connectors with USM](../monitor/connectors.md#platform-connectors) * [Deregister a Connect cluster](deregister.md#usm-deregister-connect)