<a id="usm-quickstart"></a>

# Quick Start: USM with Confluent for Kubernetes

This quick start shows you how to deploy USM using Confluent for Kubernetes to register
your Confluent Platform cluster with Confluent Cloud.

#### NOTE
New to USM? See [Unified Stream Manager (USM) in Confluent Platform](overview.md#usm-overview) for a conceptual introduction
before you start this quick start.

In this guide, you:

* Deploy Confluent for Kubernetes and Confluent Platform
* Register your Confluent Platform cluster in Confluent Cloud
* Deploy the USM Agent
* Update Confluent Platform components to use the USM Agent
* Verify metrics are flowing to Confluent Cloud
* Register Connect cluster to monitor managed connectors
* Deregister Confluent Platform and Connect clusters

## Before you begin

Ensure your environment meets the following requirements:

* **Confluent Platform**: Version 7.9.6 or later within the 7.9.x series, or 8.1.0 or later.
  Confluent Platform 8.0.x is not supported. If you use an unsupported version, the USM Agent
  fails to connect to Confluent Cloud without reporting an error.
* **Kubernetes cluster**: A running Confluent for Kubernetes cluster on AWS EKS in a tier 1 region.
* **Confluent Cloud**: An organization existing in Confluent Cloud with the **Advanced Stream Governance** package.
* **Confluent CLI**: Version 4.39 or later. See [Install the Confluent CLI](https://docs.confluent.io/confluent-cli/current/install.html) for installation and upgrade instructions.

<a id="usm-qs-deploy-cp"></a>

## Step 1: Deploy Confluent for Kubernetes and Confluent Platform

The Confluent for Kubernetes operator must be installed on your Kubernetes cluster. If you have not installed it, see the [Confluent for Kubernetes](https://docs.confluent.io/operator/current/overview.html) documentation.

In this section, you deploy Confluent Platform 8.1 using Confluent for Kubernetes without the USM Agent. You add the agent after registering in Confluent Cloud and obtaining the agent configuration. Use the following example YAML.

1. Create a file named `confluent-platform-without-agent.yaml`. Expand the **YAML configuration** dropdown and copy the content into the file.

   ### **YAML configuration**

   ```yaml
   apiVersion: platform.confluent.io/v1beta1
   kind: KRaftController
   metadata:
     name: kraftcontroller
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     dataVolumeCapacity: 10G
     storageClass:
       name: gp2
     image:
       application: confluentinc/cp-server:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     replicas: 3
   ---
   apiVersion: platform.confluent.io/v1beta1
   kind: Kafka
   metadata:
     name: kafka
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     dataVolumeCapacity: 10G
     storageClass:
       name: gp2
     image:
       application: confluentinc/cp-server:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     replicas: 3
     metricReporter:
       enabled: false
     dependencies:
       kRaftController:
         clusterRef:
           name: kraftcontroller
   ---
   apiVersion: platform.confluent.io/v1beta1
   kind: Connect
   metadata:
     name: connect
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     build:
       type: onDemand
       onDemand:
         plugins:
           locationType: confluentHub
           confluentHub:
             - name: kafka-connect-datagen
               owner: confluentinc
               version: 0.5.2
     replicas: 3
     image:
       application: confluentinc/cp-server-connect:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     dependencies:
       kafka:
         bootstrapEndpoint: kafka:9071
   ```

   Replace the following placeholders in the file:
   * `<your-namespace>`: Your Kubernetes namespace
   * `<your-id>`: Your identifier for managed resources
2. Apply the configuration to your cluster:
   ```bash
   kubectl apply -f confluent-platform-without-agent.yaml
   ```

<a id="usm-qs-register-cp"></a>

## Step 2: Register the Confluent Platform cluster in Confluent Cloud

After deploying Confluent for Kubernetes and Confluent Platform, the next step is to register your cluster in Confluent Cloud to generate the credentials used by the USM Agent. For detailed instructions, see [Register your Confluent Platform cluster](https://docs.confluent.io/cloud/current/usm/register/overview.html).

1. Log in to Confluent Cloud Console at [https://confluent.cloud](https://confluent.cloud).
2. Navigate to **Environments**, select your environment, and click **Clusters**.
3. On the **Clusters** page, click **Start Unified Stream Manager setup**.
4. Read the benefits and click **Next** to proceed. If you have the Essentials governance package, you’ll be prompted to upgrade to the **Advanced Stream Governance** package (required for USM); complete the upgrade to continue.
5. Choose your agent deployment type:
   * **New agent**: Select this option to deploy a fresh agent in the environment where your Confluent Platform cluster resides.
   * **Existing agent**: Select this to reuse an agent already deployed in that environment.
6. Click **Get started** to proceed to the detailed registration steps.
7. Follow the guided flow in the UI to:
   1. **Configure private networking** for your AWS VPC and click **Next**. For detailed instructions, see [Configure private networking](https://docs.confluent.io/cloud/current/usm/register/usm-network.html).
   2. **Configure a service account** for the USM Agent (create a new one or select an existing one) and click **Next**. For detailed instructions, see [Configure a service account](https://docs.confluent.io/cloud/current/usm/register/usm-service-account.html).
8. On the **Deploy Unified Stream Manager agent** page, click **Generate configuration file**, and then click **Next**. For detailed instructions, see [Deploy the USM Agent](https://docs.confluent.io/cloud/current/usm/register/deploy-agent.html).

   The JSON file contains credentials for the USM Agent:
   ```json
   {
     "ENVIRONMENT_ID": "",
     "SERVICE_ACCOUNT_API_KEY": "",
     "SERVICE_ACCOUNT_API_SECRET": "",
     "SCHEMA_REGISTRY_API_KEY": "",
     "SCHEMA_REGISTRY_API_SECRET": "",
     "FRONTDOOR_URL": "",
     "SCHEMA_REGISTRY_URL": ""
   }
   ```
9. On the **Register Confluent Platform cluster** page, do the following:
   1. Get your Confluent Platform cluster ID. Log in to the Kubernetes cluster and run the following commands:
      ```bash
      kubectl get kafka -n <your-namespace>
      kubectl describe kafka <kafka-resource-name> -n <your-namespace>
      ```
   2. In **Confluent Platform cluster ID**, enter the cluster ID. The **Metadata** region is automatically selected based on your private network configuration. You cannot change this setting.
   3. In **Cluster name in Confluent Cloud**, enter a display name for the cluster.
   4. Click **Register cluster**.

After successful registration, a cluster card appears with a **Metrics delayed** status. This status changes to **Running** after metrics start flowing. Click **Close wizard** to exit the wizard.

<a id="usm-qs-deploy-agent"></a>

## Step 3: Deploy the USM Agent

If you selected **Existing agent** in the registration wizard, skip this section and [Upgrade Confluent Platform components to use the USM Agent](#usm-qs-upgrade-cp).

1. Create a credentials file named `setu-ccloud-cred.txt` with the following content:
   ```text
   username=<SERVICE_ACCOUNT_API_KEY>
   password=<SERVICE_ACCOUNT_API_SECRET>
   ```

   Replace `<SERVICE_ACCOUNT_API_KEY>` and `<SERVICE_ACCOUNT_API_SECRET>` with the values from the configuration file you downloaded during registration.
2. Create a Kubernetes secret from the credentials file:
   ```bash
   kubectl -n <your-namespace> create secret generic setu-ccloud-cred \
     --from-file=basic.txt=setu-ccloud-cred.txt
   ```
3. Create a file named `usmagent.yaml`. Expand the **YAML configuration** dropdown and copy the content into the file.

   ### **YAML configuration**

   ```yaml
   apiVersion: platform.confluent.io/v1beta1
   kind: USMAgent
   metadata:
     name: usm-agent
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     # Enable debug logs
     envVars:
       - name: USM_AGENT_CUSTOM_ARGS
         value: -l trace
     replicas: 1
     image:
       application: confluentinc/cp-usm-agent:1.0.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     externalAccess:
       type: loadBalancer
       loadBalancer:
         domain: my.domain
         prefix: usm-agent
     # Basic authentication
     #authentication:
     #type: basic/mtls
     # if auth type basic
     # basic:
     #secretRef: setu-basic-auth
     # TLS secret
     #tls:
     #secretRef: tls-usmagent
     confluentCloudClient:
       endpoint: <Frontdoor-URL-from-agent-config-downloaded-during-registration>
       environmentId: <env-id-from-agent-config-file>
       authentication:
         type: basic
         basic:
           secretRef: setu-ccloud-cred
   ```

   Replace the following placeholders:
   * `<your-namespace>`: Your Kubernetes namespace
   * `<your-id>`: Your identifier for managed resources
   * `<Frontdoor-URL-from-agent-config-downloaded-during-registration>`: The `FRONTDOOR_URL` value from the configuration file
   * `<env-id-from-agent-config-file>`: The `ENVIRONMENT_ID` value from the configuration file
4. Apply the USM Agent configuration:
   ```bash
   kubectl apply -f usmagent.yaml
   ```
5. Verify that the USM Agent connects to Confluent Cloud before proceeding to the next step:
   ```bash
   kubectl confluent cluster usmagent usm-agent-validz <usm-agent-resource-name> -n <your-namespace>
   ```

   If the command reports a connectivity or configuration issue, see
   [Validate Unified Stream Manager Agent connectivity](https://docs.confluent.io/operator/current/co-plugin-cli/kubectl_confluent_cluster_usmagent_usm-agent-validz.html)
   before continuing.

<a id="usm-qs-upgrade-cp"></a>

## Step 4: Upgrade Confluent Platform components to use the USM Agent

Update your Confluent Platform YAML to include the USM Agent dependency and upgrade your deployment.

1. Create a file named `confluent-platform-with-agent.yaml`. Expand the **YAML configuration** dropdown and copy the content into the file.

   ### **YAML configuration**

   ```yaml
   apiVersion: platform.confluent.io/v1beta1
   kind: KRaftController
   metadata:
     name: kraftcontroller
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     dataVolumeCapacity: 10G
     storageClass:
       name: gp2
     image:
       application: confluentinc/cp-server:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     replicas: 3
     dependencies:
       usmAgentClient:
         url: http://usm-agent.<your-namespace>.svc.cluster.local:10000
   ---
   apiVersion: platform.confluent.io/v1beta1
   kind: Kafka
   metadata:
     name: kafka
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     dataVolumeCapacity: 10G
     storageClass:
       name: gp2
     image:
       application: confluentinc/cp-server:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     replicas: 3
     metricReporter:
       enabled: false
     dependencies:
       kRaftController:
         clusterRef:
           name: kraftcontroller
       usmAgentClient:
         url: http://usm-agent.<your-namespace>.svc.cluster.local:10000
   ---
   apiVersion: platform.confluent.io/v1beta1
   kind: Connect
   metadata:
     name: connect
     namespace: <your-namespace>
     labels:
       confluent.io/managed-by: "<your-id>"
   spec:
     podTemplate:
       labels:
         confluent.io/managed-by: "<your-id>"
     build:
       type: onDemand
       onDemand:
         plugins:
           locationType: confluentHub
           confluentHub:
             - name: kafka-connect-datagen
               owner: confluentinc
               version: 0.5.2
     replicas: 3
     image:
       application: confluentinc/cp-server-connect:8.1.0
       init: confluentinc/confluent-init-container:3.1.0
       pullPolicy: IfNotPresent
     #pullSecretRef:
     # - confluent-registry
     dependencies:
       kafka:
         bootstrapEndpoint: kafka:9071
       usmAgentClient:
         url: http://usm-agent.<your-namespace>.svc.cluster.local:10000
   ```

   Replace the following placeholders:
   * `<your-namespace>`: Your Kubernetes namespace
   * `<your-id>`: Your identifier for managed resources
2. Apply the updated configuration:
   ```bash
   kubectl apply -f confluent-platform-with-agent.yaml
   ```

<a id="usm-qs-verify-metrics"></a>

## Step 5: Verify metrics are flowing to Confluent Cloud

Verify that your cluster is successfully registered and metrics are flowing to Confluent Cloud.

1. Log in to Confluent Cloud and navigate to **Environments** > **Clusters**.
2. Locate your registered Confluent Platform cluster. After metrics start flowing, the cluster card status changes from **Metrics delayed** to **Running**.
3. Click on the cluster to view detailed metrics and verify they are populated.
4. Add metadata to your resources:
   * Attach descriptions and tags to your cluster, topics, and connectors
   * For topics, define owner details such as name and email
5. Navigate to **Data Portal** to discover your Confluent Platform resources, such as topics and connectors.
6. Inspect data flow for your topics using **Stream lineage**.

<a id="usm-qs-register-connect"></a>

## Step 6: Register Connect cluster to monitor managed connectors

Register your Connect cluster to monitor managed connectors.

1. In the Confluent Cloud console, navigate to the **Connector** page of your registered Confluent Platform cluster.
2. Click **Register Connect Cluster**.
3. Retrieve your Connect cluster group ID:
   ```bash
   kubectl describe connect <connect_resource_name> -n <your-namespace>
   ```

   Look for the `group.id` value in the output. This is your Connect cluster group ID.
4. Enter your `group.id` value from the previous step in the pop-up and click **Register**.
5. After the Connect cluster is successfully registered, connector cards start appearing in the Confluent Cloud console.

<a id="usm-qs-deregister"></a>

## Step 7: Deregister clusters and clean up

Use the Confluent CLI to deregister your Confluent Platform and Connect clusters.
Deregistration is only available through the Confluent CLI.

### Deregister Confluent Platform cluster

Deregister your Confluent Platform cluster from Confluent Cloud.

1. Log in to Confluent Cloud with your Confluent CLI:
   ```bash
   confluent login
   ```
2. Get the Environment ID using the following steps:
   1. In the Confluent Cloud Console, navigate to the **Environments** page and select your environment.
   2. Click the **Details** tab and copy the ID from the **About** section.
3. List your Confluent Platform clusters to retrieve the unique cluster ID:
   ```bash
   confluent unified-stream-manager kafka list --environment <Environment-ID>
   ```
4. Identify the unique Confluent Platform cluster ID starting with `usmkc` for the cluster you want to deregister.
5. Deregister the cluster:
   ```bash
   confluent unified-stream-manager kafka deregister <usmkc-xxx> --environment <Environment-ID>
   ```

   After successful deregistration, the cluster card disappears from the Confluent Cloud.

### Deregister Connect cluster

Deregister your Connect cluster from Confluent Cloud.

1. List your Connect clusters:
   ```bash
   confluent unified-stream-manager connect list --environment <Environment-ID>
   ```
2. Identify the unique cluster ID starting with `usmcc-` for the Connect cluster you want to deregister.
3. Deregister the Connect cluster:
   ```bash
   confluent unified-stream-manager connect deregister <usmcc-xxxx> --environment <Environment-ID>
   ```

## Related content

* [Unified Stream Manager (USM) in Confluent Platform](overview.md#usm-overview)
* [Getting Started with USM](get-started.md#usm-start)
* [Register your Confluent Platform cluster](https://docs.confluent.io/cloud/current/usm/register/overview.html)