# Configure, Deploy, and Manage Unified Stream Manager Using Confluent for Kubernetes Confluent Unified Stream Manager connects customer managed clusters with Confluent Cloud to enable Confluent Cloud features for Confluent Platform clusters. The Unified Stream Manager Agent acts as a centralized proxy/gateway for Kafka, and Confluent for Kubernetes (CFK) acts as a tool to deploy Unified Stream Manager Agent in a Kubernetes environment. This topic presents the steps and guidance for deploying Unified Stream Manager in Confluent Platform using CFK. This process is part of the [Register your Confluent Platform Kafka cluster in Confluent Cloud](http://docs.confluent.io/platform/current/usm/get-started.html#registration-process-overview) workflow. Review the steps provided in the above topic before proceeding with Unified Stream Manager deployment. ## Requirements and considerations Before deploying Unified Stream Manager with Confluent for Kubernetes, ensure you have the following prerequisites and understand the key considerations: - Ensure your Kubernetes environment is set up and CFK is installed. Use the appropriate version that supports your Confluent Platform and Unified Stream Manager requirements. - Unified Stream Manager Agent to Confluent Cloud communication only supports basic authentication. - Unified Stream Manager Agent to Confluent Platform components communication supports basic authentication or mTLS. Both together are not supported. There are 3 authentication combinations possible: basic authentication, basic authentication + TLS, and mTLS. ## Configure and deploy Unified Stream Manager 1. Configure **Unified Stream Manager Agent** using the USMAgent custom resource (CR), and then apply the CR using the `kubectl apply` command. ```yaml kind: USMAgent spec: replicas: image: application: --- [1] init: --- [2] authentication: type: --- [3] basic: secretRef: --- [4] tls: secretRef: --- [5] confluentCloudClient: endpoint: --- [6] environmentId: --- [7] authentication: type: --- [8] basic: secretRef: --- [9] externalAccess: --- [10] type: --- [11] loadBalancer: --- [12] nodePort: --- [13] ``` * [1] Set to the Unified Stream Manager application image. * [2] Set to the Unified Stream Manager CFK init container image. * [3] Set to `basic` or `mtls`. * [4] Required for basic authentication. Specify the secret containing the basic authentication credentials. * [5] For TLS between Unified Stream Manager Agent and Confluent Platform components, specify the secret containing the TLS certificate, the key, and the root certificate authority (CA) files. * [6] Specify the Confluent Cloud endpoint. The Confluent Cloud endpoint is available in the output file generated when you perform the first step in the registration process. See [Generate the configuration file](https://docs.confluent.io/cloud/current/usm/register/deploy-agent.html#generate-and-download-the-configuration-file). This step has to be completed before you deploy Unified Stream Manager Agent. * [7] Specify the Confluent Cloud Environment ID. The Environment ID is available in the output file generated when you perform the first step in the registration process. See [Generate the configuration file](https://docs.confluent.io/cloud/current/usm/register/deploy-agent.html#generate-and-download-the-configuration-file). This step has to be completed before you deploy Unified Stream Manager Agent. * [8] Set to `basic` for basic authentication. * [8] Set to `basic` for basic authentication. * [9] Required for basic authentication. Specify the secret containing the Cloud Api key and secret. The values are available in the output file generated when you perform the first step in the registration process. See [Generate the configuration file](https://docs.confluent.io/cloud/current/usm/register/deploy-agent.html#generate-and-download-the-configuration-file). This step has to be completed before you deploy Unified Stream Manager Agent. * [10] Optional. External access is optional for Unified Stream Manager Agent. * [11] Set to `loadBalancer`, `nodePort`, or `route` to specify the external access type. For OpenShift support, see [USM support for OpenShift](#co-configure-usm-openshift). * [12] Required when externalAccess type ([11]) is set to `loadBalancer`. For configuring load balancers, see [Configure Load Balancers for Confluent Platform in Confluent for Kubernetes](co-loadbalancers.md#co-loadbalancers). * [13] Required when externalAccess type ([11]) is set to `nodePort`. For configuring node ports, see [Configure Node Ports to Access Confluent Platform Components Using Confluent for Kubernetes](co-nodeports.md#co-nodeports). 2. Configure the **client-side properties** in Kafka, KRaft, and Connect for communication with Unified Stream Manager Agent, and then apply the changes to the CRs with the `kubectl apply` command. ```yaml spec: dependencies: usmAgentClient: url: --- [1] clientMetricsEnabled: --- [2] authentication: type: --- [3] basic: secretRef: --- [4] dpic: --- [5] tls: enabled: --- [6] secretRef: --- [7] dpic: --- [8] ``` * [1] Specify the Unified Stream Manager Agent URL. * [2] Optional. Set to `true` to enable client metrics. Supported in CFK 3.2.x with Confluent Platform 7.9.6 or later, 8.1.2 or later, and 8.2.0 or later. * [3] Set to `basic` or `mtls` to specify the authentication type. See [Basic authentication credentials](co-authenticate-cp.md#co-basic-server-creds) for the required format. * [4] Specify the secret containing the basic authentication credentials. * [5] Specify the basic authentication credential secret path in the container. For details, see [Provide secrets in HashiCorp Vault](co-credentials.md#co-directory-path-in-container). * [6] Set to `true` or `false` to enable or disable TLS. * [7] Specify the secret containing the TLS certificate. * [8] Specify the TLS certificate secret path in the container. For details, see [Provide secrets in HashiCorp Vault](co-credentials.md#co-directory-path-in-container). 3. [Register your Confluent Platform Connect cluster in Confluent Cloud](http://docs.confluent.io/cloud/current/usm/register-connect.html). You can use the following options to retrieve the Connect cluster ID (also known as the group ID) that is required to register the Connect cluster in Confluent Cloud: * Use the `kubectl describe connect` command, and fetch the Group ID under the `Status` section. * If you have the Confluent CLI installed, you can use the command `confluent connect cluster list` as described in the above registration topic. ## USM support for OpenShift Confluent for Kubernetes supports deploying USM Agent on OpenShift clusters. This section describes how to configure external access to the USM Agent using OpenShift routes, which provide a way to expose your services outside the OpenShift cluster. ### Prerequisites - An OpenShift cluster - CFK installed on the cluster ### Configure external access using OpenShift routes OpenShift routes require TLS to be enabled. The USM Agent is accessible at `https://.:443`. For example, `https://usm-agent.apps.example.com:443`. #### Set up namespace and security policies 1. Create the namespace: ```bash oc create namespace confluent ``` 2. Configure security context constraints: ```bash oc adm policy add-scc-to-user anyuid -z default -n confluent oc adm policy add-scc-to-user anyuid -z confluent-for-kubernetes -n confluent ``` #### Create TLS certificates and secrets You can create custom certificates (for production) or use auto-generated certificates (for testing). For custom certificates: 1. Create a certificate directory and generate the CA: ```bash mkdir -p ~/usm-certs && cd ~/usm-certs openssl genrsa -out ca-key.pem 2048 openssl req -x509 -new -nodes -key ca-key.pem -days 365 -out ca.pem -subj "/CN=USMAgentCA/O=Confluent" ``` 2. Create a SAN configuration file. Replace `apps.example.com` with your OpenShift domain: ```bash cat > usm-agent-san.cnf << 'EOF' [req] distinguished_name = req_distinguished_name req_extensions = v3_req prompt = no [req_distinguished_name] CN = usm-agent [v3_req] keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth subjectAltName = @alt_names [alt_names] DNS.1 = usm-agent DNS.2 = usm-agent.confluent DNS.3 = usm-agent.confluent.svc DNS.4 = usm-agent.confluent.svc.cluster.local DNS.5 = *.apps.example.com DNS.6 = usm-agent.apps.example.com EOF ``` 3. Generate the server certificate: ```bash openssl genrsa -out usm-agent-key.pem 2048 openssl req -new -key usm-agent-key.pem -out usm-agent.csr -config usm-agent-san.cnf openssl x509 -req -in usm-agent.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial \ -out usm-agent.pem -days 365 -extensions v3_req -extfile usm-agent-san.cnf ``` 4. Create a JKS truststore: ```bash keytool -importcert -alias usm-agent-ca -file ca.pem -keystore truststore.jks \ -storepass changeit -noprompt ``` 5. Create secrets: ```bash oc -n confluent create secret tls usm-agent-tls \ --cert=$HOME/usm-certs/usm-agent.pem \ --key=$HOME/usm-certs/usm-agent-key.pem oc -n confluent patch secret usm-agent-tls \ -p "{\"data\":{\"ca.crt\":\"$(base64 < $HOME/usm-certs/ca.pem | tr -d '\n')\"}}" oc -n confluent create secret generic usm-ca-trust \ --from-file=truststore.jks=$HOME/usm-certs/truststore.jks \ --from-literal=jksPassword.txt='jksPassword=changeit' ``` 6. Verify secrets: ```bash oc -n confluent get secrets | grep -E "usm-agent-tls|usm-ca-trust" ``` Example output: ```text usm-agent-tls kubernetes.io/tls 3 10s usm-ca-trust Opaque 2 5s ``` For auto-generated certificates, set `spec.tls.autoGeneratedCerts: true` and omit `secretRef` in the custom resource. #### Deploy USM Agent 1. Create a USM Agent CR YAML file: These are some of the configuration fields: * `autoGeneratedCerts`: Set to `false` for custom certificates. Set to `true` for auto-generated. * `secretRef`: The secret name. If you are using custom certificates, set ths to `usm-agent-tls`. * `domain`: Your OpenShift cluster domain. For example, `apps.example.com`. * `prefix`: The route prefix. The default is `usm-agent`. * `wildcardPolicy`: `Subdomain` or `None`. The default is `None`. * `annotations`: OpenShift route annotations. This is optional. Here is an example configuration: ```yaml apiVersion: platform.confluent.io/v1beta1 kind: USMAgent metadata: name: usm-agent namespace: confluent spec: tls: autoGeneratedCerts: false secretRef: usm-agent-tls externalAccess: type: route route: domain: apps.example.com prefix: usm-agent annotations: haproxy.router.openshift.io/timeout: 5500ms wildcardPolicy: None replicas: 1 image: application: confluentinc/cp-usm-agent:latest init: confluentinc/confluent-init-container:latest confluentCloudClient: endpoint: https://api.confluent.cloud:443 environmentId: env-xxxxx authentication: type: basic basic: secretRef: ccloud-credentials ``` 2. Apply the configuration: ```bash oc apply -f ``` 3. Get the OpenShift router external IP: ```bash oc get svc --namespace openshift-ingress ``` Example output: ```text NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE router-default LoadBalancer 172.30.84.52 20.189.181.8 80:31294/TCP,443:32145/TCP 42h ``` 4. Get the route hostname: ```bash oc get routes | grep usm-agent ``` Example output: ```text NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD usm-agent-bootstrap usm-agent.apps.example.com usm-agent external passthrough None ``` 5. Add the DNS entry in your DNS provider mapping the external IP to the route hostname: ```text 20.189.181.8 usm-agent.apps.example.com ``` 6. Test the USM Agent endpoint: ```bash ROUTE_HOST=$(oc get route usm-agent-bootstrap -o jsonpath='{.spec.host}') curl -k "https://${ROUTE_HOST}" ``` For additional validation steps, see [Validate connections](co-routes.md#co-routes-validate).