Decommission an LKC

Decommission a Logical Kafka Cluster (LKC) to permanently delete it from a multi-tenant Physical Kafka Cluster (PKC) and free its capacity. Platform operators perform this procedure when they no longer need an LKC.

Caution

You can’t undo decommissioning an LKC. Deleting an LKC permanently deletes its topics and access control lists (ACLs) and stops its clients from connecting. Complete Prepare the LKC for decommissioning before you continue.

Prerequisites

Before you begin, ensure you have the following:

kubectl access to delete the LogicalKafkaCluster custom resource (CR) and the ecosystem-component, ConfluentRolebinding, and KafkaTopic CRs that reference the LKC.
The KafkaRestClass that you used to create the LKC.

Step 1: Prepare the LKC for decommissioning

Notify the TenantAdmin that you plan to decommission their LKC, and give them enough lead time to complete the following before you continue:

Export any data their team needs to keep. Confluent Private Cloud doesn’t back up an LKC’s topics before deleting them, so exporting is the LKC owner’s own responsibility. Cluster Linking can mirror the LKC’s topics to another cluster first. For more information, see the Migrate Existing Clusters to Multi-Tenant LKCs page. After mirroring completes, promote or fail over the mirror topics on the destination cluster before you continue.

Delete every cluster link that uses this LKC as a source or destination, including any link that mirrors the LKC’s topics to another cluster. After the LKC is deactivated, its bootstrap URL and credentials no longer work, so you can’t remove a link through the LKC:

kafka-cluster-links --bootstrap-server <lkc-bootstrap-endpoint> \
  --command-config <lkc-command-config> --list

kafka-cluster-links --bootstrap-server <lkc-bootstrap-endpoint> \
  --command-config <lkc-command-config> --delete \
  --link <link-name>

Export any schemas, connector configurations, or ksqlDB queries from ecosystem components dedicated to this LKC. Delete dependent resources deletes these components.
Stop, or redirect elsewhere, any producer and consumer applications that still connect to this LKC.

Step 2: Delete dependent resources

Before you delete the LKC, delete the following. CFK doesn’t clean these up automatically:

Any Kafka Connect, Schema Registry, ksqlDB, or REST Proxy CR dedicated to this LKC. For example, kubectl delete connect <name> -n <your-namespace>. For more information about ecosystem components, see Deploy Ecosystem Components on a Multi-Tenant PKC.
Any ConfluentRolebinding or KafkaTopic CR that references this LKC from a Kubernetes namespace other than the LKC’s own. When you delete the LKC, CFK automatically deletes any ConfluentRolebinding or KafkaTopic CR in the LKC’s own namespace, such as the TenantAdmin binding from Step 7 of Onboard an LKC to a Multi-Tenant Physical Kafka Cluster and developer bindings from Step 1 of Onboard a Developer to an LKC. This automatic cleanup doesn’t extend to a role binding or topic CR in a different namespace. To find these CRs, run:
```
kubectl get confluentrolebinding,kafkatopic -A \
  -o custom-columns='KIND:.kind,NAMESPACE:.metadata.namespace,NAME:.metadata.name,REF_KIND:.spec.kafkaClusterRef.kind,REF_NAME:.spec.kafkaClusterRef.name,REF_NAMESPACE:.spec.kafkaClusterRef.namespace'
```
Delete each result whose reference names this LKC but whose own NAMESPACE column differs from the LKC’s, with kubectl delete <kind> <name> -n <namespace>.

Step 3: Delete the `LogicalKafkaCluster` CR

Run the following command, where <lkc-name> is the LKC’s Kubernetes object name, not the LKC display name. For the difference between the two, see Step 3 of Onboard an LKC to a Multi-Tenant Physical Kafka Cluster:

kubectl delete logicalkafkacluster <lkc-name> -n <your-namespace>

This command returns when deletion completes, which normally takes seconds. While it’s in progress, status.state reads DELETE-REQUESTED. If the command doesn’t complete, see Troubleshooting.

Caution

Delete the LogicalKafkaCluster CR itself to decommission an LKC. Don’t delete the LKC’s Kubernetes namespace or the PKC’s Kafka CR as a shortcut. Doing so can leave the LKC active, with its topics never deleted and its capacity never freed.

Step 4: Wait for broker-side teardown

Deleting the LKC CR doesn’t immediately delete the LKC’s topics and ACLs. The broker removes them in the background, which can take several minutes, and longer for an LKC with many topics. Wait a few minutes before you verify the decommissioning.

Verification

Confirm that kubectl get logicalkafkacluster <lkc-name> -n <your-namespace> returns a not found error, which shows that deleting the LogicalKafkaCluster CR completed.
Confirm that the LKC no longer appears in Confluent Control Center after you delete the LogicalKafkaCluster CR.
Using a principal that’s still valid, such as a platform operator identity rather than one you revoked in Prepare the LKC for decommissioning, confirm that a client connecting to the LKC’s former bootstrap URL can’t connect. For more information about connection failures, see the Connection and authentication errors section of Onboard a Developer to an LKC.

Troubleshooting

`kubectl delete` doesn’t complete

By default, kubectl delete logicalkafkacluster blocks your terminal until deletion completes, which Delete the LogicalKafkaCluster CR says normally takes seconds. If your terminal is still blocked after that, open a second terminal and check status.state instead of waiting:

kubectl get logicalkafkacluster <lkc-name> -n <your-namespace> \
  -o jsonpath='{.status.state}'

While a failure persists, status.state can alternate between ERROR and DELETE-REQUESTED as CFK retries, so don’t rely on its value. Run kubectl get logicalkafkacluster <lkc-name> -n <your-namespace> -o yaml and check the CR’s status conditions for a DeleteFailed condition and its message. If there’s no DeleteFailed condition, the deletion is still in progress, so give it a little longer. Common causes of a failure include:

CFK can’t reach Metadata Service (MDS), or its KafkaRestClass doesn’t have a resolvable endpoint. Confirm that MDS is reachable and that the KafkaRestClass CFK uses is configured correctly, then wait. CFK retries automatically, so you don’t need to run kubectl delete again.
The KafkaRestClass CFK uses doesn’t authenticate as a principal with the OrganizationAdmin role on the PKC, and MDS rejects the deactivation request. Grant the OrganizationAdmin role on the PKC to that principal. CFK retries automatically, so you don’t need to run kubectl delete again.

If neither applies and status.state stays at ERROR, contact Confluent Support before you take further action.

Orphaned dependent resources

If you deleted the LKC before completing Delete dependent resources, delete the remaining dedicated Kafka Connect, Schema Registry, ksqlDB, and REST Proxy CRs, and any cross-namespace ConfluentRolebinding or KafkaTopic CRs, individually. Deleting them after the LKC is already gone doesn’t cause an error.

Stale cluster links

If you deleted the LKC before completing Prepare the LKC for decommissioning, deactivating the LKC moved any cluster link that used it as a source or destination into a failed state, without removing the link record itself. The LKC’s own bootstrap URL and credentials no longer work, so you can’t remove the link the way Step 1: Prepare the LKC for decommissioning describes. Contact Confluent Support for help removing the stale link record.