Onboard an LKC to a Multi-Tenant Physical Kafka Cluster

Provision a Logical Kafka Cluster (LKC) on an existing multi-tenant Physical Kafka Cluster (PKC) in Confluent Private Cloud. Platform operators perform this procedure once per LKC.

Prerequisites

Before you begin, ensure you have the following:

• A multi-tenant PKC. For more information, see the Deploy a Multi-Tenant Physical Kafka Cluster page.

• A user principal or service account configured in your shared identity provider (IdP). To secure your IdP integration, see Plan Security for Confluent for Kubernetes.

• A KafkaRestClass in the same namespace as the LKC, whose principal holds the OrganizationAdmin role on the PKC. A TenantAdmin can’t create an LKC. Name the KafkaRestClass default, or reference it with spec.kafkaRestClassRef in both the LogicalKafkaCluster manifest and the role binding manifest in Step 7. Alternatively, set spec.kafkaRest on the LogicalKafkaCluster to provide the connection details inline. You can’t set both spec.kafkaRestClassRef and spec.kafkaRest.

Step 1: Assess PKC capacity

Before provisioning an LKC, estimate the LKC’s workload requirements:

• Producer and consumer byte rates

• Total client connection count

• Total topic partition count

Compare these requirements against the PKC capacity baseline established during initial cluster deployment. For more information, see the Deploy a Multi-Tenant Physical Kafka Cluster page. If host capacity is insufficient, scale the underlying PKC before proceeding.

Important

A multi-tenant cluster stores tenant metadata in the cluster registry and supports up to 25 LKCs by default. To support more than 25 LKCs, enable compression by setting confluent.metadata.server.cluster.registry.compression.enable=true in the Metadata Service (MDS) configuration before reaching the limit. You cannot disable compression after you enable it.

Step 2: Configure resource quotas and limits

Set the following quotas and limits on the LogicalKafkaCluster custom resource (CR). Size each value as a share of the PKC’s capacity-planning baseline from Step 1, so that the combined values of all LKCs stay within what the PKC can deliver.

• spec.quotas.ingressByteRate and spec.quotas.egressByteRate: The LKC’s producer and consumer throughput limits, in bytes per second. The minimum value is 1. If you omit a value, the LKC has no limit of its own in that direction. Only the PKC’s broker-wide limits apply, and all LKCs on the PKC share them.

• spec.quotas.brokerRequestThreads: The request-processing thread time that the LKC can use on each broker. Specify the value as a thread count, not a percentage. For example, 1.0 is one thread fully busy, and 2.5 allows 2.5 threads fully busy. The value must be greater than 0. If you omit the value, it defaults to 2.5 threads.

• spec.limits.maxPartitions: The maximum partition count across the LKC’s topics. If you omit the value, the broker’s default limit, 4096 partitions, applies.

• spec.limits.maxMessageSizeBytes: The maximum max.message.bytes value a client can set on a topic in the LKC. If you omit the value, the broker’s default maximum applies. A topic that a client creates without max.message.bytes uses the broker’s message.max.bytes value, whatever this limit is. To keep every topic in the LKC within the limit, set the limit at or above message.max.bytes.

The LKC also has a limit on the number of client connections. Each broker limits an LKC to heapGiB × 1024 connections, where heapGiB is the broker’s JVM heap size in GiB. You can’t change this limit for an individual LKC. For the broker-level capacity values, see the Deploy a Multi-Tenant Physical Kafka Cluster page.

Caution

When you update an LKC’s manifest, include every subfield of spec.quotas and spec.limits that you want to keep, not only the ones you’re changing. CFK clears any subfield you omit.

Step 3: Define a unique LKC name

Choose a unique name for the LKC. If you leave spec.name unset, CFK uses the LKC CR’s object name.

The name must be unique on the PKC and one to 64 characters long. It can contain letters, numbers, periods, underscores, and hyphens, and it must start with a letter or number. Names of the form lkc-*-external are reserved.

Step 4: Create the LogicalKafkaCluster manifest

1. Create a manifest file named lkc.yaml with the following configuration:

   apiVersion: platform.confluent.io/v1beta1
   kind: LogicalKafkaCluster
   metadata:
     name: <lkc-name>
     namespace: <lkc-namespace>
   spec:
     name: <lkc-display-name>
     kafkaClusterRef:
       kind: Kafka
       name: <pkc-name>
       namespace: <pkc-namespace>
     quotas:
       ingressByteRate: <ingress-byte-rate>
       egressByteRate: <egress-byte-rate>
       brokerRequestThreads: <broker-request-threads>
     limits:
       maxPartitions: <max-partitions>
       maxMessageSizeBytes: <max-message-size-bytes>
   

   Replace the placeholders with your values:

   • <lkc-name>: A Kubernetes object name for the LKC CR. This name is separate from the LKC name you chose in Step 3, although you can use the same value.

   • <lkc-namespace>: The Kubernetes namespace for the LKC CR. The KafkaRestClass that CFK uses to create the LKC must be in this namespace.

   • <lkc-display-name>: The LKC name you chose in Step 3.

   • <pkc-name>: The name of the PKC’s Kafka CR you configured by following the Deploy a Multi-Tenant Physical Kafka Cluster page.

   • <pkc-namespace>: The namespace of the PKC’s Kafka CR. This can differ from the LKC’s namespace.

   • <ingress-byte-rate>, <egress-byte-rate>, <broker-request-threads>, <max-partitions>, <max-message-size-bytes>: The values you sized in Step 2.

   Caution

   Verify spec.name and spec.kafkaClusterRef before you apply the manifest. You can’t change either field after you create the LKC. To change either value, decommission the existing LKC and create a new one.

2. Apply the configuration:

   kubectl apply -f lkc.yaml
   

Step 5: Verify LKC provisioning and operational isolation

1. Confirm that the STATE column reports CREATED and the PHASE column reports Running. Note the LKC-ID value:

   kubectl get logicalkafkacluster <lkc-name> -n <your-namespace>
   

2. Retrieve the LKC’s dedicated bootstrap URL from its status.bootstrapEndpoint field:

   kubectl get logicalkafkacluster <lkc-name> -n <your-namespace> \
     -o jsonpath='{.status.bootstrapEndpoint}'
   

   Make sure that your DNS resolves this URL and that the broker certificate includes the LKC hostnames. For the DNS records and certificate names, see Configure TLS Certificates and DNS for Multi-Tenant PKCs.

3. Create a client configuration file named client.properties for a principal that holds the OrganizationAdmin role on the PKC, such as the identity you used to deploy the PKC:

   security.protocol=SASL_SSL
   sasl.mechanism=PLAIN
   sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<username>" password="<password>";
   ssl.truststore.type=PEM
   ssl.truststore.location=<ca-certificate-path>
   

   Replace the placeholders with your values:

   • <username> and <password>: The principal’s credentials for the PKC’s external listener.

   • <ca-certificate-path>: The path to the CA certificate that signed the PKC’s broker certificate.

   This example uses SASL/PLAIN. If your PKC’s external listener uses a different mechanism, configure the client for that mechanism. For more information, see Authentication in Confluent Platform.

4. Connect to the bootstrap URL, create a test topic, and confirm that it appears in the LKC’s own topic list. For the kafka-topics command syntax, see Use the command-line tools:

   kafka-topics --bootstrap-server <lkc-bootstrap-url> \
     --command-config client.properties \
     --create --topic test-topic --partitions 3 --replication-factor 3
   
   kafka-topics --bootstrap-server <lkc-bootstrap-url> \
     --command-config client.properties --list
   

   Replace <lkc-bootstrap-url> with the status.bootstrapEndpoint value from the previous step.

5. Using the same client, produce and consume test messages on that topic. For kafka-console-producer and kafka-console-consumer usage, see the same command-line tools page.

6. If this isn’t the first LKC on the PKC, connect to another LKC’s bootstrap URL. Confirm that the client can’t list or access the test topic you created in this LKC.

Step 6: Identify the TenantAdmin principal

Choose an existing principal in your shared IdP to receive the TenantAdmin role for this LKC.

Step 7: Assign the TenantAdmin role

1. Create a manifest file named rolebinding.yaml to grant the TenantAdmin role, scoped to this LKC:

   apiVersion: platform.confluent.io/v1beta1
   kind: ConfluentRolebinding
   metadata:
     name: <tenant-admin-binding-name>
     namespace: <your-namespace>
   spec:
     principal:
       type: user
       name: <tenant-admin-identity>
     role: TenantAdmin
     kafkaClusterRef:
       kind: LogicalKafkaCluster
       name: <lkc-name>
   

   Replace the placeholders with your values:

   • <tenant-admin-binding-name>: A name for the ConfluentRolebinding CR.

   • <tenant-admin-identity>: The TenantAdmin’s identity from Step 6.

   • <lkc-name>: The name of the LKC you created in Step 4. The LogicalKafkaCluster CR must be in the same namespace as the role binding.

2. Apply the configuration:

   kubectl apply -f rolebinding.yaml
   

For more information about managing role-based access control (RBAC) with CRs, see Manage RBAC for Confluent Platform Using Confluent for Kubernetes.

Step 8: Provide connection details to the TenantAdmin

Provide the following to the TenantAdmin so that they can start onboarding their own developers:

• The LKC’s dedicated bootstrap URL, from Step 5.

• The authentication mechanisms configured on the PKC’s shared IdP.

• The Confluent Control Center URL, so that the TenantAdmin can monitor their own LKC’s resources and usage statistics.

• The quota and limit boundaries you set in Step 2.

• An escalation path for requesting a quota increase or reporting an issue.

Related content

• Deploy a Multi-Tenant Physical Kafka Cluster

• Multi-Tenancy Overview

• Decommission an LKC