# Deploy a Multi-Tenant Physical Kafka Cluster
Learn how to deploy a Physical Kafka Cluster (PKC) with
Multi-Tenancy enabled using Confluent for Kubernetes (CFK). A platform operator
completes this setup once per PKC before onboarding any tenants.
To move the topics of an existing cluster
to Logical Kafka Clusters (LKCs), see the Migrate Existing Clusters to
Multi-Tenant LKCs page.
## Prerequisites
Before you begin, ensure you have the following:
- A Kubernetes cluster with Confluent for Kubernetes (CFK) 3.4 or later installed.
For installation instructions, see [Deploy Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-deploy-cfk.html).
- Confluent Platform 8.4 or later.
- The ability to create wildcard DNS records for the PKC’s domain. For more
information, see [Configure TLS Certificates and DNS for Multi-Tenant PKCs](tls-options.md#multi-tenancy-tls-options).
- Familiarity with the Multi-Tenancy model. For an introduction, see
[Multi-Tenancy Overview](overview.md#multi-tenancy-overview).
- A Confluent Private Cloud license. Without a license, you can use up to three LKCs for 30
days. For more information, see
[Manage Confluent Platform Licenses](../../installation/license.md#cp-license-overview).
## Requirements for a multi-tenant PKC
A multi-tenant PKC has the following requirements:
- **Brokers:** At least three. Multi-Tenancy stores LKC metadata in an
internal topic that requires three replicas.
- **Authorization:** role-based access control (RBAC) authorization with Metadata Service (MDS)
on the `Kafka` and `KRaftController` CRs.
- **Encryption:** TLS on the internal and external listeners.
- **Routing:** Server Name Indication (SNI) routing and wildcard DNS records
for the LKC hostnames.
- **Deployment topology:** A single-region PKC. Multi-Tenancy isn’t supported
on multi-region clusters (MRCs) in this release.
For the `Kafka` CR settings that enforce these requirements, see
[Step 3](#multi-tenancy-deploy-step-3).
## Step 1: Plan capacity
Before you deploy, establish a broker-capacity baseline from each
broker’s JVM heap size. CFK sets the heap to half of the pod memory, up
to 32 GiB. Use the baseline to size the broker memory in
[Step 3: Deploy the PKC with Multi-Tenancy enabled](#multi-tenancy-deploy-step-3) and as the resource ceiling when
you calculate tenant quota allocations. The values are per broker:
| Capacity dimension | Calculation formula |
|----------------------------|--------------------------|
| Producer byte rate | `heapGiB × 6.25 MiB/s` |
| Consumer byte rate | `producer byte rate × 3` |
| Max connections | `heapGiB × 4096` |
| Max connections per tenant | `heapGiB × 1024` |
For example, a broker configured with an 8 GiB JVM heap supports a
baseline capacity of 50 MiB/s producer throughput, 150 MiB/s consumer
throughput, 32,768 total connections, and 8,192 connections per tenant.
CFK also limits how many new connections per second each LKC and each
tenant listener can open on a broker. These limits don’t scale linearly
with heap size, so CFK sets them by heap size range:
| Broker JVM heap | Per LKC (new connections per second) | Per tenant listener (new connections per second) |
|-------------------------------|----------------------------------------|----------------------------------------------------|
| 4 GiB or less | 40 | 84 |
| More than 4 GiB, up to 16 GiB | 50 | 250 |
| More than 16 GiB | 63 | 750 |
## Step 2: Deploy KRaft controllers
Deploy the KRaft controllers first. The `Kafka` custom resource (CR) that
provisions your PKC references the `KRaftController` CR, and CFK waits for
KRaft to be up before it starts Kafka.
Configure the `KRaftController` CR with RBAC authorization
with `spec.authorization.type: rbac` and an `mdsKafkaCluster`
dependency. Without RBAC, the controllers run as single-tenant and
can’t authorize topic creation in LKCs.
1. Create a manifest file named `kraftcontroller.yaml` with the following
configuration:
```yaml
apiVersion: platform.confluent.io/v1beta1
kind: KRaftController
metadata:
name:
namespace:
spec:
replicas: 3
image:
application: confluentinc/cp-server:
init: confluentinc/confluent-init-container:
dataVolumeCapacity: 10G
configOverrides:
server:
- default.replication.factor=3
- min.insync.replicas=2
authorization:
type: rbac
superUsers:
- User:
- User:
listeners:
controller:
tls:
enabled: true
authentication:
type: plain
jaasConfig:
secretRef:
tls:
autoGeneratedCerts: true
dependencies:
mdsKafkaCluster:
bootstrapEndpoint: ..svc.cluster.local:9071
authentication:
type: plain
jaasConfig:
secretRef:
tls:
enabled: true
```
Replace the placeholders with your values:
- ``: The name of the `KRaftController` CR.
- ``: Your Kubernetes namespace.
- ``: The Confluent Platform version to run, such as `8.4.0`.
- ``: The `confluent-init-container` version that
matches your CFK version.
- `` and ``: The principals that the
controllers treat as super users.
- `` and ``:
The names of the secrets that hold the credentials for the controller
listener and for the connection to the Kafka cluster that hosts Metadata Service (MDS).
- ``: The name that you give the PKC in
[the next step](#multi-tenancy-deploy-step-3). The controllers connect
to the PKC’s internal listener on port `9071`.
The controllers assign replicas to new topics, so set
`default.replication.factor` and `min.insync.replicas` under
`configOverrides.server` on the `KRaftController` CR. Without them,
topics that clients create without a replication factor get a replication
factor of `1`.
This example uses `plain` authentication and auto-generated certificates.
Adjust your authentication and TLS settings to match your Kafka
configuration. For full configuration options, see
[Configure and Manage KRaft for Confluent Platform
Using Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-configure-kraft.html).
2. Apply the configuration:
```bash
kubectl apply -f kraftcontroller.yaml
```
## Step 3: Deploy the PKC with Multi-Tenancy enabled
Enable Multi-Tenancy on the `Kafka` CR that provisions your PKC.
CFK rejects a `Kafka` CR with `spec.multiTenant.enabled` set to
`true` unless the CR also configures internal and external listener
TLS, Metadata Service (MDS), role-based access control (RBAC) authorization, and broker pod memory.
1. Configure these prerequisites on the same CR, alongside `multiTenant`:
- Set `spec.listeners.internal.tls.enabled` and
`spec.listeners.external.tls.enabled` to `true`.
- Configure `spec.services.mds`. For instructions, see [Configure
Authentication for Kafka, KRaft, and MDS in Confluent Platform Using
Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-authenticate-kafka.html).
- Set `spec.authorization.type` to `rbac`.
- Set `spec.podTemplate.resources` to a memory request or limit.
Multi-Tenancy derives its capacity-planning baseline from this value.
2. Ensure these `Kafka` CR settings are also set. CFK accepts a `Kafka`
CR that omits them, but then reports an error and doesn’t deploy the
brokers:
- Configure `spec.dependencies.kafkaRest.authentication`. Without it,
the CR reports `.spec.dependencies.kafkaRest.authentication must be
configured`.
- Configure `spec.tls`. For example, use `autoGeneratedCerts: true`.
Without it, a CR that enables TLS on a listener reports `missing
.spec.tls. in spec`.
3. Create a manifest file named `kafka.yaml` with the following configuration:
```yaml
apiVersion: platform.confluent.io/v1beta1
kind: Kafka
metadata:
name:
namespace:
spec:
replicas: 3
image:
application: confluentinc/cp-server:
init: confluentinc/confluent-init-container:
dataVolumeCapacity: 100Gi
authorization:
type: rbac
tls:
autoGeneratedCerts: true
services:
mds:
tokenKeyPair:
secretRef:
provider:
ldap:
address:
authentication:
type: simple
simple:
secretRef:
configurations:
userNameAttribute:
userObjectClass:
userSearchBase:
groupSearchBase:
authentication:
type: bearer
listeners:
internal:
authentication:
type: plain
jaasConfig:
secretRef:
tls:
enabled: true
external:
authentication:
type: plain
jaasConfig:
secretRef:
externalAccess:
type: loadBalancer
loadBalancer:
domain:
tls:
enabled: true
podTemplate:
resources:
requests:
memory:
multiTenant:
enabled: true
dependencies:
kRaftController:
clusterRef:
name:
controllerListener:
tls:
enabled: true
authentication:
type: plain
jaasConfig:
secretRef:
kafkaRest:
authentication:
type: bearer
bearer:
secretRef:
```
Replace the placeholders with your values:
- ``: The name of your PKC.
- ``: Your Kubernetes namespace.
- ``: The Confluent Platform version to run, such as `8.4.0`.
- ``: The `confluent-init-container` version that
matches your CFK version.
- ``: The memory request for each broker pod,
such as `8Gi`. The broker’s JVM heap is smaller than this value.
Use the JVM heap size, not the pod memory request, in the
capacity-planning formulas in
[Step 1: Plan capacity](#multi-tenancy-deploy-step-1).
- ``: The name of the `KRaftController` CR you
deployed in [Step 2: Deploy KRaft controllers](#multi-tenancy-deploy-step-2).
- ``: The name of the secret that holds the
credentials for the internal listener. The controllers connect to this
listener through `mdsKafkaCluster`, so the secret must include the
user that `` in
[Step 2: Deploy KRaft controllers](#multi-tenancy-deploy-step-2) uses.
- ``: The name of the secret that holds the
credentials for the external listener. Clients outside your Kubernetes
cluster use these credentials to connect to an LKC’s bootstrap URL.
- ``: The name of the secret that holds
the credentials for the controller listener. Use the same secret you
set in [Step 2: Deploy KRaft controllers](#multi-tenancy-deploy-step-2).
- ``: The name of the secret that holds the MDS
token key pair.
- ``: The name of the secret that holds
the bearer credentials for the Kafka REST API.
- ``, ``,
``, ``,
``, and ``: The connection
details and search settings for your LDAP directory. This example
uses LDAP. To use a different identity provider, configure
`provider` for that provider instead.
The search settings must match your directory’s layout. If they don’t,
Metadata Service (MDS) can’t search for groups or can’t find users. In that case,
Metadata Service (MDS) doesn’t start, or it rejects logins, and the role
bindings in [Step 7: Configure the RBAC foundation and role hierarchy](#multi-tenancy-deploy-step-7) fail with
`connection refused` or `401` errors. For
the LDAP settings and their defaults, see
[Configure LDAP Group-Based Authorization](../../security/authorization/ldap/configure.md#configure-ldap-authorization).
- ``: The domain that LKC bootstrap hostnames use,
such as `mt.example.com`.
`clusterRef` only tells the brokers which controllers to use. The
`controllerListener` section tells them how to connect to the controller
listener, so it must match the `listeners.controller` settings on the
`KRaftController` CR. Without it, CFK configures the brokers to connect
to the controllers over plaintext with no authentication, and the brokers
can’t register with the controllers that you configured for TLS and
`plain` authentication in [Step 2: Deploy KRaft controllers](#multi-tenancy-deploy-step-2).
Without authentication on the external listener, clients outside the
cluster can’t connect to an LKC’s bootstrap URL. The connection fails with
`Unexpected handshake request with client mechanism PLAIN, enabled
mechanisms are []`.
By default, CFK doesn’t create Services for LKC hostnames. Configure
the DNS rewrite rules described in [Configure TLS Certificates and DNS for Multi-Tenant PKCs](tls-options.md#multi-tenancy-tls-options). Set
`spec.multiTenant.createTenantNameServices` to `true` only as a
fallback when you can’t edit your cluster’s DNS. CFK then creates a
Service for every broker of every LKC, which uses up the cluster’s Service
IP range as you add LKCs and brokers. For an example of how quickly the
number of Services grows, see [Configure TLS Certificates and DNS for Multi-Tenant PKCs](tls-options.md#multi-tenancy-tls-options) and
[Limitations](overview.md#multi-tenancy-limitations).
The preceding example is a starting point. Adjust the authentication,
TLS certificate, and LDAP settings to match your environment. For full
listener,
authentication, and MDS configuration options, see [Configure
Network Encryption for Confluent Platform Using Confluent for
Kubernetes](https://docs.confluent.io/operator/current/co-network-encryption.html)
and [Manage RBAC for Confluent Platform Using Confluent for
Kubernetes](https://docs.confluent.io/operator/current/co-manage-rbac.html).
4. Apply the configuration:
```bash
kubectl apply -f kafka.yaml
```
## Step 4: Configure PKC-level settings
Configure the following PKC-wide settings on the `Kafka` CR before
onboarding tenants. These settings are not specific to Multi-Tenancy.
Configure them by referring to the documentation that each setting links to:
- **Identity provider**: All LKCs on the same
PKC share a single identity provider, such as LDAP or OpenID Connect (OIDC). Plan for
this shared identity-provider constraint before configuring
authentication. For guidance on securing your identity provider, see
[Plan Security for Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-plan-security.html).
- **TLS certificates**: Ensure every broker presents a TLS certificate
valid for all hosted LKC hostnames. You provision these certificates
as part of the TLS approach you choose. For more information, see
[Manage Certificates](https://docs.confluent.io/operator/current/co-manage-certificates.html).
- **Storage capacity**: Choose storage classes and size
`spec.dataVolumeCapacity` to accommodate the total storage
footprint of all planned tenants. For more information, see [Configure
Storage for Confluent Components Using Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-storage.html).
- **Broker defaults**: Set cluster-wide broker configuration under
`spec.configOverrides.server` for the PKC.
## Step 5: Configure a TLS approach
Clients reach each LKC through its own bootstrap URL, which SNI routes at the
TLS layer. Brokers advertise LKC hostnames with a
broker label, such as `lkc--kafka.b0.`. Every broker needs a
certificate that’s valid for the LKC hostnames it serves, and the required
Subject Alternative Names (SANs) depend on the TLS approach that you choose.
Before you onboard your first tenant, choose your TLS approach and complete its
certificate setup. For guidance on choosing an approach and the SANs that each
approach requires, see [Configure TLS Certificates and DNS for Multi-Tenant PKCs](tls-options.md#multi-tenancy-tls-options).
The `autoGeneratedCerts: true` setting in the
[Step 3: Deploy the PKC with Multi-Tenancy enabled](#multi-tenancy-deploy-step-3) example generates a certificate that
covers only in-cluster names. A client outside the cluster then fails the TLS
handshake with `No subject alternative DNS name matching`. To add the
wildcard names to the generated certificate, set the
`platform.confluent.io/managed-cert-add-sans` annotation on the `Kafka` CR
when you create it. For example,
`"*.,*.b0.,*.b1.,*.b2."`.
CFK reads this annotation from the `Kafka` CR only if `managedCerts` is
enabled in the CFK Helm values. Set `managedCerts.enabled` to `true` and
set `managedCerts.caCertificate.secretRef` to the secret that holds your
certificate authority (CA) certificate and key pair. With the default Helm
values, CFK doesn’t read the annotation. For more
details, see [Manage auto-generated certificates](https://docs.confluent.io/operator/current/co-manage-certificates.html#co-manage-auto-generated-certificates).
## Step 6: Verify Confluent Control Center’s auto-detection
If you deploy Confluent Control Center, it requires version 2.7 or later. Set
`spec.authorization.type` to `rbac`. For deployment instructions, see
[Install Confluent Platform Using Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-deploy-cp.html).
CFK automatically configures Confluent Control Center for Multi-Tenancy based on the `Kafka`
CR that the `ControlCenter` CR references. The `ControlCenter` CR must
reference the PKC through `spec.dependencies.kafka.discovery`. An explicit
`bootstrapEndpoint` doesn’t enable Multi-Tenancy.
Verify that Confluent Control Center detects the PKC as multi-tenant:
```bash
kubectl get controlcenter -n -o jsonpath='{.status.multiTenant}'
```
A multi-tenant PKC returns `{"enabled":true}`. If the command returns
an empty response, confirm the `ControlCenter` CR’s
Kafka dependency points to the PKC you
configured in [Step 3: Deploy the PKC with Multi-Tenancy enabled](#multi-tenancy-deploy-step-3), and that
its image is Confluent Control Center 2.7 or later. CFK doesn’t enable Multi-Tenancy on
earlier images. For more information, see
[Multi-tenancy settings](/control-center/current/installation/configuration.html#multi-tenancy-settings).
## Step 7: Configure the RBAC foundation and role hierarchy
Multi-Tenancy divides RBAC access across three roles:
- **Platform operator**, scoped to the whole PKC. Holds the
`OrganizationAdmin` role.
- `TenantAdmin`, scoped to one or more LKCs. Holds the
`TenantAdmin` role, scoped to those LKCs.
- **Developer**, scoped to individual resources within an LKC. Holds a
standard role, such as `DeveloperRead` or `DeveloperWrite`,
that the `TenantAdmin` grants.
For the full breakdown of responsibilities, see
[Roles and responsibilities](overview.md#multi-tenancy-roles-and-responsibilities).
This step sets up the PKC-wide `OrganizationAdmin` foundation. Grant
the `TenantAdmin` role to individual users during tenant onboarding.
The `TenantAdmin` then grants developer-level roles within their own
LKCs.
1. A `ConfluentRolebinding` CR uses the `KafkaRestClass` CR that
`spec.kafkaRestClassRef` names. If you don’t set
`spec.kafkaRestClassRef`, it uses the `KafkaRestClass` CR named
`default`. Create a `KafkaRestClass` CR before you apply the role
binding:
```yaml
apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestClass
metadata:
name: default
namespace:
spec:
kafkaRest:
authentication:
type: bearer
bearer:
secretRef:
```
Replace `` with the name of the secret that holds
the bearer credentials for the Kafka REST API.
2. Create a manifest file named `rolebinding.yaml` to assign the
`OrganizationAdmin` role:
```yaml
apiVersion: platform.confluent.io/v1beta1
kind: ConfluentRolebinding
metadata:
name:
namespace:
spec:
principal:
type: user
name:
role: OrganizationAdmin
clustersScopeByIds:
kafkaClusterId:
```
Replace the placeholders with your values:
- ``: A name for the
`ConfluentRolebinding` CR.
- ``: The platform operator’s identity in
your identity provider.
- ``: The Kafka cluster ID of the PKC. For
instructions on finding cluster IDs, see [Cluster Identifiers in Confluent Platform](../../security/authorization/rbac/rbac-get-cluster-ids.md#rbac-get-cluster-ids).
3. Apply the configuration:
```bash
kubectl apply -f rolebinding.yaml
```
The identity that CFK uses to create LKCs, whether through a
`KafkaRestClass` or through a cluster reference with inline authentication
details, also needs the `OrganizationAdmin` role on the PKC. Grant the role
the same way as in the preceding example.
For general RBAC CR mechanics, see [Manage
RBAC for Confluent Platform Using Confluent for Kubernetes](https://docs.confluent.io/operator/current/co-manage-rbac.html).
## Verify your deployment
Confirm that all PKC custom resources report a healthy phase before
onboarding tenants:
```bash
kubectl get kraftcontroller -n
kubectl get kafka -n
kubectl get controlcenter -n
```
Verify that each command displays `RUNNING` in the `STATUS` column.
Confluent Control Center is also required to visualize a multi-tenant PKC’s brokers, LKCs,
and per-tenant quota usage, rather than only its CRs’ reconciliation
state. A principal with a PKC-scoped role binding, such as one for
`OrganizationAdmin`, can use the following pages:
- [Home page](/control-center/current/overview.html#multi-tenancy): A PKC-wide summary
of resources, including LKCs, topics, partitions, Connect clusters, and
ksqlDB clusters. On a newly deployed PKC with no tenants yet, the summary
shows zero LKCs.
- [Quota Overview page](/control-center/current/quota.html): After you onboard tenants,
quota usage for every LKC, including 95th-percentile (p95) production and
consumption rates, active connection counts, partition counts, and quota
configuration.
## Related content
- [Multi-Tenancy Overview](overview.md#multi-tenancy-overview)
- [Configure TLS Certificates and DNS for Multi-Tenant PKCs](tls-options.md#multi-tenancy-tls-options)
- [Deploy Ecosystem Components on a Multi-Tenant PKC](ecosystem-components.md#multi-tenancy-ecosystem-components)
- [Multi-tenancy in Control Center](/control-center/current/overview.html#multi-tenancy)
- [View Tenant Quotas in Control Center](/control-center/current/quota.html)