Multi-Tenancy Overview
Multi-Tenancy enables platform teams to run multiple isolated Logical Kafka Clusters (LKCs) on a single shared Physical Kafka Cluster (PKC).
Provisioning every application team its own fully dedicated PKC means installing, upgrading, and patching a full set of brokers, KRaft controllers, and auxiliary components for every application team. At scale, this creates cluster sprawl, low hardware use, and slow onboarding.
Multi-Tenancy creates logical cluster boundaries without provisioning new physical infrastructure, consolidating workloads onto shared physical brokers while maintaining strict isolation guarantees. One shared PKC can host many LKCs, each behaving like its own isolated Kafka cluster from the application team’s point of view.
Multi-Tenancy is available through Confluent for Kubernetes (CFK), which deploys and manages the PKC and every LKC provisioned through it.
Key concepts
The following terms describe the core building blocks of the Multi-Tenancy model.
Term |
Definition |
|---|---|
Physical Kafka Cluster (PKC) |
The physical infrastructure including brokers, KRaft
controllers, and storage, represented by a single |
Logical Kafka Cluster (LKC) |
A logical cluster provisioned on a shared PKC that functions
as a fully isolated, independent Kafka cluster. Some APIs, roles,
and metrics refer to an LKC as a tenant. For example, the
MDS Tenant Management API, the |
Resource quota |
Per-LKC bytes-in, bytes-out, request-handling, partition, and connection limits that the broker enforces directly. Confluent Control Center shows each LKC’s live production rate, consumption rate, connection count, and partition count against these limits. For more information, see How Multi-Tenancy works. |
How Multi-Tenancy works
Multi-Tenancy is built on broker-native architecture where the Kafka broker engine itself enforces LKC boundaries within the shared PKC. This enforcement spans across three foundational layers:
Experience isolation: The broker scopes topics and consumer groups to each LKC, ensuring clients interact only with the resources in their own LKC as if operating an independent, fully owned cluster.
Authorization isolation: The broker enforces LKC-scoped RBAC boundaries directly at request time to prevent data leakage across LKCs.
Resource quota: The broker limits each LKC’s bytes-in and bytes-out rates, request-handling capacity, and partition and connection counts, so one LKC’s workload can’t degrade another’s.
Roles and responsibilities
Multi-Tenancy divides responsibility for a shared PKC across three personas, each scoped to a different layer of the model:
Role |
Scope |
Responsibilities |
|---|---|---|
Platform operator |
PKC-wide |
Provisions and scales physical cluster infrastructure, sets up
DNS and TLS certificates for LKC hostnames, provisions LKCs,
defines capacity-planning baselines, delegates
|
|
LKC-scoped |
Self-manages topics, consumer groups, and RBAC-based access
policies for existing identities within their assigned LKC.
Holds the RBAC Creating new identities or service-account credentials requires the platform operator or the shared identity provider (IdP) administrator. |
Developer |
Resource-scoped |
Connects standard Kafka applications to an LKC bootstrap URL to
produce and consume messages using access permissions granted by
the From inside the LKC, only that LKC’s own topics and resources are visible. |
Limitations
The following limitations apply to Multi-Tenancy:
All LKCs on a PKC share the PKC’s identity provider configuration. You can’t configure a dedicated identity provider (IdP) for an individual LKC.
Of the listeners you configure on the PKC, only the external listener (
spec.listeners.external) supports LKC-based routing. You can’t make internal, replication, or custom (spec.listeners.custom) listeners LKC-aware. Custom listeners on a multi-tenant PKC work as standard listeners that aren’t LKC-aware.LKC-based routing requires the
loadBalancerorstaticForHostBasedRoutingexternal access type. WithnodePortorstaticForPortBasedRouting, brokers don’t advertise LKC hostnames, so LKC clients can’t reach the brokers.CFK registers an LKC’s components (such as Kafka Connect or ksqlDB) in the Metadata Service (MDS) Cluster Registry only when the component references the LKC through
discoverywithkindset toLogicalKafkaCluster. A component that usesbootstrapEndpoint, or that omitskind, can connect to the LKC but isn’t registered. To configurediscovery, see Component deployment model.A multi-tenant cluster stores tenant metadata in the cluster registry and supports up to 25 LKCs by default. To support more than 25 LKCs, enable compression by setting
confluent.metadata.server.cluster.registry.compression.enable=truein the Metadata Service (MDS) configuration before reaching the limit. You cannot disable compression after you enable it.On clusters where you can’t edit DNS, such as Google Kubernetes Engine (GKE) clusters that use Cloud DNS as the cluster DNS provider, you must set
spec.multiTenant.createTenantNameServicestotrue. CFK then creates a Service for every LKC broker, which can exhaust the cluster’s Service IP range. For more details, see Configure TLS Certificates and DNS for Multi-Tenant PKCs.Multi-region clusters (MRCs) do not currently support multi-tenancy. Support is planned for a future release.
Tip
To try Multi-Tenancy hands-on, use the Quick Start for Multi-Tenancy in Confluent Private Cloud to deploy a multi-tenant PKC on a local Kubernetes cluster, create two LKCs, and verify tenant isolation.