Confluent Gateway for Confluent Platform

Confluent Gateway is an Apache Kafka®-protocol-aware proxy solution that provides platform teams centralized control over Kafka infrastructure. It streamlines complex operations including migrations, upgrades, and disaster recovery, without requiring code changes or application restarts from application developers.

Unlike generic proxy solutions that operate at the network layer, Confluent Gateway natively understands the Kafka protocol. It manages traffic by:

• Intercepting client traffic through stable, virtualized endpoints.

• Rewriting broker metadata in real time to dynamically route requests to the appropriate cluster.

• Enforcing policies directly at the proxy layer.

Routing changes and cluster switchovers don’t require client reconfiguration or application restarts. For the restart-related impact of a switchover, see Considerations.

Note

Confluent Gateway is available as an add-on to Confluent Platform. It works with Confluent Platform clusters that run on VMs or on Kubernetes, and you don’t need to change your existing brokers.

To use more than four routes, apply a Confluent Enterprise license for Confluent Platform that includes the Confluent Gateway add-on. You can evaluate Confluent Gateway with up to four routes and no license. For more information, see License modes.

Why use Confluent Gateway?

• Application-transparent migrations: Move workloads between clusters or to the cloud by updating a single routing rule, rather than coordinating with multiple application teams.

• Centralized security and governance: Enforce mTLS, authentication, and schema-based governance policies at Confluent Gateway, so you don’t configure each client individually.

• Stable endpoints for your clusters: Clients connect to Confluent Gateway endpoints instead of individual brokers, so you can change brokers, upgrade clusters, or fail over without changing client configurations.

• Low recovery time objective (RTO) during failures: Reduce recovery time during disasters by switching traffic to a standby cluster, without requiring application restarts or client reconfiguration.

• Simplified client configuration: Eliminate the need for clients to track cluster changes, broker lists, and security configurations by centralizing these concerns at the Confluent Gateway.

Confluent Gateway supports the following use cases, reflecting patterns common in enterprise-scale Kafka deployments:

• Cloud migration: Migrate on-premises clients to Confluent Cloud without client changes.

• Incremental topic migration (Early Access): Move topics between clusters in an incremental way while clients keep the same bootstrap address. For more information, see Migrate Topics Incrementally with Topic-Based Routing and Fencing.

• On-premises disaster recovery: Switch over from an unhealthy cluster to a healthy cluster without client changes.

• Secure external partner access: Grant trusted external partners direct access to a private Kafka cluster through a dedicated Confluent Gateway endpoint, enforcing authentication and access policies at the boundary without modifying the upstream cluster or exposing direct network access to it.

• Custom domains for Kafka listeners: Define organization-branded hostnames for Kafka endpoints, decoupling client-facing DNS naming from physical cluster addresses.

• Blue-green Kafka version upgrades: Validate a new Kafka version under production traffic before completing the cutover. Shift load incrementally and roll back immediately if issues arise, without involvement from application teams.

How Confluent Gateway routes and secures Kafka traffic

Confluent Gateway intercepts all Kafka traffic from client applications, securely routing Kafka protocol messages to the appropriate upstream Kafka clusters.

By rewriting broker addresses within metadata responses, Confluent Gateway exposes only virtualized endpoints to clients. This abstracts the underlying infrastructure, ensuring that any backend cluster changes remain transparent to your applications.

Core components

• Routes: The virtualized Confluent Gateway endpoints exposed to Kafka client applications to stream data.

  Confluent Gateway uses routes to create an abstraction layer that decouples client applications from backend Kafka brokers, allowing traffic to be dynamically redirected without client-side configuration changes.

• Streaming Domains: The logical representations of your upstream Kafka clusters within Confluent Gateway. They act as the backend targets that map your virtualized routes to actual physical infrastructure.

• Policies: Governance and client behavior rules that Confluent Gateway enforces on records flowing through a route, such as schema validation and encryption rules. You define policies once and Confluent Gateway applies them centrally to every client, without reconfiguring individual applications or brokers.

Request lifecycle

When a client application sends a request to a route, the Confluent Gateway performs the following steps:

1. Intercept: A Kafka client sends a request to a Confluent Gateway route.

2. Resolve: The Confluent Gateway identifies which Streaming Domain the route is mapped to and determines the upstream cluster to handle the request.

3. Protocol transformation: The Confluent Gateway performs any necessary authentication swapping, such as converting an OIDC token to SASL credentials required by the backend cluster.

4. Optional: Enforce governance: If you enable centralized governance, Confluent Gateway validates each record against Schema Registry based on your governance settings, then forwards or rejects it. On the fetch path, Confluent Gateway applies data contract rules, such as decryption, before returning records to the client.

5. Forward traffic: The Confluent Gateway forwards the request to the appropriate upstream Kafka broker. When the broker responds, the Confluent Gateway rewrites the broker addresses in the metadata response so that clients see only Confluent Gateway virtual endpoints, keeping cluster topology changes transparent to applications.

Protocol support

Confluent Gateway is compatible with Kafka protocol versions 3.x and 4.x. It provides transparent proxying for all standard Kafka client libraries without code changes.

Supported features

• Authentication and identity management: Authenticate clients using mTLS, SASL/PLAIN, SASL/SCRAM, OAuth, or OIDC. Confluent Gateway can pass the client identity directly to the upstream cluster or perform authentication swapping between the Confluent Gateway and the brokers. For more information, see Authentication and Authentication swapping.

• Secure credential storage: Retrieve broker and Schema Registry credentials from external secret stores, including HashiCorp Vault, AWS Secrets Manager, and Azure Key Vault. This eliminates the need to store plaintext secrets in the gateway.yaml configuration file. For more information, see Secret stores.

• Metrics, tracing, and troubleshooting: Monitor request volume, latency, and validation outcomes by exporting Prometheus-format metrics through the admin endpoint. You can also use the fencing filter to block incoming requests and return standard BROKER_NOT_AVAILABLE error codes to clients. For more information, see Administration and metrics and Fencing filter.

• Centralized governance enforcement: Validate schemas and enforce data contracts centrally against Confluent Schema Registry so that only compliant records reach your Kafka clusters. Confluent Gateway supports schema ID enforcement, deep schema validation, and data contract rules, including field-level and full-payload encryption, for records that use Apache Avro®, JSON Schema, or Protobuf. For more information, see Centralized governance.

Current limitations

• No legacy protocol support: Supports Kafka protocol versions 3.x and 4.x only. Kafka server and client versions 2.x or earlier are not supported.

• Centralized deployment only: Must be deployed as a centralized or ingress proxy tier. Distributed sidecar deployment models are not supported.

• No Schema Registry proxying: Confluent Gateway connects to Schema Registry as a client for governance validation but does not proxy client-to-Schema Registry HTTP/REST traffic.

Get started

To get started with Confluent Gateway, deploy Confluent Gateway on Docker or CFK. Review system requirements, supported Kafka and client versions, and step-by-step instructions for both deployment models in Configure and Deploy Gateway.