Configure and Deploy Confluent Gateway using Confluent for Kubernetes

Configure and deploy Confluent Gateway using Confluent for Kubernetes (CFK). Use this deployment path to run Confluent Gateway on-premises, in virtual private clouds (VPCs), and across hybrid environments.

Before deploying Confluent Gateway with CFK, ensure you meet the following prerequisites in addition to the general Confluent Gateway requirements:

A Kubernetes environment with CFK and Confluent Platform installed. For details on installing CFK, see Deploy Confluent for Kubernetes. Use a version of CFK that supports your Confluent Platform and Confluent Gateway requirements. For the CFK, Confluent Platform, and Confluent Private Cloud Gateway version compatibility information, see Supported environments and prerequisites.
A valid CFK license. If you already have a CFK license for other Confluent Platform components such as Apache Kafka® or Schema Registry, you can reuse it for Confluent Gateway without purchasing an additional license.

For general requirements, license modes, and supported component compatibility, see Confluent Gateway Requirements and Considerations.

The high-level steps to deploy Confluent Gateway using CFK are:

Configure Gateway: Configure Confluent Gateway by defining the streaming domains that connect to your Kafka clusters, the routes that clients use to reach them, external access for clients outside the Kubernetes cluster, and the administration and metrics settings.
Deploy Gateway: Deploy the Confluent Gateway service using the configuration settings from the previous step.
Reconfigure clients: Point client applications to the Confluent Gateway route endpoint instead of connecting to brokers directly, by setting bootstrap.servers to the route endpoint you configured.
Apply security configuration: Configure security for Confluent Gateway, including secret stores, TLS/SSL, passwords, and authentication swapping.

Note

The Kubernetes commands in this guide assume you have set a default namespace, so they omit the --namespace flag for brevity. If you have not set a default namespace, add --namespace <your-namespace> to each command.

Configure Confluent Gateway

To configure Confluent Gateway, create a YAML file that defines a Confluent Gateway custom resource (CR), for example gateway.yaml, and add the configuration settings as described in the following sections. In the Deploy Confluent Gateway step, you apply this file to your Kubernetes cluster with kubectl apply.

The top-level layout for the Confluent Gateway CR is:

kind: Gateway
metadata:
  name:                   --- [1]
  namespace:              --- [2]
spec:
  image:
    application:          --- [3]
    init:                 --- [4]
  streamingDomains:       --- [5]
  secretStores:           --- [6]
  routes:                 --- [7]
  admin:                  --- [8]
  externalAccess:         --- [9]

[1] The Confluent Gateway name.
[2] The Confluent Gateway namespace.
[3] The Confluent Gateway application image in the following pattern.
- For Confluent Private Cloud Gateway, specify confluentinc/cpc-gateway:<version-tag>.
- For Confluent Cloud Gateway, specify confluentinc/confluent-gateway-for-cloud:<version-tag>.
<version-tag> is the Confluent Gateway release.
[4] The CFK init container image and its release tag, confluentinc/confluent-init-container:3.3.0.
[5] A list of streaming domains that represent Kafka clusters. For more information, see streaming domains.
[6] A list of credential stores. Required for authentication swapping. For more information, see secret stores.
[7] A list of client-traffic listeners that enforce governance policies and forward requests to the appropriate streaming domain. For more information, see routes.
[8] Administration and management configuration. For more information, see administration and metrics.
[9] Optional. Configuration to expose Confluent Gateway routes to clients outside the Kubernetes cluster. For more information, see external access.

Configure Confluent Gateway license

Choose a license mode before you configure the remaining CR fields. The license mode determines whether a license key is required and how many routes you can configure. The type of backing Kafka cluster determines which Confluent Gateway image you deploy. Confluent Gateway operates in Trial mode (default) or Enterprise mode, as summarized in the following table.

License mode	License key	Route limit
Trial (default)	Not required	Maximum of four routes
Enterprise	Valid license key required	Unlimited routes

For a full comparison of the license modes, see License modes.

In Enterprise mode, the license key and Confluent Gateway image depend on the type of Kafka clusters that Confluent Gateway forwards traffic to:

Enterprise deployment	License key	Image
Self-managed Kafka (Confluent Private Cloud)	Valid Confluent Private Cloud Enterprise license key	`confluentinc/cpc-gateway`
Confluent Cloud (Confluent Cloud Gateway)	Valid Confluent Cloud Gateway license key	`confluentinc/confluent-gateway-for-cloud`

Trial mode works with either image. To configure the Enterprise license mode:

Get one or more valid license keys from Confluent for production use.
Store your license key or keys in a Kubernetes secret named confluent-gateway-licenses. Use either of the following options:
- Option A: Create the secret from a file.
  1. Create a licenses.txt file with one license key per line:
    echo "your-license-key-here" > licenses.txt
    To add more keys, append each license key to the file:
    echo "your-second-license-key-here" >> licenses.txt
  2. Create the secret from the file:
    kubectl create secret generic confluent-gateway-licenses \ --from-file=licenses.txt=licenses.txt
- Option B: Create the secret from a license key without a file:
```
kubectl create secret generic confluent-gateway-licenses \
  --from-literal=licenses.txt='your-license-key'
```

Verify that the secret exists:

kubectl get secret confluent-gateway-licenses

Configure the Confluent Gateway CR to reference the license secret you created in the previous step.

kind: Gateway
spec:
  podTemplate:
    envVars:
      - name: GATEWAY_LICENSES
        valueFrom:
          secretKeyRef:
            name: confluent-gateway-licenses
            key: licenses.txt

Verify that Confluent Gateway recognizes the license by inspecting its startup logs:
```
kubectl logs -l app=confluent-gateway | grep -i license
```

For example license configurations, see Confluent Gateway examples in GitHub.

Configure streaming domains

A streaming domain is the logical representation of your Kafka cluster in Confluent Gateway. Each streaming domain defines how Confluent Gateway connects to that cluster, including its bootstrap servers and broker node ID ranges, so routes can forward client traffic to it.

kind: Gateway
spec:
  streamingDomains:
    - name:               --- [1]
      type:               --- [2]
      kafkaCluster:       --- [3]
        name:             --- [4]
        bootstrapServers: --- [5]
          - id:           --- [6]
            endpoint:     --- [7]
            tls:          --- [8]
        nodeIdRanges:     --- [9]
          - name:         --- [10]
            start:        --- [11]
            end:          --- [12]

[1] A unique name for the streaming domain.
[2] The type of the streaming domain. The only supported value is kafka, which is also the default.
[3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or self-managed Kafka, that the Confluent Gateway can route traffic to.
[4] The name of the Kafka cluster.
[5] A list of Kafka bootstrap servers. Each Kafka cluster requires at least one bootstrap server. Define one bootstrap server per security protocol listener.
[6] A unique identifier for the bootstrap server. Use the protocol name, such as SASL_PLAINTEXT, SSL, or PLAINTEXT_SASL_PLAIN, to clearly distinguish the TLS channel type and SASL mechanism.
[7] The full bootstrap endpoint URL of the Kafka broker, in the format protocol://host:port. For example, PLAINTEXT://kafka0.example.com:9092.
[8] The TLS configuration for the bootstrap endpoint. Only required for SSL and SASL_SSL endpoints.
For more information, see TLS/SSL configuration.
[9] Optional. The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.
[10] The name of the node ID range.
[11] The start of the node ID range, inclusive.
[12] The end of the node ID range, inclusive.

The following example configures a Confluent Gateway streaming domain:

kind: Gateway
spec:
  streamingDomains:
    - name: sales # unique across gateway
      type: kafka # default: kafka
      kafkaCluster:
        name: sales-cluster # default: <streamingDomain.name>
        bootstrapServers:
          - id: PLAINTEXT-1
            endpoint: PLAINTEXT://kafka0.example.com:9092
          - id: SASL_PLAINTEXT-1
            endpoint: SASL_PLAINTEXT://kafka0.example.com:9093
        nodeIdRanges:
          - name: default
            start: 0              # inclusive
            end: 3                # inclusive

Configure routes

Routes are Confluent Gateway endpoints that define how client applications connect to Kafka clusters to stream data. Clients interact with the Confluent Gateway as if it were a native Kafka cluster, while the Confluent Gateway handles routing and governance.

kind: Gateway
spec:
  routes:
    - name:                         --- [1]
      endpoint:                     --- [2]
      brokerIdentificationStrategy: --- [3]
        type:                       --- [4]
        pattern:                    --- [5]
      streamingDomain:              --- [6]
        name:                       --- [7]
        bootstrapServerId:          --- [8]
      fence:                        --- [9]
      security:                     --- [10]
      logNetwork:                   --- [11]
      logFrames:                    --- [12]

[1] The unique name of the route. Can be up to 30 characters. If not specified, defaults to route-<index>.
[2] The host:port combination that Confluent Gateway listens on. This is the external address clients use to bootstrap to the Kafka cluster.
[3] The strategy for mapping client requests to a specific Kafka broker.
[4] The type of broker identification strategy. Set to port (default) or host.
- Port strategy: The Confluent Gateway identifies each Kafka broker using a unique port number. This is the default strategy. Clients connect to different ports to reach specific brokers. For example, port 9092 to connect with broker-0, and port 9093 to connect with broker-1. The Confluent Gateway uses the nodeIdRanges you set for the streaming domain in Configure streaming domains.
  Define nodeIdRanges for all clusters associated with the route’s streaming domain.
- Host strategy: The Confluent Gateway represents each Kafka broker using a unique hostname and routes each request to the target broker based on the Server Name Indication (SNI) header. Clients use different hostnames to reach specific brokers, for example broker-0.kafka.company.com and broker-1.kafka.company.com.
  Define the hostname pattern that Confluent Gateway uses to derive these hostnames in the pattern setting ([5]).
[5] The pattern for the broker identification strategy. Required if the type of broker identification strategy is host. For example, broker-$(nodeId).eu-gw.sales.example.com:9092.
Confluent Gateway replaces $(nodeId) with the actual broker node ID at runtime.
[6] References the streaming domain that this route forwards client traffic to, identified by name ([7]) and bootstrap server ID ([8]).
[7] The name of the streaming domain. Must be a valid name from the gateway.streamingDomains[].name.
[8] The bootstrap server ID. Must match one of kafkaCluster.bootstrapServers[].id.
[9] Optional. The fencing filter configuration to control traffic flow on the route. For more information, see fencing filter.
[10] The security configuration. For more information, see security.
[11] Optional. When set to true, logs TCP-level connection details for the route. The default is false.
[12] Optional. When set to true, logs Kafka protocol frames for the route. The default is false.

The following example configures a Confluent Gateway route:

kind: Gateway
spec:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_PLAINTEXT-1
      security:
        auth: passthrough
      logNetwork: true
      logFrames: true

Configure route filters

Use filters to control and manage traffic flow for specific routes in Confluent Gateway. You can apply filters at the route level to enforce policies, block or allow requests, and customize error handling for different scenarios.

Fencing filter

The fencing filter in Confluent Gateway is a route-level filter that lets you control the traffic flow on a specific route. It supports two fencing scopes:

ALL: Blocks all requests on the specified route. This is useful for scenarios such as maintenance windows or when you want to temporarily turn off access to a route.
NONE: Allows all requests on the specified route.

kind: Gateway
spec:
  routes:
    - name:
      fence:
        scope:          --- [1]
        errorCode:      --- [2]
        errorMessage:   --- [3]

[1] The fencing scope of the filter. The default value is NONE. Set to ALL to block all requests on the route.
[2] Optional. The error code to return for blocked requests. The default value is BROKER_NOT_AVAILABLE.
For a list of valid error codes, see Kafka protocol error codes.
[3] Optional. The error message to return for blocked requests. The default value is This route is currently unavailable - all requests are blocked.

The following example configures a Confluent Gateway route with a fencing filter:

kind: Gateway
spec:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_PLAINTEXT-1
      fence:
        scope: ALL
        errorCode: BROKER_NOT_AVAILABLE
        errorMessage: "Maintenance in progress - route temporarily unavailable"
      security:
        auth: passthrough

Configure administration and metrics

The admin section configures the Confluent Gateway administrative service, including the bind address, TCP port, metrics endpoint, Java Virtual Machine (JVM) metrics collection, and custom metric tags.

kind: Gateway
spec:
  admin:
    bindAddress:             --- [1]
    port:                    --- [2]
    endpoints:
      metrics:               --- [3]
    jvmMetrics:              --- [4]
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:              --- [5]
      host:
      region:

[1] The local network address that the Confluent Gateway administration service listens on for incoming connections. The default value is 0.0.0.0, which binds to all available network interfaces.
This address determines which network interfaces the Confluent Gateway uses to expose its administrative endpoints, such as metrics and liveness checks.
[2] The TCP port for exposing admin endpoints. The default value is 9190.
The Confluent Gateway serves metrics at http://{bindAddress}:{port}/metrics and liveness check at http://{bindAddress}:{port}/livez.
[3] The metrics endpoints to enable and expose. The default value is true.
[4] The standard JVM metrics. By default, Confluent Gateway enables JvmGcMetrics, JvmMemoryMetrics, JvmThreadMetrics, ProcessorMetrics, and UptimeMetrics.
[5] Optional. Key-value tags that Confluent Gateway applies to all metrics, such as host and region.

The following example configures Confluent Gateway administration and metrics:

kind: Gateway
spec:
  admin:
    bindAddress: 0.0.0.0
    port: 9190
    endpoints:
      metrics: true
    jvmMetrics:
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:
      host: pod-0
      region: us-west-2

Configure external access

To allow client applications outside the Kubernetes cluster to connect to Confluent Gateway routes, configure the externalAccess field in the Confluent Gateway CR. Confluent Gateway uses the same external access mechanism as other Confluent Platform components deployed with CFK. For an overview of networking concepts and supported external access methods, see Configure Networking for Confluent Platform in Confluent for Kubernetes.

kind: Gateway
spec:
  externalAccess:
    type:              --- [1]
    loadBalancer:      --- [2]
      domain:
    nodePort:          --- [3]
      nodePortOffset:
      host:

[1] Required. The external access method. Set to loadBalancer or nodePort.
[2] Required if the external access method ([1]) is loadBalancer. For more information, see Load balancer.
[3] Required if the external access method ([1]) is nodePort. For more information, see Node port.

The loadBalancer and nodePort blocks are mutually exclusive. Configure only the block that matches the external access method ([1]) you set.

Load balancer

Use a load balancer to expose Confluent Gateway routes through your Kubernetes provider’s load balancer service.

kind: Gateway
spec:
  externalAccess:
    type: loadBalancer
    loadBalancer:
      domain:      --- [1]
      prefix:      --- [2]

[1] Required. The base domain used to construct the DNS hostname (<prefix>.<domain>) that CFK publishes for the load balancer service.
[2] Optional. The sub-domain or prefix added before domain. Defaults to the Confluent Gateway CR’s metadata.name.

To configure service hostnames and route DNS records, see External access DNS configuration.

The following example configures external load balancer access for Confluent Gateway:

kind: Gateway
spec:
  externalAccess:
    type: loadBalancer
    loadBalancer:
      domain: example.com
      prefix: eu-gw

For internal load balancers, provider-specific annotations, and other load balancer options, see External access to other Confluent Platform components using load balancers.

Node port

Use node ports to expose Confluent Gateway routes at a static port on each Kubernetes worker node.

kind: Gateway
spec:
  externalAccess:
    type: nodePort
    nodePort:
      nodePortOffset:    --- [1]
      host:              --- [2]

[1] Required. The starting port for the NodePort range. The value must be within the node port range configured on your Kubernetes API server. The default range is 30000 to 32767.
[2] Required. The fully qualified domain name (FQDN) that CFK publishes for the Confluent Gateway node port service.

To configure service hostnames and route DNS records, see External access DNS configuration.

The following example configures Confluent Gateway to use node ports for external access:

kind: Gateway
spec:
  externalAccess:
    type: nodePort
    nodePort:
      nodePortOffset: 30000
      host: gateway.example.com

For firewall rules and other node port considerations, see Configure external access to other Confluent Platform components using node ports.

External access DNS configuration

External access requires configuring DNS records for both the underlying service and your individual route endpoints.

Service hostnames and automated DNS

CFK applies your configured hostname (the load balancer domain and prefix or the node port host) as an external-dns.alpha.kubernetes.io/hostname annotation on the service:

Automated DNS: If your cluster runs external-dns, the DNS record is created automatically.
Manual DNS: If you do not use external-dns, manually create a DNS record assigning the hostname to your external address (the load balancer IP or CNAME, or worker node IPs).

Service hostnames are independent of the route endpoint values configured in Configure routes. You must ensure that every hostname used by your routes resolves to your external address.

Caution

Updating fields: Modifying domain, prefix, host, or nodePortOffset updates only the underlying service and does not require rolling cluster resources.
Changing access type: Changing type provisions a new service with a different external address instead of updating the existing one. CFK does not automatically delete the original service. You must manually delete the service after verifying that client traffic has migrated to the new address.

Route endpoint DNS records

Beyond the service hostname, create DNS records that resolve your route endpoints to your external address (the load balancer or worker nodes).

The required records depend on your broker identification strategy. For more information, see Configure routes:

Port strategy: Create a single A record for the endpoint host:
```
gateway.example.com    A    <external-address>
```
Host strategy: Create an A record and a wildcard A record so every broker hostname resolves to the same address:
```
gateway.example.com    A    <external-address>
*.gateway.example.com  A    <external-address>
```

Deploy Confluent Gateway

Deploy Confluent Gateway by applying the CR file you created in Configure Confluent Gateway:

kubectl apply -f <gateway-CR-file>

Reconfigure clients

After Confluent Gateway is running, connect your client applications to the Confluent Gateway route endpoint instead of connecting to brokers directly. Set bootstrap.servers to the route endpoint you configured. Routing client traffic through Confluent Gateway simplifies credential management and avoids exposing your brokers directly to clients.

Apply security configuration

Configure security for Confluent Gateway, including secret stores, TLS/SSL, passwords, and authentication swapping.

Next steps

To diagnose and resolve deployment issues, see Troubleshoot Confluent Gateway CFK Deployments.