Configure and Deploy Confluent Gateway using Confluent for Kubernetes

Configure and deploy Confluent Gateway using Confluent for Kubernetes (CFK). Use this deployment path to run Confluent Gateway on-premises, in virtual private clouds (VPCs), and across hybrid environments.

Before deploying Confluent Gateway with CFK, ensure you meet the following prerequisites in addition to the general Confluent Gateway requirements:

A Kubernetes environment with CFK and Confluent Platform installed. For details on installing CFK, see Deploy Confluent for Kubernetes. Use a version of CFK that supports your Confluent Platform and Confluent Gateway requirements.
A valid CFK license. If you already have a CFK license for other Confluent Platform components such as Apache Kafka® or Schema Registry, you can reuse it for Confluent Gateway without purchasing an additional license.

For general requirements, license modes, and supported component compatibility, see Confluent Gateway Requirements and Considerations.

The high-level steps to deploy Confluent Gateway using CFK are:

Configure Gateway: Configure Confluent Gateway by defining the streaming domains that connect to your Kafka clusters, the routes that clients use to reach them, and the administration and metrics settings.
Deploy Gateway: Deploy the Confluent Gateway service using the configuration settings from the previous step.
Reconfigure clients: Point client applications to the Confluent Gateway route endpoint instead of connecting to brokers directly, by setting bootstrap.servers to the route endpoint you configured.
Apply security configuration: Configure security for Confluent Gateway, including secret stores, TLS/SSL, passwords, and authentication swapping.

Note

The Kubernetes commands in this guide assume you have set a default namespace, so they omit the --namespace flag for brevity. If you have not set a default namespace, add --namespace <your-namespace> to each command.

Configure Confluent Gateway

To configure Confluent Gateway, create a YAML file that defines a Confluent Gateway custom resource (CR), for example gateway.yaml, and add the configuration settings as described in the following sections. In the Deploy Confluent Gateway step, you apply this file to your Kubernetes cluster with kubectl apply.

The top-level layout for the Confluent Gateway CR is:

kind: Gateway
metadata:
  name:                   --- [1]
  namespace:              --- [2]
spec:
  image:
    application:          --- [3]
    init:                 --- [4]
  streamingDomains:       --- [5]
  secretStores:           --- [6]
  routes:                 --- [7]
  admin:                  --- [8]

[1] The Confluent Gateway name.
[2] The Confluent Gateway namespace.
[3] The Confluent Gateway application image in the following pattern.
- For Confluent Private Cloud Gateway, specify confluentinc/cpc-gateway:<version-tag>.
- For Confluent Cloud Gateway, specify confluentinc/confluent-gateway-for-cloud:<version-tag>.
<version-tag> is the Confluent Gateway release.
[4] The CFK init container image and its release tag, confluentinc/confluent-init-container:3.3.0.
[5] A list of streaming domains that represent Kafka clusters. For more information, see streaming domains.
[6] A list of credential stores. Required for authentication swapping. For more information, see secret stores.
[7] A list of client-traffic listeners that enforce governance policies and forward requests to the appropriate streaming domain. For more information, see routes.
[8] Administration and management configuration. For more information, see administration and metrics.

Configure Confluent Gateway license

Choose a license mode before you configure the remaining CR fields. The license mode determines whether a license key is required and how many routes you can configure. The type of backing Kafka cluster determines which Confluent Gateway image you deploy. Confluent Gateway operates in Trial mode (default) or Enterprise mode, as summarized in the following table.

License mode	License key	Route limit
Trial (default)	Not required	Maximum of four routes
Enterprise	Valid license key required	Unlimited routes

For a full comparison of the license modes, see License modes.

In Enterprise mode, the license key and Confluent Gateway image depend on the type of Kafka clusters that Confluent Gateway forwards traffic to:

Enterprise deployment	License key	Image
Self-managed Kafka (Confluent Private Cloud)	Valid Confluent Private Cloud Enterprise license key	`confluentinc/cpc-gateway`
Confluent Cloud (Confluent Cloud Gateway)	Valid Confluent Cloud Gateway license key	`confluentinc/confluent-gateway-for-cloud`

Trial mode works with either image. To configure the Enterprise license mode:

Get one or more valid license keys from Confluent for production use.
Store your license key or keys in a Kubernetes secret named confluent-gateway-licenses. Use either of the following options:
- Option A: Create the secret from a file.
  1. Create a licenses.txt file with one license key per line:
    echo "your-license-key-here" > licenses.txt
    To add more keys, append each license key to the file:
    echo "your-second-license-key-here" >> licenses.txt
  2. Create the secret from the file:
    kubectl create secret generic confluent-gateway-licenses \ --from-file=licenses.txt=licenses.txt
- Option B: Create the secret from a license key without a file:
```
kubectl create secret generic confluent-gateway-licenses \
  --from-literal=licenses.txt='your-license-key'
```

Verify that the secret exists:

kubectl get secret confluent-gateway-licenses

Configure the Confluent Gateway CR to reference the license secret you created in the previous step.

kind: Gateway
spec:
  podTemplate:
    envVars:
      - name: GATEWAY_LICENSES
        valueFrom:
          secretKeyRef:
            name: confluent-gateway-licenses
            key: licenses.txt

Verify that Confluent Gateway recognizes the license by inspecting its startup logs:
```
kubectl logs -l app=confluent-gateway | grep -i license
```

For example license configurations, see Confluent Gateway examples in GitHub.

Configure streaming domains

A streaming domain is the logical representation of your Kafka cluster in Confluent Gateway. Each streaming domain defines how Confluent Gateway connects to that cluster, including its bootstrap servers and broker node ID ranges, so routes can forward client traffic to it.

kind: Gateway
spec:
  streamingDomains:
    - name:               --- [1]
      type:               --- [2]
      kafkaCluster:       --- [3]
        name:             --- [4]
        bootstrapServers: --- [5]
          - id:           --- [6]
            endpoint:     --- [7]
            tls:          --- [8]
        nodeIdRanges:     --- [9]
          - name:         --- [10]
            start:        --- [11]
            end:          --- [12]

[1] A unique name for the streaming domain.
[2] The type of the streaming domain. The only supported value is kafka, which is also the default.
[3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or self-managed Kafka, that the Confluent Gateway can route traffic to.
[4] The name of the Kafka cluster.
[5] A list of Kafka bootstrap servers. Each Kafka cluster requires at least one bootstrap server. Define one bootstrap server per security protocol listener.
[6] A unique identifier for the bootstrap server. Use the protocol name, such as SASL_PLAINTEXT, SSL, or PLAINTEXT_SASL_PLAIN, to clearly distinguish the TLS channel type and SASL mechanism.
[7] The full bootstrap endpoint URL of the Kafka broker, in the format protocol://host:port. For example, PLAINTEXT://kafka0.example.com:9092.
[8] The TLS configuration for the bootstrap endpoint. Only required for SSL and SASL_SSL endpoints.
For more information, see TLS/SSL configuration.
[9] Optional. The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.
[10] The name of the node ID range.
[11] The start of the node ID range, inclusive.
[12] The end of the node ID range, inclusive.

The following example configures a Confluent Gateway streaming domain:

kind: Gateway
spec:
  streamingDomains:
    - name: sales # unique across gateway
      type: kafka # default: kafka
      kafkaCluster:
        name: sales-cluster # default: <streamingDomain.name>
        bootstrapServers:
          - id: PLAINTEXT-1
            endpoint: PLAINTEXT://kafka0.example.com:9092
          - id: SASL_PLAINTEXT-1
            endpoint: SASL_PLAINTEXT://kafka0.example.com:9093
        nodeIdRanges:
          - name: default
            start: 0              # inclusive
            end: 3                # inclusive

Configure routes

Routes are Confluent Gateway endpoints that define how client applications connect to Kafka clusters to stream data. Clients interact with the Confluent Gateway as if it were a native Kafka cluster, while the Confluent Gateway handles routing and governance.

kind: Gateway
spec:
  routes:
    - name:                         --- [1]
      endpoint:                     --- [2]
      brokerIdentificationStrategy: --- [3]
        type:                       --- [4]
        pattern:                    --- [5]
      streamingDomain:              --- [6]
        name:                       --- [7]
        bootstrapServerId:          --- [8]
      fence:                        --- [9]
      security:                     --- [10]
      logNetwork:                   --- [11]
      logFrames:                    --- [12]

[1] The unique name of the route. Can be up to 30 characters. If not specified, defaults to route-<index>.
[2] The host:port combination that Confluent Gateway listens on. This is the external address clients use to bootstrap to the Kafka cluster.
[3] The strategy for mapping client requests to a specific Kafka broker.
[4] The type of broker identification strategy. Set to port (default) or host.
- Port strategy: The Confluent Gateway identifies each Kafka broker using a unique port number. This is the default strategy. Clients connect to different ports to reach specific brokers. For example, port 9092 to connect with broker-0, and port 9093 to connect with broker-1. The Confluent Gateway uses the nodeIdRanges you set for the streaming domain in Configure streaming domains.
  Define nodeIdRanges for all clusters associated with the route’s streaming domain.
- Host strategy: The Confluent Gateway represents each Kafka broker using a unique hostname and routes each request to the target broker based on the Server Name Indication (SNI) header. Clients use different hostnames to reach specific brokers, for example broker-0.kafka.company.com and broker-1.kafka.company.com.
  Define the hostname pattern that Confluent Gateway uses to derive these hostnames in the pattern setting ([5]).
[5] The pattern for the broker identification strategy. Required if the type of broker identification strategy is host. For example, broker-$(nodeId).eu-gw.sales.example.com:9092.
Confluent Gateway replaces $(nodeId) with the actual broker node ID at runtime.
[6] References the streaming domain that this route forwards client traffic to, identified by name ([7]) and bootstrap server ID ([8]).
[7] The name of the streaming domain. Must be a valid name from the gateway.streamingDomains[].name.
[8] The bootstrap server ID. Must match one of kafkaCluster.bootstrapServers[].id.
[9] Optional. The fencing filter configuration to control traffic flow on the route. For more information, see fencing filter.
[10] The security configuration. For more information, see security.
[11] Optional. When set to true, logs TCP-level connection details for the route. The default is false.
[12] Optional. When set to true, logs Kafka protocol frames for the route. The default is false.

The following example configures a Confluent Gateway route:

kind: Gateway
spec:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_PLAINTEXT-1
      security:
        auth: passthrough
      logNetwork: true
      logFrames: true

Configure route filters

Use filters to control and manage traffic flow for specific routes in Confluent Gateway. You can apply filters at the route level to enforce policies, block or allow requests, and customize error handling for different scenarios.

Fencing filter

The fencing filter in Confluent Gateway is a route-level filter that lets you control the traffic flow on a specific route. It supports two fencing scopes:

ALL: Blocks all requests on the specified route. This is useful for scenarios such as maintenance windows or when you want to temporarily disable access to a route.
NONE: Allows all requests on the specified route.

kind: Gateway
spec:
  routes:
    - name:
      fence:
        scope:          --- [1]
        errorCode:      --- [2]
        errorMessage:   --- [3]

[1] The fencing scope of the filter. The default value is NONE. Set to ALL to block all requests on the route.
[2] Optional. The error code to return for blocked requests. The default value is BROKER_NOT_AVAILABLE.
For a list of valid error codes, see Kafka protocol error codes.
[3] Optional. The error message to return for blocked requests. The default value is This route is currently unavailable - all requests are blocked.

The following example configures a Confluent Gateway route with a fencing filter:

kind: Gateway
spec:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_PLAINTEXT-1
      fence:
        scope: ALL
        errorCode: BROKER_NOT_AVAILABLE
        errorMessage: "Maintenance in progress - route temporarily unavailable"
      security:
        auth: passthrough

Configure administration and metrics

The admin section configures the Confluent Gateway administrative service, including the bind address, TCP port, metrics endpoint, Java Virtual Machine (JVM) metrics collection, and custom metric tags.

kind: Gateway
spec:
  admin:
    bindAddress:             --- [1]
    port:                    --- [2]
    endpoints:
      metrics:               --- [3]
    jvmMetrics:              --- [4]
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:              --- [5]
      host:
      region:

[1] The local network address that the Confluent Gateway administration service listens on for incoming connections. The default value is 0.0.0.0, which binds to all available network interfaces.
This address determines which network interfaces the Confluent Gateway uses to expose its administrative endpoints, such as metrics and liveness checks.
[2] The TCP port for exposing admin endpoints. The default value is 9190.
The Confluent Gateway serves metrics at http://{bindAddress}:{port}/metrics and liveness check at http://{bindAddress}:{port}/livez.
[3] The metrics endpoints to enable and expose. The default value is true.
[4] The standard JVM metrics. By default, Confluent Gateway enables JvmGcMetrics, JvmMemoryMetrics, JvmThreadMetrics, ProcessorMetrics, and UptimeMetrics.
[5] Optional. Key-value tags that Confluent Gateway applies to all metrics, such as host and region.

The following example configures Confluent Gateway administration and metrics:

kind: Gateway
spec:
  admin:
    bindAddress: 0.0.0.0
    port: 9190
    endpoints:
      metrics: true
    jvmMetrics:
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:
      host: pod-0
      region: us-west-2

Deploy Confluent Gateway

Deploy Confluent Gateway by applying the CR file you created in Configure Confluent Gateway:

kubectl apply -f <gateway-CR-file>

Reconfigure clients

After Confluent Gateway is running, connect your client applications to the Confluent Gateway route endpoint instead of connecting to brokers directly. Set bootstrap.servers to the route endpoint you configured. Routing client traffic through Confluent Gateway simplifies credential management and avoids exposing your brokers directly to clients.

Apply security configuration

Configure security for Confluent Gateway, including secret stores, TLS/SSL, passwords, and authentication swapping.

Next steps

To diagnose and resolve deployment issues, see Troubleshoot Confluent Gateway CFK Deployments.