Configure and Deploy Confluent Cloud Gateway

This guide provides instructions for configuring and deploying Confluent Cloud Gateway (Confluent Gateway) using Docker.

Confluent Gateway runs as a stateless service, often as a set of containerized instances within the target environment (for example, Confluent for Kubernetes, VM clusters, or cloud-native platforms).

A separate guide is provided for the security-related tasks. See Configure Security for Confluent Cloud Gateway:

  • Configure secret stores

  • Configure TLS/SSL

  • Configure passwords

  • Configure authentication swapping

Confluent Gateway supports deployment on-premises, in private cloud VPCs, or in hybrid environments.

The high-level steps to deploy Confluent Gateway are:

  1. Configure Confluent Gateway to connect to Kafka clusters.

    Additionally, Configure security for the Confluent Gateway to specify authentication, routing policies, and security policies according to your organizational needs.

  2. Provision the Confluent Gateway service using the configuration settings from the previous step.

  3. Reconfigure clients to communicate with the Confluent Gateway endpoint.

    This simplifies credential management and strengthens overall security posture.

Configure Confluent Gateway using Docker Compose

Provide the following configuration settings in a Docker Compose YAML file. docker-compose.yaml is used as an example in this guide.

The top-level layout for the Confluent Gateway configuration file is as follows:

gateway:
  name:                   --- [1]
  streamingDomains: []    --- [2]
  secretStores: []        --- [3]
  routes: []              --- [4]
  admin: {}               --- [5]
  advanced: {}            --- [6]

The Confluent Gateway Docker image is not a Confluent Gateway configuration setting. Set it with the image key on the Docker Compose service. Use confluentinc/confluent-gateway-for-cloud:<version-tag>, where <version-tag> is the Confluent Gateway version. For example, use confluentinc/confluent-gateway-for-cloud:1.1.0 to install version 1.1.0 of the Confluent Gateway.

Streaming domains configuration

Streaming domains are logical representations of your Kafka clusters in Confluent Gateway.

gateway:
  streamingDomains:
    - name:               --- [1]
      type:               --- [2]
      kafkaCluster:       --- [3]
        name:             --- [4]
        bootstrapServers: --- [5]
          - id:           --- [6]
            endpoint:     --- [7]
            ssl:          --- [8]
        nodeIdRanges:     --- [9]
          - name:         --- [10]
            start:        --- [11]
            end:          --- [12]
  • [1] A unique name for the streaming domain.

  • [2] The type of the streaming domain. Set to kafka by default.

  • [3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or Apache Kafka®, that the Confluent Gateway can route traffic to.

  • [4] The name of the Kafka cluster.

  • [5] A list of Kafka bootstrap servers in the format host:port. At minimum, one bootstrap server is required per Kafka cluster. For each listener, you can define one bootstrap server.

  • [6] A unique identifier for the bootstrap server. Use the protocol name (such as SASL_PLAINTEXT, SSL, or PLAINTEXT_SASL_PLAIN) for clear distinction between the TLS channel type and SASL mechanism.

  • [7] The bootstrap endpoint of the Kafka broker in the format: host:port.

  • [8] Only required for SSL/SASL_SSL configuration. Supported certificate types are JKS, PKCS12, and PEM.

    See the SSL configuration section for details.

  • [9] The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.

  • [10] The name of the node ID range.

  • [11] The start of the node ID range, inclusive.

  • [12] The end of the node ID range, inclusive.

An example configuration for a Confluent Gateway streaming domain:

gateway:
  streamingDomains:
    - name: sales # unique across gateway
      type: kafka # default: kafka
      kafkaCluster:
        name: sales-cluster # default: <streamingDomain.name>
        bootstrapServers:
          - id: PLAINTEXT-1
            endpoint: kafka0.example.com:9092
          - id: SASL_SSL-1
            endpoint: kafka0.example.com:9093
            ssl:  # required when using SASL_SSL/SSL
              ignoreTrust: false
              truststore:
                type: PKCS12  # default: JKS
                location: /opt/ssl/client-truststore.p12
                password:
                  file: /opt/secrets/client-truststore.password
              keystore: # gateway's identity when needed
                type: PKCS12 # default: JKS
                location: /opt/ssl/gw-keystore.p12
                password:
                  file: /opt/secrets/gw-keystore.password
                keyPassword:
                  value: inline-password
        nodeIdRanges:
          - name: default
            start: 0              # inclusive
            end: 3                # inclusive
  • password can be a file path or an inline value. For details, see Password configuration.

  • ignoreTrust skips certificate validation. Not recommended for production. When true, Confluent Gateway ignores the truststore setting.

Routes configuration

Routes are Confluent Gateway endpoints where client applications connect to stream data.

Confluent Gateway uses routes to define how client applications connect to Kafka clusters. Clients connect to Confluent Gateway as if it were a Kafka cluster, while Confluent Gateway handles routing and governance.

gateway:
  routes:
    - name:                         --- [1]
      endpoint:                     --- [2]
      mode:                         --- [3]
      brokerIdentificationStrategy: --- [4]
        type:                       --- [5]
        pattern:                    --- [6]
      streamingDomain:              --- [7]
        name:                       --- [8]
        bootstrapServerId:          --- [9]
      streamingDomains:             --- [10]
        - name:                     --- [11]
          bootstrapServerId:        --- [12]
      rules:                        --- [13]
      security:                     --- [14]
  • [1] The unique name for the route. The name can’t contain spaces.

  • [2] The host:port combination that Confluent Gateway listens on. This is the external address clients use to bootstrap to the Kafka cluster.

  • [3] Early Access. Optional. The routing mode of the route. Set to STATIC or DYNAMIC. The value isn’t case-sensitive.

    • STATIC: The route is bound to a single streaming domain ([7]).

    • DYNAMIC: The route uses topic-based routing. You bind it to a list of streaming domains ([10]), and it routes each topic to the streaming domain that owns it.

    If you omit mode, Confluent Gateway infers it from the streaming domain binding: streamingDomain ([7]) sets the mode to STATIC, and streamingDomains ([10]) sets the mode to DYNAMIC. If you set mode, it must match the binding. For example, mode: DYNAMIC with streamingDomain is invalid.

  • [4] Specifies the strategy for mapping client requests to a specific Kafka broker.

  • [5] The type of broker identification strategy. Set to port (the default) or host. A dynamic route ([10]) supports only the host strategy. Confluent Gateway assigns its own node IDs to the brokers of the bound clusters, so overlapping broker IDs across clusters don’t conflict.

    • port strategy: Each Kafka broker is identified using a unique port number. This is the default strategy.

      Clients connect to different ports to reach specific brokers (for example, port 9092 to connect with broker-0, port 9093 to connect with broker-1).

      The nodeIdRanges from the streaming domain configured in Streaming domains configuration is used. The nodeIdRanges must be present in the streaming domain associated with the route.

    • host strategy: Each Kafka broker is represented using a unique hostname.

      Clients use different host names to reach specific brokers (for example, broker-0.kafka.company.com, broker-1.kafka.company.com), and Confluent Gateway routes based on the Server Name Indication (SNI) header.

      The pattern setting ([6]) is used.

  • [6] The pattern for the broker identification strategy. Required if the type ([5]) is host. For example, broker-$(nodeId).eu-gw.sales.example.com:9092.

  • [7] The reference to a streaming domain. Use this setting for a static route. Required if the route doesn’t use streamingDomains ([10]).

  • [8] The name of the streaming domain. Must be a valid name from the gateway.streamingDomains[].name.

  • [9] The bootstrap server ID. Must match kafkaCluster.bootstrapServers[].id.

  • [10] Early Access. The list of references to streaming domains that the route is bound to. Use this setting for a dynamic route. Required if the route doesn’t use streamingDomain ([7]). Setting streamingDomains makes the route a dynamic route. You can’t bind the same streaming domain more than once, and the list can’t contain null entries.

  • [11] The name of a streaming domain to bind to the route. Must be a valid name from the gateway.streamingDomains[].name.

  • [12] The bootstrap server ID to use for the streaming domain ([11]). Must match kafkaCluster.bootstrapServers[].id.

  • [13] Optional. The routing rules (Early Access) and fencing rules of the route. For more information, see Routing rules configuration (Early Access) and Fencing rules configuration.

  • [14] The security configuration. For more information, see Configure Security for Confluent Cloud Gateway.

Each route must specify exactly one of streamingDomain ([7]) or streamingDomains ([10]). Specifying both or neither is invalid.

The following examples show a static route and a dynamic route:

A static route binds one streaming domain with streamingDomain.

gateway:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_SSL-1

A dynamic route binds a list of streaming domains with streamingDomains and requires rules.routing. For more information, see Routing rules configuration (Early Access).

gateway:
  routes:
    - name: migration-route
      endpoint: migration-gw.example.com:9092
      mode: DYNAMIC
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).migration-gw.example.com:9092
      streamingDomains:
        - name: cp
          bootstrapServerId: cp-bootstrap
        - name: cc
          bootstrapServerId: cc-bootstrap
      rules:
        routing:
          coordination:
            group: cc
          default: cp

Routing rules configuration (Early Access)

Configure rules.routing on dynamic routes (mode: DYNAMIC) to map topics to specific streaming domains. For more information, see Routes configuration.

You configure routing rules in the rules.routing setting of the route. A dynamic route requires rules.routing, and a static route can’t use it.

Note

Topic-based routing is an Early Access feature. Property names and YAML structure might change before General Availability (GA), and the changes might not be backward compatible.

gateway:
  routes:
    - name:
      streamingDomains:
      rules:
        routing:                        --- [1]
          coordination:                 --- [2]
            group:                      --- [3]
          conditions:                   --- [4]
            - topics:                   --- [5]
              topicPatterns:            --- [6]
              streamingDomain:          --- [7]
          conditionsFile:               --- [8]
          default:                      --- [9]
  • [1] The routing rules that map topics to the streaming domains that own them. Required on a dynamic route.

  • [2] Required. Pins consumer group coordination to a streaming domain.

  • [3] Required. The name of the streaming domain that coordinates all consumer groups on the route, including their committed offsets. Must be a streaming domain bound to the route. All consumer group offsets for the route are stored on this cluster. Disable consumer offset sync on the cluster link, and on any other replicator that you use, so that replication doesn’t overwrite offsets that clients commit through Confluent Gateway.

  • [4] Optional. The list of conditions that map topics to streaming domains. Confluent Gateway evaluates the conditions in order, and the first condition that matches a topic decides which streaming domain owns it. You can’t use conditions with conditionsFile ([8]).

  • [5] A list of exact topic names. Confluent Gateway matches topic names literally.

  • [6] A list of Java regular expressions. A pattern must match the entire topic name. For example, the pattern teamA.* matches teamA-orders, but the pattern teamA doesn’t. Confluent Gateway validates the patterns when it loads the configuration.

    To match a literal period, escape it with a backslash. How you write the backslash depends on the type of YAML string:

    • In a double-quoted string, write the backslash twice, for example "teamA\\..*".

    • In a single-quoted or unquoted string, write the backslash once, for example 'teamA\..*'.

    Both examples match topics that start with teamA..

    A condition matches a topic if the topic is listed in topics ([5]) or matches any pattern in topicPatterns ([6]). Each condition requires at least one of topics or topicPatterns.

  • [7] Required for each condition. The name of the streaming domain that owns the matching topics. Must be a streaming domain bound to the route.

  • [8] Optional. The path to a YAML file that contains the list of conditions, as an alternative to inline conditions ([4]). The path is resolved against the working directory of Confluent Gateway inside the container, so mount the file into the container, for example with a volumes entry in docker-compose.yaml. The file must be readable and contain at least one condition. Confluent Gateway reads the file when it loads the configuration and watches only the main configuration file, so a change to the conditions file takes effect only after the configuration reloads.

  • [9] Optional. The name of the streaming domain that owns every topic that no condition matches. Must be a streaming domain bound to the route. If you omit default, the route is strict. Confluent Gateway doesn’t show clients a topic that no condition matches, and it rejects requests to create, delete, read, or write that topic.

Every streaming domain that you reference in coordination, conditions, and default must be one of the streaming domains in the route’s streamingDomains list.

The following example shows the rules.routing settings of a route:

gateway:
  routes:
    - name: migration-route
      rules:
        routing:
          coordination:
            group: cc
          conditions:
            - topics: ["payments"]
              streamingDomain: cp
            - topicPatterns: ["teamA\\..*"]
              streamingDomain: cc
          default: cp

For an example of a complete route, see the configure step and the reroute step of the migration procedure.

To keep the conditions in a separate file, set conditionsFile to the path of a YAML file instead of listing conditions. The file contains the list of conditions, with the same fields as the inline conditions list:

- topics: ["payments"]
  streamingDomain: cp
- topicPatterns: ["teamA\\..*"]
  streamingDomain: cc

Route filters configuration

Filters allow you to control and manage traffic flow for specific routes in Confluent Gateway. You can apply filters at the route level to enforce policies, block or allow requests, and customize error handling for different scenarios.

Fencing filter

The fencing filter in Confluent Gateway is a route-level filter that allows you to control the traffic flow on a specific route. You can configure fencing in two ways:

  • fence (deprecated): Blocks or allows all requests on a route. Use rules.fencing instead. For more information, see Fence an entire route.

  • rules.fencing: Blocks or allows traffic by traffic type and topic. You can use rules.fencing on static and dynamic routes. For more information, see Fencing rules configuration.

A route can use fence or rules.fencing, but not both.

Fence an entire route

The fence configuration supports two fencing scopes:

  • ALL: Blocks all requests on the specified route. Useful for scenarios such as maintenance windows or when you want to temporarily disable access to a route.

  • NONE: Allows all requests on the specified route.

gateway:
  routes:
    - name:
      fence:
        scope:          --- [1]
        errorCode:      --- [2]
        errorMessage:   --- [3]
  • [1] Optional. The fencing scope of the filter. The default value is ALL, which blocks all requests on the route. Set to NONE to allow all requests on the route. If you add a fence section without scope, Confluent Gateway blocks all requests on the route.

  • [2] Optional. The error code to return for blocked requests. The default value is BROKER_NOT_AVAILABLE.

    For a list of valid error codes, see Kafka protocol error codes.

  • [3] Optional. The error message to return for blocked requests. The default value is This route is currently unavailable - all requests are blocked.

    Some Kafka protocol APIs such as ApiVersions do not include a message field in their error response schema. For these APIs, Confluent Gateway can’t return custom errorMessage strings to the client, and blocked requests might appear as generic connection or handshake failures rather than the configured error.

The following is an example configuration for a Confluent Gateway route with the fencing filter enabled:

gateway:
  routes:
    - name: eu-sales
      endpoint: eu-gw.sales.example.com:9092
      fence:
        scope: ALL
        errorCode: BROKER_NOT_AVAILABLE
        errorMessage: "Maintenance in progress - route temporarily unavailable"
      brokerIdentificationStrategy:
        type: host
        pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
      streamingDomain:
        name: sales
        bootstrapServerId: SASL_SSL-1

This example shows only the settings that the fencing filter requires. A route that uses type: host also requires an SSL keystore in its security settings. For more information, see Configure Security for Confluent Cloud Gateway.

Fencing rules configuration

Use the rules.fencing setting to block traffic by traffic type and topic, instead of blocking the entire route. For example, you can block produce requests to a set of topics while those topics migrate to another cluster, and let all other traffic continue. To learn how to use fencing to migrate a topic, see Migrate topics between clusters.

gateway:
  routes:
    - name:
      rules:
        fencing:                        --- [1]
          - trafficType:                --- [2]
            topics:                     --- [3]
            topicPatterns:              --- [4]
            blocked:                    --- [5]
  • [1] The list of fencing rules. Confluent Gateway evaluates the rules in order, and the first rule that matches a request decides whether to block it. Confluent Gateway doesn’t fence requests that match no rule. Each rule must set at least one of trafficType ([2]), topics ([3]), or topicPatterns ([4]). A route can’t use both rules.fencing and fence.

  • [2] Optional. The traffic type that the rule applies to. Set to PRODUCE or CONSUME. If you omit trafficType, the rule applies to both types.

    • PRODUCE: Requests that write records.

    • CONSUME: Requests that read records or manage consumer groups and offsets. For the requests that each type covers, see the responses table.

    A rule that lists topics or topicPatterns doesn’t fence consumer group membership requests (join, sync, heartbeat, and leave), because they don’t name topics. To fence them, use a CONSUME rule that sets neither topics nor topicPatterns. Such a rule blocks all consume traffic on the route, for every topic and consumer group, not only for the topics that you migrate.

  • [3] Optional. A list of exact topic names that the rule applies to.

  • [4] Optional. A list of Java regular expressions that the rule applies to, with the same syntax as topicPatterns in routing rules. For more information about pattern syntax, see Routing rules configuration (Early Access).

    A rule matches a topic if the topic is listed in topics ([3]) or matches any pattern in topicPatterns ([4]).

  • [5] Required. Set to true to block the traffic that matches the rule, or false to allow it. To make an exception to a broader true rule, list a false rule for those topics before it, because the first matching rule wins.

A rule that sets only trafficType applies to all traffic of that type.

The following example blocks produce requests to the orders topic and to all topics that start with teamA., except for teamA.audit. The false rule comes first because Confluent Gateway evaluates the rules in order:

gateway:
  routes:
    - name: migration-route
      rules:
        fencing:
          - trafficType: PRODUCE
            topics: ["teamA.audit"]
            blocked: false
          - trafficType: PRODUCE
            topics: ["orders"]
            topicPatterns: ["teamA\\..*"]
            blocked: true

When a rule blocks a request, the response depends on the request type. Fencing returns errors that clients can retry, so most clients retry instead of failing. Clients can retry these errors, which aligns with Kafka behavior, so client applications don’t need extra error handling. Unlike fence, which has an errorCode setting, fencing rules don’t let you configure the error codes. The following table lists the response that each type of request receives when a fencing rule blocks it:

Traffic type

Request

Response for blocked requests

PRODUCE

Produce

NOT_ENOUGH_REPLICAS

CONSUME

Fetch

An empty response with no error, so the consumer polls again.

CONSUME

ListOffsets

OFFSET_NOT_AVAILABLE

CONSUME

OffsetCommit

COORDINATOR_LOAD_IN_PROGRESS

CONSUME

OffsetForLeaderEpoch

NOT_LEADER_OR_FOLLOWER

CONSUME

JoinGroup, SyncGroup, Heartbeat, LeaveGroup

COORDINATOR_LOAD_IN_PROGRESS

Client timeouts limit how long a fence can last. A producer stops retrying when its delivery.timeout.ms expires (120,000 ms by default). If a consumer can’t send heartbeats before its session.timeout.ms expires, the group coordinator might remove it from its consumer group. Keep fences as short as possible.

If a request names more than one topic and only some of them are fenced, Confluent Gateway returns the fencing error, or an empty result for fetch requests, for the fenced topics and processes the rest.

For the requests that Confluent Gateway doesn’t fence, such as OffsetFetch, and the effect of fencing on producers that use acks=0, see Limitations.

Administration and metrics configuration

gateway:
  admin:
    bindAddress:             --- [1]
    port:                    --- [2]
    endpoints:
      metrics:               --- [3]
    jvmMetrics:              --- [4]
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:              --- [5]
      host:
      region:
  • [1] The local address to bind the admin service and to listen on for incoming connections. The default value is 0.0.0.0.

    This address determines which network interfaces the Confluent Gateway uses to expose its administrative endpoints, such as metrics and liveness checks.

  • [2] The TCP port for exposing admin endpoints. The default value is 9190.

    The Confluent Gateway serves metrics at http://{bindAddress}:{port}/metrics. It exposes a liveness check at http://{bindAddress}:{port}/livez.

  • [3] The metrics endpoints to be enabled and exposed. The default value is true.

  • [4] Standard JVM metrics captured using Micrometer classes. By default, JvmGcMetrics, JvmMemoryMetrics, JvmThreadMetrics, ProcessorMetrics, and UptimeMetrics are enabled.

  • [5] Optional. The common tags to be applied to all the metrics.

An example configuration for Confluent Gateway administration and metrics:

gateway:
  admin:
    bindAddress: 0.0.0.0
    port: 9190
    endpoints:
      metrics: true
    jvmMetrics:
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:
      host: pod-0
      region: us-west-2

For a recommended set of metrics to monitor and example alerting queries, see Monitor Confluent Cloud Gateway Metrics.

License configuration

The table below summarizes the license modes supported by Confluent Cloud Gateway.

Deployment mode

Trial mode (default)

Enterprise mode for Confluent Cloud deployments

License required

No license required

Valid Confluent Cloud Gateway license(s)

Limitation

Maximum of 4 routes can be configured

Supports only Confluent Cloud streaming domains

Purpose

Evaluation and testing

Gateway forwarding to Confluent Cloud clusters

Duration

N/A

As specified in the license claim

Image

Use confluentinc/confluent-gateway-for-cloud

Use confluentinc/confluent-gateway-for-cloud

Self-managed and Confluent Cloud routing require separate license grants. Trial mode enables both features automatically. In Enterprise mode, your license key must include both stacked grants to migrate or fail over clients from a self-managed cluster to Confluent Cloud. For more information, see Migrate between Kafka Clusters using Confluent Cloud Gateway.

To configure the Enterprise license mode:

  1. Contact Confluent Support to obtain one or more valid Confluent Cloud Gateway license keys for production use.

  2. Configure the GATEWAY_LICENSES environment variable using the license keys you obtained in the previous step.

    • Option A: Set the environment variable before starting the Confluent Gateway.

      export GATEWAY_LICENSES="<your-license-key>"
      

      To add multiple license keys, use the following syntax using the dollar sign ($) to escape the newline character (\n).

      export GATEWAY_LICENSES=$'<your-first-license-key>\n<your-second-license-key>'
      
    • Option B: Edit docker-compose.yaml. You can add multiple license keys, one per line.

      gateway:
        environment:
          GATEWAY_LICENSES: |
            <your-first-license-key>
            <your-second-license-key>
      
  3. To verify the license status, inspect the Confluent Gateway logs for license information after starting the Confluent Gateway.

    docker logs gateway | grep -i license
    

For example license configurations, see Confluent Gateway examples in GitHub.

After completing your configuration, save the file as docker-compose.yaml. To deploy Confluent Gateway, see Install Confluent Gateway using Docker.

Install Confluent Gateway using Docker

Deploy Confluent Gateway using the configuration file (docker-compose.yaml by default) you set up in the previous section, Configure Confluent Gateway using Docker Compose.

  1. Pull the Confluent Gateway Docker image:

    docker pull confluentinc/confluent-gateway-for-cloud:<version-tag>
    

    Replace <version-tag> with the Confluent Gateway version you want to install. For example, use 1.2.0 to install version 1.2.0.

  2. Start Confluent Gateway:

    docker compose -f docker-compose.yaml up -d