Configure and Deploy Confluent Private Cloud Gateway

This guide provides instructions for configuring and deploying Confluent Private Cloud Gateway using Docker.

Confluent Private Cloud Gateway runs as a stateless service, often as a set of containerized instances within the target environment (for example, Confluent for Kubernetes, VM clusters, or cloud-native platforms).

This topic covers the following tasks using Docker.

A separate guide is provided for the following security-related tasks. See Configure Security for Confluent Private Cloud Gateway:

  • Configure Secret Stores

  • Configure SSL Configuration

  • Configure Password Configuration

  • Configure Authentication Swapping

Confluent Gateway supports deployment on-premises, in private cloud VPCs, or in hybrid environments.

The following are the high-level steps to deploy Confluent Gateway:

  1. Configure Confluent Gateway to connect to Kafka clusters, specifying authentication, routing policies, and security policies according to your organizational needs.

    Additionally, Configure security for the Confluent Gateway.

  2. Provision the Confluent Gateway service using the configuration settings from the previous step.

  3. Reconfigure clients to communicate with the Confluent Gateway endpoint.

    This simplifies credential management and strengthens overall security posture.

Requirements and considerations

Before you begin, make sure that the following requirements and considerations are satisfied:

  • System requirements

    • CPU and Memory

      • Minimum: 2 vCPUs, 4 GB RAM

      • Recommended: 4 vCPUs, 8 GB RAM

    • Network throughput: 45 MB/sec of sustained workload for a 1 Gigabit Link (1Gbps)

    • Storage: 10 GB of disk space

  • Docker

    For Docker-based deployments, Docker Desktop or Docker Engine with Compose v2 is installed (Docker Engine 20.x or later is recommended).

  • CFK and Confluent Platform

    For Confluent for Kubernetes (CFK)-based deployments, ensure your Kubernetes environment is set up and CFK and Confluent Platform are installed.

    Use the appropriate version of CFK that supports your Confluent Platform and Confluent Gateway requirements.

  • Network Access

    The Confluent Gateway container must have access to necessary Kafka clusters. Configure firewall ports for bidirectional traffic as needed.

Configure Confluent Gateway using Docker Compose

Provide the following configuration settings in a Docker Compose YAML file. docker-compose.yaml is used as an example in this guide.

The top-level layout for the Confluent Gateway configuration file is as follows:

gateway:
  image:                  --- [1]
  name:                   --- [2]
  streamingDomains: []    --- [3]
  secretStores: []        --- [4]
  routes: []              --- [5]
  admin: {}               --- [6]
  advanced: {}            --- [7]
  • [1] The Gateway Docker image in the following pattern: confluentinc/cpc-gateway:<version-tag>. <version-tag> is the Confluent Gateway version.

  • [2] The Gateway instance identifier.

  • [3] A list of Streaming Domains that represent Kafka clusters.

  • [4] A list of credential stores. Required for authentication swapping. See Secret store configuration section for details.

  • [5] A list of client routing rules.

  • [6] Admin and metrics configuration.

  • [7] Advanced settings.

Streaming Domains configuration

Streaming Domains are logical representations of your Kafka clusters at the Confluent Gateway.

gateway:
  streamingDomains:
    - name:               --- [1]
      type:               --- [2]
      kafkaCluster:       --- [3]
        name:             --- [4]
        bootstrapServers: --- [5]
          - id:           --- [6]
            endpoint:     --- [7]
            ssl:          --- [8]
        nodeIdRanges:     --- [9]
          - name:         --- [10]
            start:        --- [11]
            end:          --- [12]
  • [1] A unique name for the streaming domain.

  • [2] The type of the streaming domain. Set to kafka by default.

  • [3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or Apache Kafka®, that the Confluent Gateway can route traffic to.

  • [4] The name of the Kafka cluster.

  • [5] A list of Kafka bootstrap servers in the format host:port. At minimum, one bootstrap server is required per Kafka cluster. For each listener, one can define one bootstrap server.

  • [6] A unique identifier for the bootstrap-server. Recommended to use the protocol, such as SASL_PLAINTEXT, SSL or PLAINTEXT_SASL_PLAIN for clear distinction between both TLS channel type and SASL mechanism.

  • [7] The bootstrap endpoint of the Kafka broker in the format, protocol://host:port

  • [8] Only required for SSL/SASL_SSL configuration.

    See the SSL configuration section for details.

  • [9] The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.

  • [10] The name of the node ID range.

  • [11] The start of the node ID range, inclusive.

  • [12] The end of the node ID range, inclusive.

The following is an example configuration for a Confluent Gateway Streaming Domain:

streamingDomains:
  - name: sales # unique across gateway
    type: kafka # default: kafka
    kafkaCluster:
      name: sales-cluster # default: <streamingDomain.name>
      bootstrapServers:
        - id: PLAINTEXT-1
          endpoint: kafka0.example.com:9092
        - id: SASL_SSL-1
          endpoint: kafka0.example.com:9093
          ssl:  # required when using SASL_SSL/SSL
            ignoreTrust: false
            truststore:
              type: PKCS12  # default: JKS
              location: /opt/ssl/client-truststore.p12
              password:
                file: /opt/secrets/client-truststore.password
            keystore: # gateway's identity when needed
              type: PKCS12 # default: JKS
              location: /opt/ssl/gw-keystore.p12
              password:
                file: /opt/secrets/gw-keystore.password
              keyPassword:
                value: inline-password
      nodeIdRanges:
        - name: default
          start: 0              # inclusive
          end: 3                # inclusive
  • password can be a file path or an inline value.

  • ignoreTrust is used to skip certificate validation (not recommended for production). Other settings will be ignored if ignoreTrust is true

Routes configuration

Routes are Confluent Private Cloud Gateway endpoints where client applications connect to stream data.

Confluent Gateway uses Routes to define how client applications connect to Kafka clusters. Clients connect to the Gateway as if it were a Kafka cluster, while the Gateway handles routing and governance.

gateway:
  routes:
    - name:                         --- [1]
      endpoint:                     --- [2]
      brokerIdentificationStrategy: --- [3]
        type:                       --- [4]
        pattern:                    --- [5]
      streamingDomain:              --- [6]
        name:                       --- [7]
        bootstrapServerId:          --- [8]
      security:                     --- [9]
  • [1] The unique name for the route.

  • [2] The host:port combination that Confluent Gateway will listen on. This is the external address clients use to bootstrap to the Kafka cluster.

  • [3] Specifies the strategy for mapping client requests to a specific Kafka broker.

  • [4] The type of broker identification strategy. Set to port (default) or host.

    • port strategy: Each Kafka broker is identified using a unique port number. This is the default strategy.

      Clients connect to different ports to reach specific brokers (for example, port 9092 to connect with broker-0, port 9093 to connect with broker-1).

      The nodeIdRanges for the Streaming Domain you set in Streaming Domains configuration is used.

      nodeIdRanges should be present in all of the clusters associated with Route’s Streaming Domain.

    • host strategy: Each Kafka broker is represented using a unique hostname.

      Clients use different host names to reach specific brokers (for example, broker-0.kafka.company.com, broker-1.kafka.company.com), and the gateway routes based on the SNI header.

      The pattern setting ([5]) is used.

  • [5] The pattern for the broker identification strategy. Required if the type ([4]) is host. For example, broker-$(nodeId).eu-gw.sales.example.com:9092.

  • [6] The reference to a Streaming Domain.

  • [7] The name of the Streaming Domain. Must be a valid name from the gateway.streamingDomains[].name.

  • [8] The bootstrap server ID. Must match kafkaCluster.bootstrapServers[].id.

  • [9] The security configuration. See Configure Security for Confluent Private Cloud Gateway section for details.

The following is an example configuration for Confluent Gateway Routes:

routes:
  - name: eu-sales
    endpoint: eu-gw.sales.example.com:9092
    brokerIdentificationStrategy:
      type: host
      pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
    streamingDomain:
      name: sales
      bootstrapServerId: SASL_SSL-1

Admin and Metrics configuration

gateway:
  admin:
    bindAddress:             --- [1]
    port:                    --- [2]
    endpoints:
      metrics:               --- [3]
    jvmMetrics:              --- [4]
      - JvmGcMetrics
      - JvmMemoryMetrics
      - JvmThreadMetrics
      - ProcessorMetrics
      - UptimeMetrics
    commonTags:              --- [5]
      host:
      region:
  • [1] The local address to bind admin service and to listen on for incoming connections. The default value is 0.0.0.0.

    This address determines which network interfaces the gateway will use to expose its administrative endpoints, such as metrics and liveness checks.

  • [2] The TCP port for exposing admin endpoints. The default value is 9190.

    The Confluent Gateway will serve metrics at http://{bindAddress}:{port}/metrics and liveness check at http://{bindAddress}:{port}/livez.

  • [3] The metrics endpoints to be enabled and exposed. The default value is true.

  • [4] The standard JVM Metrics (uses Micrometer classes). By default JvmGcMetrics, JvmMemoryMetrics, JvmThreadMetrics, ProcessorMetrics, UptimeMetrics are enabled.

  • [5] Optional. The common tags to be applied to all the metrics.

The following is an example configuration for a Confluent Gateway Admin and Metrics:

admin:
  bindAddress: 0.0.0.0
  port: 9190
  endpoints:
    metrics: true
  jvmMetrics:
    - JvmGcMetrics
    - JvmMemoryMetrics
    - JvmThreadMetrics
    - ProcessorMetrics
    - UptimeMetrics
  commonTags:
    host: pod-0
    region: us-west-2

Install Confluent Gateway using Docker

Deploy Confluent Gateway using the configuration file (docker-compose.yaml by default) you set up in the previous section, Configure Confluent Gateway using Docker Compose.

docker compose -f docker-compose.yaml up -d