Configure and Deploy Confluent Private Cloud Gateway using Docker
Configure and deploy Confluent Private Cloud Gateway (Confluent Gateway) using Docker. You can use this deployment path to run Confluent Gateway on-premises, in private cloud VPCs, and across hybrid environments.
For general requirements and considerations, see Requirements and considerations. For Docker-specific requirements, see Docker software requirements.
The high-level steps to deploy Confluent Gateway using Docker are:
Configure Confluent Gateway by defining streaming domains, routes, and administration settings in a Docker Compose file.
Deploy Confluent Gateway by pulling the Confluent Gateway Docker image and starting the service with Docker Compose.
After deployment, configure security and reconfigure clients to use the Confluent Gateway endpoint. For more information, see Next steps.
Configure Confluent Gateway using Docker Compose
Provide the configuration settings in a Docker Compose YAML file. This guide uses docker-compose.yaml as an example.
The top-level layout for the Confluent Gateway configuration file is as follows:
gateway:
image: --- [1]
name: --- [2]
streamingDomains: [] --- [3]
secretStores: [] --- [4]
routes: [] --- [5]
admin: {} --- [6]
[1] The Confluent Gateway Docker image in the pattern:
confluentinc/cpc-gateway:<version_tag>.<version_tag>is the Confluent Gateway version.[2] The Confluent Gateway instance identifier.
[3] A list of streaming domains that represent Apache Kafka® clusters. For more information, see Streaming domains configuration.
[4] A list of secret stores. Required for authentication swapping. See Secret store configuration for details.
[5] A list of client routing rules. For more information, see Routes configuration.
[6] Admin and metrics configuration. For more information, see Administration and metrics configuration.
Streaming domains configuration
Streaming domains are logical representations of your Kafka clusters in Confluent Gateway.
gateway:
streamingDomains:
- name: --- [1]
type: --- [2]
kafkaCluster: --- [3]
name: --- [4]
bootstrapServers: --- [5]
- id: --- [6]
endpoint: --- [7]
ssl: --- [8]
nodeIdRanges: --- [9]
- name: --- [10]
start: --- [11]
end: --- [12]
[1] A unique name for the streaming domain.
[2] The type of the streaming domain. Set to
kafkaby default.[3] A Kafka cluster, including Confluent Server, Confluent Private Cloud Kafka, or Apache Kafka®, that the Confluent Gateway can route traffic to.
[4] The name of the Kafka cluster.
[5] A list of Kafka bootstrap servers in the format
host:port. At minimum, one bootstrap server is required per Kafka cluster. For each listener, you can define one bootstrap server.[6] A unique identifier for the bootstrap server. Use the protocol name (such as
SASL_PLAINTEXT,SSL, orPLAINTEXT_SASL_PLAIN) for clear distinction between the TLS channel type and SASL mechanism.[7] The bootstrap endpoint of the Kafka broker in the format:
host:port.[8] Only required for SSL/SASL_SSL configuration. Supported certificate types are
JKS,PKCS12, andPEM.See the SSL configuration section for details.
[9] The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.
[10] The name of the node ID range.
[11] The start of the node ID range, inclusive.
[12] The end of the node ID range, inclusive.
An example configuration for a Confluent Gateway streaming domain:
streamingDomains:
- name: sales # unique across gateway
type: kafka # default: kafka
kafkaCluster:
name: sales-cluster # default: <streamingDomain.name>
bootstrapServers:
- id: PLAINTEXT-1
endpoint: kafka0.example.com:9092
- id: SASL_SSL-1
endpoint: kafka0.example.com:9093
ssl: # required when using SASL_SSL/SSL
ignoreTrust: false
truststore:
type: PKCS12 # default: JKS
location: /opt/ssl/client-truststore.p12
password:
file: /opt/secrets/client-truststore.password
keystore: # gateway's identity when needed
type: PKCS12 # default: JKS
location: /opt/ssl/gw-keystore.p12
password:
file: /opt/secrets/gw-keystore.password
keyPassword:
value: inline-password
nodeIdRanges:
- name: default
start: 0 # inclusive
end: 3 # inclusive
passwordcan be a file path or an inline value. For details, see Password configuration.ignoreTrustskips certificate validation. Not recommended for production. Whentrue, Confluent Gateway ignores all other SSL settings.
Routes configuration
Routes are Confluent Gateway endpoints where client applications connect to stream data.
Confluent Gateway uses routes to define how client applications connect to Kafka clusters. Clients connect to Confluent Gateway as if it were a Kafka cluster, while Confluent Gateway handles routing and governance.
gateway:
routes:
- name: --- [1]
endpoint: --- [2]
brokerIdentificationStrategy: --- [3]
type: --- [4]
pattern: --- [5]
streamingDomain: --- [6]
name: --- [7]
bootstrapServerId: --- [8]
security: --- [9]
fence: --- [10]
[1] The unique name for the route.
[2] The
host:portcombination that Confluent Gateway listens on. This is the external address clients use to bootstrap to the Kafka cluster.[3] Specifies the strategy for mapping client requests to a specific Kafka broker.
[4] The type of broker identification strategy. Set to
port(default) orhost.portstrategy: Each Kafka broker is identified using a unique port number. This is the default strategy.Clients connect to different ports to reach specific brokers (for example, port 9092 to connect with broker-0, port 9093 to connect with broker-1).
The
nodeIdRangesfrom the streaming domain configured in Streaming domains configuration is used. ThenodeIdRangesmust be present in the streaming domain associated with the route.hoststrategy: Each Kafka broker is represented using a unique hostname.Clients use different host names to reach specific brokers (for example,
broker-0.kafka.company.com,broker-1.kafka.company.com), and Confluent Gateway routes based on the Server Name Indication (SNI) header.The
patternsetting ([5]) is used.
[5] The pattern for the broker identification strategy. Required if the type ([4]) is
host. For example,broker-$(nodeId).eu-gw.sales.example.com:9092.[6] The reference to a streaming domain.
[7] The name of the streaming domain. Must be a valid name from the
gateway.streamingDomains[].name.[8] The bootstrap server ID. Must match
kafkaCluster.bootstrapServers[].id.[9] The security configuration. For more information, see Configure Security for Confluent Private Cloud Gateway.
[10] Optional. The fencing filter configuration. See Fencing filter for details.
An example configuration for Confluent Gateway routes:
gateway:
routes:
- name: eu-sales
endpoint: eu-gw.sales.example.com:9092
brokerIdentificationStrategy:
type: host
pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
streamingDomain:
name: sales
bootstrapServerId: SASL_SSL-1
Route filters configuration
Filters allow you to control and manage traffic flow for specific routes in Confluent Gateway. You can apply filters at the route level to enforce policies, block or allow requests, and customize error handling for different scenarios.
Fencing filter
The fencing filter in Confluent Gateway is a route-level filter that allows you to control the traffic flow on a specific route. It supports two fencing scopes:
ALL: Blocks all requests on the specified route. Useful for scenarios such as maintenance windows or when you want to temporarily disable access to a route.NONE: Allows all requests on the specified route.
gateway:
routes:
- name:
fence:
scope: --- [1]
errorCode: --- [2]
errorMessage: --- [3]
[1] The fencing scope of the filter. The default value is
NONE. Set toALLto block all requests on the route.[2] Optional. The error code to return for blocked requests. The default value is
BROKER_NOT_AVAILABLE.For a list of valid error codes, see Kafka protocol error codes.
[3] Optional. The error message to return for blocked requests. The default value is
This route is currently unavailable - all requests are blocked.
The following is an example configuration for a Confluent Gateway route with the fencing filter enabled:
gateway:
routes:
- name: eu-sales
endpoint: eu-gw.sales.example.com:9092
fence:
scope: ALL
errorCode: BROKER_NOT_AVAILABLE
errorMessage: "Maintenance in progress - route temporarily unavailable"
brokerIdentificationStrategy:
type: host
pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
streamingDomain:
name: sales
bootstrapServerId: SASL_SSL-1
Administration and metrics configuration
The admin section configures the Confluent Gateway administrative service, including the bind address, TCP port, metrics endpoint, JVM metrics collection, and custom metric tags.
gateway:
admin:
bindAddress: --- [1]
port: --- [2]
endpoints:
metrics: --- [3]
jvmMetrics: --- [4]
- JvmGcMetrics
- JvmMemoryMetrics
- JvmThreadMetrics
- ProcessorMetrics
- UptimeMetrics
commonTags: --- [5]
host:
region:
[1] The local address to bind admin service and to listen on for incoming connections. The default value is
0.0.0.0.This address determines which network interfaces the Confluent Gateway uses to expose its administrative endpoints, such as metrics and liveness checks.
[2] The TCP port for exposing admin endpoints. The default value is
9190.The Confluent Gateway serves metrics at
http://{bindAddress}:{port}/metrics. It exposes a liveness check athttp://{bindAddress}:{port}/livez.[3] The metrics endpoints to be enabled and exposed. The default value is
true.[4] Standard JVM metrics captured using Micrometer classes. By default,
JvmGcMetrics,JvmMemoryMetrics,JvmThreadMetrics,ProcessorMetrics, andUptimeMetricsare enabled.[5] Optional. The common tags to be applied to all the metrics.
An example configuration for Confluent Gateway administration and metrics:
admin:
bindAddress: 0.0.0.0
port: 9190
endpoints:
metrics: true
jvmMetrics:
- JvmGcMetrics
- JvmMemoryMetrics
- JvmThreadMetrics
- ProcessorMetrics
- UptimeMetrics
commonTags:
host: pod-0
region: us-west-2
Enterprise license configuration (optional)
Enterprise mode enables unlimited routes and requires a valid Confluent Private Cloud Enterprise license key. Trial mode (default, no key required) is limited to a maximum of four routes. For a full comparison of modes, see Deploy Confluent Private Cloud Gateway.
If you’re using Enterprise mode, follow these steps to apply your license key:
Contact Confluent to get one or more valid license keys for production use.
Configure the
GATEWAY_LICENSESenvironment variable using the license key you obtained in the previous step.Option A: Set the environment variable before starting the Confluent Gateway.
export GATEWAY_LICENSES="your-license-key"
To add multiple license keys, use the following syntax using the dollar sign (
$) to escape the newline character (\n).export GATEWAY_LICENSES=$'your-first-license-key\nyour-second-license-key'
Option B: Edit
docker-compose.yaml. You can add multiple license keys, one per line.gateway: environment: GATEWAY_LICENSES: | your-first-license-key your-second-license-key
To verify the license status, check the Confluent Gateway logs for license information after starting the Confluent Gateway.
docker logs gateway | grep -i license
For example license configurations, see Confluent Gateway examples in GitHub.
After completing your configuration, save the file as docker-compose.yaml.
Deploy Confluent Gateway using Docker
To deploy Confluent Gateway:
Pull the Confluent Gateway Docker image to ensure you have the correct version and avoid using a stale locally cached image. For available version tags, see Confluent Gateway on GitHub.
docker pull confluentinc/cpc-gateway:<version_tag>
For example, to pull version 1.2.0:
docker pull confluentinc/cpc-gateway:1.2.0
Start Confluent Gateway using the
docker-compose.yamlconfiguration file you set up in Configure Confluent Gateway using Docker Compose.docker compose -f docker-compose.yaml up -d
Verify that Confluent Gateway is running.
docker psConfluent Gateway starts in the background after the
docker composecommand completes. Confirm the Confluent Gateway container appears in the output.
Next steps
After you deploy Confluent Gateway:
Verify the deployment and resolve any startup or connectivity problems. See Troubleshoot Confluent Gateway Docker Deployments for diagnostic steps and symptom-based resolutions.
Configure security for the Confluent Gateway. Security configuration covers authentication, SSL/TLS, secret stores, and passwords for client-to-Confluent Gateway and Confluent Gateway-to-Kafka cluster connections.
Reconfigure clients to communicate with the Confluent Gateway endpoint by pointing their
bootstrap.serversto the route endpoint defined in the Confluent Gateway configuration. For migration scenarios between Kafka clusters, see Migrate between Kafka Clusters using Confluent Private Cloud Gateway.