Configure and Deploy Confluent Private Cloud Gateway
This guide provides instructions for configuring and deploying Confluent Private Cloud Gateway using Docker.
Confluent Private Cloud Gateway runs as a stateless service, often as a set of containerized instances within the target environment (for example, Confluent for Kubernetes, VM clusters, or cloud-native platforms).
This topic covers the following tasks using Docker.
A separate guide is provided for the following security-related tasks. See Configure Security for Confluent Private Cloud Gateway:
Configure Secret Stores
Configure SSL Configuration
Configure Password Configuration
Configure Authentication Swapping
Confluent Gateway supports deployment on-premises, in private cloud VPCs, or in hybrid environments.
The following are the high-level steps to deploy Confluent Gateway:
Configure Confluent Gateway to connect to Kafka clusters, specifying authentication, routing policies, and security policies according to your organizational needs.
Additionally, Configure security for the Confluent Gateway.
Provision the Confluent Gateway service using the configuration settings from the previous step.
Reconfigure clients to communicate with the Confluent Gateway endpoint.
This simplifies credential management and strengthens overall security posture.
Requirements and considerations
Before you begin, make sure that the following requirements and considerations are satisfied:
System requirements
CPU and Memory
Minimum: 2 vCPUs, 4 GB RAM
Recommended: 4 vCPUs, 8 GB RAM
Network throughput: 45 MB/sec of sustained workload for a 1 Gigabit Link (1Gbps)
Storage: 10 GB of disk space
Docker
For Docker-based deployments, Docker Desktop or Docker Engine with Compose v2 is installed (Docker Engine 20.x or later is recommended).
CFK and Confluent Platform
For Confluent for Kubernetes (CFK)-based deployments, ensure your Kubernetes environment is set up and CFK and Confluent Platform are installed.
Use the appropriate version of CFK that supports your Confluent Platform and Confluent Gateway requirements.
Network Access
The Confluent Gateway container must have access to necessary Kafka clusters. Configure firewall ports for bidirectional traffic as needed.
Configure Confluent Gateway using Docker Compose
Provide the following configuration settings in a Docker Compose YAML file. docker-compose.yaml
is used as an example in this guide.
The top-level layout for the Confluent Gateway configuration file is as follows:
gateway:
image: --- [1]
name: --- [2]
streamingDomains: [] --- [3]
secretStores: [] --- [4]
routes: [] --- [5]
admin: {} --- [6]
advanced: {} --- [7]
[1] The Gateway Docker image in the following pattern:
confluentinc/cpc-gateway:<version-tag>
.<version-tag>
is the Confluent Gateway version.[2] The Gateway instance identifier.
[3] A list of Streaming Domains that represent Kafka clusters.
[4] A list of credential stores. Required for authentication swapping. See Secret store configuration section for details.
[5] A list of client routing rules.
[6] Admin and metrics configuration.
[7] Advanced settings.
Streaming Domains configuration
Streaming Domains are logical representations of your Kafka clusters at the Confluent Gateway.
gateway:
streamingDomains:
- name: --- [1]
type: --- [2]
kafkaCluster: --- [3]
name: --- [4]
bootstrapServers: --- [5]
- id: --- [6]
endpoint: --- [7]
ssl: --- [8]
nodeIdRanges: --- [9]
- name: --- [10]
start: --- [11]
end: --- [12]
[1] A unique name for the streaming domain.
[2] The type of the streaming domain. Set to
kafka
by default.[3] A Kafka cluster, including Confluent Server, Confluent Cloud Kafka, or Apache Kafka®, that the Confluent Gateway can route traffic to.
[4] The name of the Kafka cluster.
[5] A list of Kafka bootstrap servers in the format
host:port
. At minimum, one bootstrap server is required per Kafka cluster. For each listener, one can define one bootstrap server.[6] A unique identifier for the bootstrap-server. Recommended to use the protocol, such as
SASL_PLAINTEXT
,SSL
orPLAINTEXT_SASL_PLAIN
for clear distinction between both TLS channel type and SASL mechanism.[7] The bootstrap endpoint of the Kafka broker in the format,
protocol://host:port
[8] Only required for SSL/SASL_SSL configuration.
See the SSL configuration section for details.
[9] The node ID ranges. Only required for port-based broker identification. These should be the broker IDs defined in the Kafka deployment.
[10] The name of the node ID range.
[11] The start of the node ID range, inclusive.
[12] The end of the node ID range, inclusive.
The following is an example configuration for a Confluent Gateway Streaming Domain:
streamingDomains:
- name: sales # unique across gateway
type: kafka # default: kafka
kafkaCluster:
name: sales-cluster # default: <streamingDomain.name>
bootstrapServers:
- id: PLAINTEXT-1
endpoint: kafka0.example.com:9092
- id: SASL_SSL-1
endpoint: kafka0.example.com:9093
ssl: # required when using SASL_SSL/SSL
ignoreTrust: false
truststore:
type: PKCS12 # default: JKS
location: /opt/ssl/client-truststore.p12
password:
file: /opt/secrets/client-truststore.password
keystore: # gateway's identity when needed
type: PKCS12 # default: JKS
location: /opt/ssl/gw-keystore.p12
password:
file: /opt/secrets/gw-keystore.password
keyPassword:
value: inline-password
nodeIdRanges:
- name: default
start: 0 # inclusive
end: 3 # inclusive
password
can be a file path or an inline value.ignoreTrust
is used to skip certificate validation (not recommended for production). Other settings will be ignored ifignoreTrust
is true
Routes configuration
Routes are Confluent Private Cloud Gateway endpoints where client applications connect to stream data.
Confluent Gateway uses Routes to define how client applications connect to Kafka clusters. Clients connect to the Gateway as if it were a Kafka cluster, while the Gateway handles routing and governance.
gateway:
routes:
- name: --- [1]
endpoint: --- [2]
brokerIdentificationStrategy: --- [3]
type: --- [4]
pattern: --- [5]
streamingDomain: --- [6]
name: --- [7]
bootstrapServerId: --- [8]
security: --- [9]
[1] The unique name for the route.
[2] The
host:port
combination that Confluent Gateway will listen on. This is the external address clients use to bootstrap to the Kafka cluster.[3] Specifies the strategy for mapping client requests to a specific Kafka broker.
[4] The type of broker identification strategy. Set to
port
(default) orhost
.port
strategy: Each Kafka broker is identified using a unique port number. This is the default strategy.Clients connect to different ports to reach specific brokers (for example, port 9092 to connect with broker-0, port 9093 to connect with broker-1).
The
nodeIdRanges
for the Streaming Domain you set in Streaming Domains configuration is used.nodeIdRanges
should be present in all of the clusters associated with Route’s Streaming Domain.host
strategy: Each Kafka broker is represented using a unique hostname.Clients use different host names to reach specific brokers (for example,
broker-0.kafka.company.com
,broker-1.kafka.company.com
), and the gateway routes based on the SNI header.The
pattern
setting ([5]) is used.
[5] The pattern for the broker identification strategy. Required if the type ([4]) is
host
. For example,broker-$(nodeId).eu-gw.sales.example.com:9092
.[6] The reference to a Streaming Domain.
[7] The name of the Streaming Domain. Must be a valid name from the
gateway.streamingDomains[].name
.[8] The bootstrap server ID. Must match
kafkaCluster.bootstrapServers[].id
.[9] The security configuration. See Configure Security for Confluent Private Cloud Gateway section for details.
The following is an example configuration for Confluent Gateway Routes:
routes:
- name: eu-sales
endpoint: eu-gw.sales.example.com:9092
brokerIdentificationStrategy:
type: host
pattern: broker-$(nodeId).eu-gw.sales.example.com:9092
streamingDomain:
name: sales
bootstrapServerId: SASL_SSL-1
Admin and Metrics configuration
gateway:
admin:
bindAddress: --- [1]
port: --- [2]
endpoints:
metrics: --- [3]
jvmMetrics: --- [4]
- JvmGcMetrics
- JvmMemoryMetrics
- JvmThreadMetrics
- ProcessorMetrics
- UptimeMetrics
commonTags: --- [5]
host:
region:
[1] The local address to bind admin service and to listen on for incoming connections. The default value is
0.0.0.0
.This address determines which network interfaces the gateway will use to expose its administrative endpoints, such as metrics and liveness checks.
[2] The TCP port for exposing admin endpoints. The default value is
9190
.The Confluent Gateway will serve metrics at
http://{bindAddress}:{port}/metrics
and liveness check athttp://{bindAddress}:{port}/livez
.[3] The metrics endpoints to be enabled and exposed. The default value is
true
.[4] The standard JVM Metrics (uses Micrometer classes). By default
JvmGcMetrics
,JvmMemoryMetrics
,JvmThreadMetrics
,ProcessorMetrics
,UptimeMetrics
are enabled.[5] Optional. The common tags to be applied to all the metrics.
The following is an example configuration for a Confluent Gateway Admin and Metrics:
admin:
bindAddress: 0.0.0.0
port: 9190
endpoints:
metrics: true
jvmMetrics:
- JvmGcMetrics
- JvmMemoryMetrics
- JvmThreadMetrics
- ProcessorMetrics
- UptimeMetrics
commonTags:
host: pod-0
region: us-west-2
Install Confluent Gateway using Docker
Deploy Confluent Gateway using the configuration file (docker-compose.yaml by default) you set up in the previous section, Configure Confluent Gateway using Docker Compose.
docker compose -f docker-compose.yaml up -d