Configure Node Ports to Access Confluent Platform Components Using Confluent for Kubernetes¶
When you configure Confluent components with the NodePort service for external client access, Kubernetes allocates a different port on each node of Confluent components. Each Kafka broker, the bootstrap service, and each component is accessible on a separate port.
Configure external access to Kafka using node ports¶
For a Kafka cluster with N brokers, (N+1) number of NodePort services are created:
- One for the bootstrap server for the initial connection
- N services, one for each broker, for subsequent direct connections to the brokers
Create a DNS record using the address of one or more of the nodes in your Kubernetes cluster.
To configure Kafka with node ports, set the following in the Kafka custom resource (CR) and apply the configuration with the
kubectl apply -f
command:spec: listeners: external: externalAccess: type: nodePort nodePort: nodePortOffset: --- [1] host: --- [2]
[1] Required. The value should be be in the range between 30000 and 32767, inclusive.
If you change this value on a running cluster, you must roll the cluster.
[2] Required. Specify the FQDN that will be used to configure all advertised listeners.
If you change this value on a running cluster, you must roll the cluster.
Create firewall rules to allow connections at the NodePort range that you plan to use. For the steps to create firewall rules, see Using Google Cloud firewall rules.
Verify the NodePort services are correctly created by listing the services in the namespace using the following command:
kubectl get services -n <namespace> | grep NodePort
For a tutorial scenario on configuring external access using NodePort, see the quickstart tutorial for using node port.
Configure external access to MDS using node ports¶
When you set up external access to MDS with role-based access control (RBAC) enabled, additional networking configuration steps required.
For a RBAC-enabled Kafka cluster with N brokers, if you enable external access to MDS, (N+2) number of NodePort services are created:
- One for the bootstrap server for the initial connection
- One for the MDS on the bootstrap server
- N services, one for each broker, for subsequent direct connections to the brokers
To support external access to Kafka Metadata Service (MDS) using node ports, configure the
following in the Kafka custom resource (CR), and apply the configuration using
the kubectl apply -f
command:
spec
services
mds:
externalAccess:
type: nodePort
nodePort:
externalTrafficPolicy: --- [1]
host: --- [2]
nodePortOffset: --- [3]
advertisedURL: --- [4]
[1] Specifies external traffic policy for the MDS service. Set to
Cluster
orLocal
.[2] Host name of the MDS.
[3] Port number of the MDS. The value should be between
30000
and32767
, inclusive.[4] If set, instead of using the internal endpoint, the MDS advertised listener for each broker will be set to:
<httpSchema>://<host>:<nodePortOffset + podId + 1>
where<podId>
ranges from0
toreplicaCount -1
.Use this property only if you cannot add internal SANs to the TLS certificates for MDS, and the external DNS must be resolved inside the Kubernetes cluster.
The endpoint to externally access MDS using node port is
http(s)://<host>:<nodePortOffset>
.
For example:
spec
services
mds:
externalAccess:
type: nodePort
nodePort:
externalTrafficPolicy: Cluster
host: example.com
nodePortOffset: 30001
Configure external access to other Confluent Platform components using node ports¶
To configure other Confluent components with node ports:
Set the following in the component CRs and apply the configuration using the
kubectl apply -f
command:spec: externalAccess: type: nodePort nodePort: nodePortOffset: --- [1] host: --- [2] sessionAffinity: --- [3] sessionAffinityConfig: --- [4] clientIP: timeoutSeconds: --- [5] configOverrides: server: - advertised.listeners= --- [6]
The access endpoint of each Confluent Platform component will be:
<host>:<nodePortOffset>
[1] Required. The value should be in the range between 30000 and 32767, inclusive.
If you change this value on a running cluster, you must roll the cluster.
[2] Required. Specify the FQDN that will be used to configure all advertised listeners.
If you change this value on a running cluster, you must roll the cluster.
[3] Required for consumer REST Proxy to enable client IP-based session affinity.
For REST Proxy to be used for Kafka consumers, set to
ClientIP
. See Kubernetes Service for more information about session affinity.[4] Contains the configurations of session affinity if set
sessionAffinity: ClientIP
in [3].[5] Specifies the seconds of
ClientIP
type session sticky time. The value must be bigger than0
and less than or equal to86400
(1 day).Default value is
10800
(3 hours).[6] Set to the external DNS name used for node port. This configuration is used to generate absolute URLs in V3 responses. The HTTP and HTTPS protocols are supported.
Create firewall rules to allow connections at the NodePort range that you plan to use. For the steps to create firewall rules, see Using Google Cloud firewall rules.
Verify the NodePort services are correctly created by listing the services in the namespace using the following command:
kubectl get services -n <namespace> | grep NodePort
For a tutorial scenario on configuring external access using NodePort, see the quickstart tutorial for using node port.