Configure Node Ports
A NodePort service exposes each Apache Kafka® broker, the bootstrap service, and other Confluent Platform components on a separate port on every Kubernetes node for external client access. For more information, see NodePort service.
This is one of the external access methods described in Configure Networking. For load balancer-based access, see Configure Load Balancers.
Configure external access to Kafka using node ports
For a Kafka cluster with N brokers, CFK creates (N+1) NodePort services:
One for the bootstrap server for the initial connection
N services, one for each broker, for subsequent direct connections to the brokers
Create a DNS record that maps the FQDN you set as
host(see Step 2, item [2]) to the address of one or more nodes in your Kubernetes cluster.To configure Kafka with node ports, set the following in the Kafka custom resource (CR) and apply the configuration with the
kubectl apply -fcommand:spec: listeners: external: externalAccess: type: nodePort nodePort: nodePortOffset: --- [1] host: --- [2] publishNotReadyAddresses: --- [3]
[1] Required. The value should be in the range between
30000and32767, inclusive.If you change this value on a running cluster, you must roll the cluster.
[2] Required. Specify the fully qualified domain name (FQDN) used to configure all advertised listeners.
If you change this value on a running cluster, you must roll the cluster.
[3] Optional. Set
publishNotReadyAddressestotrueso that the per-broker node port service keeps routing to a broker while not yet ready, for example, during a rolling restart. The default isfalse.This can help prevent partitions from going offline when you roll many brokers concurrently. For the replication listener, this value is always
true, regardless of this setting.
Create firewall rules to allow connections at the NodePort range that you plan to use. For the steps to create firewall rules, see Using Google Cloud firewall rules.
Verify the NodePort services are correctly created by listing the services in the namespace using the following command:
kubectl get services -n <namespace> | grep NodePort
For a tutorial scenario on configuring external access using node port, see the quickstart tutorial for using node port.
Configure external access to MDS using node ports
When you set up external access to Metadata Service (MDS) with role-based access control (RBAC) enabled, you need additional networking configuration steps.
To support external access to Kafka MDS using node ports, configure the
following in the Kafka custom resource (CR), and apply the configuration using
the kubectl apply -f command:
spec:
services:
mds:
externalAccess:
type: nodePort
nodePort:
externalTrafficPolicy: --- [1]
host: --- [2]
nodePortOffset: --- [3]
advertisedURL:
enabled: --- [4]
[1] Specifies external traffic policy for the MDS service. Set to
ClusterorLocal.[2] Host name of the MDS.
[3] Port number of the MDS. The value should be between
30000and32767, inclusive.[4] If set, instead of using the internal endpoint, CFK sets the MDS advertised listener for each broker to:
<httpSchema>://<host>:<nodePortOffset + podId + 1>where<podId>ranges from0toreplicaCount -1.Use this property only if you cannot add internal Subject Alternative Names (SANs) to the TLS certificates for MDS, and the external DNS must be resolved inside the Kubernetes cluster.
The endpoint to externally access MDS using node port is
http(s)://<host>:<nodePortOffset>.
CFK creates (N+2) NodePort services when you enable external access to MDS on an RBAC-enabled Kafka cluster with N brokers:
One for the bootstrap server for the initial connection
One for the MDS on the bootstrap server
N services, one for each broker, for subsequent direct connections to the brokers
For example:
spec:
services:
mds:
externalAccess:
type: nodePort
nodePort:
externalTrafficPolicy: Cluster
host: example.com
nodePortOffset: 30001
Configure external access to other Confluent Platform components using node ports
To configure other Confluent components with node ports:
Set the following in the component custom resources (CRs) and apply the configuration using the
kubectl apply -fcommand:spec: externalAccess: type: nodePort nodePort: nodePortOffset: --- [1] host: --- [2] sessionAffinity: --- [3] sessionAffinityConfig: --- [4] clientIP: timeoutSeconds: --- [5] publishNotReadyAddresses: --- [6] configOverrides: server: - advertised.listeners= --- [7]
The access endpoint of each Confluent Platform component is:
<host>:<nodePortOffset>[1] Required. The value should be in the range between
30000and32767, inclusive.If you change this value on a running cluster, you must roll the cluster.
[2] Required. Specify the FQDN used to configure all advertised listeners.
If you change this value on a running cluster, you must roll the cluster.
[3] Required for consumer REST Proxy to enable client IP-based session affinity.
For REST Proxy to be used for Kafka consumers, set to
ClientIP. See Kubernetes Service for more information about session affinity.[4] Contains the configurations of session affinity if you set
sessionAffinity: ClientIPin [3].[5] Specifies the session timeout, in seconds, for
ClientIPsession affinity. The value must be greater than0and less than or equal to86400(1 day).Default value is
10800(3 hours).[6] Optional. Set
publishNotReadyAddressestotrueso that the per-replica node port service keeps routing to a pod while not yet ready, for example, during a rolling restart. The default isfalse.[7] Set to the external DNS name used for node port. This configuration is used to generate absolute URLs in REST Proxy v3 responses. The HTTP and HTTPS protocols are supported.
Create firewall rules to allow connections at the NodePort range that you plan to use. For the steps to create firewall rules, see Using Google Cloud firewall rules.
Verify the NodePort services are correctly created by listing the services in the namespace using the following command:
kubectl get services -n <namespace> | grep NodePort
For a tutorial scenario on configuring external access using node port, see the quickstart tutorial for using node port.