Configure Port-Based Static Access
Port-based static access configures Apache Kafka® advertised listeners using a single host and a distinct port for each broker, without provisioning per-broker Kubernetes load balancers or node ports.
Set up Kafka for port-based static access to configure the advertised listeners with the same host and different ports.
This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using an NGINX Ingress controller.
To configure static external access to Kafka using port-based routing:
Deploy Kafka with the
staticForPortBasedRoutingexternal access type:listeners: external: externalAccess: type: staticForPortBasedRouting staticForPortBasedRouting: portOffset: --- [1] host: --- [2]
[1] Required. The starting port number.
If you change this value on a running cluster, you must roll the cluster.
[2] Required. The fully qualified domain name (FQDN) used to configure all advertised listeners.
If you change this value on a running cluster, you must roll the cluster.
For example, the following are Kafka advertised listeners for three Kafka brokers with
portOffsetset to9093andhostset totest.example.com:test.example.com:9093test.example.com:9094test.example.com:9095
Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.
Specify the mappings between the TCP port and the Kafka service as shown in the following example command. Map each broker to the TCP port based on the
portOffsetyou set in Step #1. Use the following command to see the KafkaClusterIPservices and the ports:kubectl get services -n <namespace>
The following example is a Helm command to install an NGINX Ingress controller, mapping the ports, 9093, 9094, and 9095, to Kafka services and service ports (three brokers).
helm upgrade --install ingress-nginx ingress-nginx \ --repo https://kubernetes.github.io/ingress-nginx \ --namespace <namespace> \ --set controller.ingressClass=kafka \ --set tcp.9093="operator/kafka-0-internal:9092" \ --set tcp.9094="operator/kafka-1-internal:9092" \ --set tcp.9095="operator/kafka-2-internal:9092"
Verify that the Ingress controller is correctly configured. Refer to your specific Ingress controller documentation for details.
For NGINX, run the following commands to get the ConfigMap name and to verify the ConfigMap that the Ingress controller created:
kubectl get configmap -n <namespace> kubectl describe configmap <configmap_name> -n <namespace>
The output should include:
The namespace name
The Kafka broker service name
The port, as in the Helm command used to install the Ingress controller in Step 2
Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.
Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported. For detail on deploying the NGINX controller and configuring an Ingress resource, see the tutorial for deploying NGINX Ingress on GKE.
The following example is to create an Ingress resource for an NGINX Ingress controller to expose three Kafka brokers:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-without-sni namespace: <namespace> annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: test.example.com http: paths: - path: / pathType: Prefix backend: service: name: kafka-0-internal port: number: 9092 - path: / pathType: Prefix backend: service: name: kafka-1-internal port: number: 9092 - path: / pathType: Prefix backend: service: name: kafka-2-internal port: number: 9092
Create a DNS record with the host name you provided in Step #1 for Kafka and the external load balancer IP of the Ingress controller. The host name should resolve to the
externalIPof the Ingress controller load balancer.You can retrieve the external IP using the following command:
kubectl get services -n <namespace>
For a tutorial scenario on configuring external access using port-based static access, see the quickstart tutorial for port-based static access.