Configure Port-Based Static Access

Port-based static access configures Apache Kafka® advertised listeners using a single host and a distinct port for each broker, without provisioning per-broker Kubernetes load balancers or node ports.

Set up Kafka for port-based static access to configure the advertised listeners with the same host and different ports.

This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using an NGINX Ingress controller.

To configure static external access to Kafka using port-based routing:

  1. Deploy Kafka with the staticForPortBasedRouting external access type:

    listeners:
      external:
        externalAccess:
          type: staticForPortBasedRouting
          staticForPortBasedRouting:
            portOffset:    --- [1]
            host:          --- [2]
    
    • [1] Required. The starting port number.

      If you change this value on a running cluster, you must roll the cluster.

    • [2] Required. The fully qualified domain name (FQDN) used to configure all advertised listeners.

      If you change this value on a running cluster, you must roll the cluster.

    For example, the following are Kafka advertised listeners for three Kafka brokers with portOffset set to 9093 and host set to test.example.com:

    • test.example.com:9093

    • test.example.com:9094

    • test.example.com:9095

  2. Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.

    Specify the mappings between the TCP port and the Kafka service as shown in the following example command. Map each broker to the TCP port based on the portOffset you set in Step #1. Use the following command to see the Kafka ClusterIP services and the ports:

    kubectl get services -n <namespace>
    

    The following example is a Helm command to install an NGINX Ingress controller, mapping the ports, 9093, 9094, and 9095, to Kafka services and service ports (three brokers).

    helm upgrade --install ingress-nginx ingress-nginx \
      --repo https://kubernetes.github.io/ingress-nginx \
      --namespace <namespace> \
      --set controller.ingressClass=kafka \
      --set tcp.9093="operator/kafka-0-internal:9092" \
      --set tcp.9094="operator/kafka-1-internal:9092" \
      --set tcp.9095="operator/kafka-2-internal:9092"
    
  3. Verify that the Ingress controller is correctly configured. Refer to your specific Ingress controller documentation for details.

    For NGINX, run the following commands to get the ConfigMap name and to verify the ConfigMap that the Ingress controller created:

    kubectl get configmap -n <namespace>
    kubectl describe configmap <configmap_name> -n <namespace>
    

    The output should include:

    • The namespace name

    • The Kafka broker service name

    • The port, as in the Helm command used to install the Ingress controller in Step 2

  4. Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.

    Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported. For detail on deploying the NGINX controller and configuring an Ingress resource, see the tutorial for deploying NGINX Ingress on GKE.

    The following example is to create an Ingress resource for an NGINX Ingress controller to expose three Kafka brokers:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-without-sni
      namespace: <namespace>
      annotations:
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/rewrite-target: /
    spec:
      rules:
        - host: test.example.com
          http:
            paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-0-internal
                    port:
                      number: 9092
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-1-internal
                    port:
                      number: 9092
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-2-internal
                    port:
                      number: 9092
    
  5. Create a DNS record with the host name you provided in Step #1 for Kafka and the external load balancer IP of the Ingress controller. The host name should resolve to the externalIP of the Ingress controller load balancer.

    You can retrieve the external IP using the following command:

    kubectl get services -n <namespace>
    

For a tutorial scenario on configuring external access using port-based static access, see the quickstart tutorial for port-based static access.