Configure Port-based Static Access to Confluent Platform Using Confluent for Kubernetes

When you configure Kafka for port-based static access, Kafka advertised listeners are set up with the same host but with different ports.

This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using NGINX ingress controller.

To configure static external access to Kafka using port-based routing:

  1. Deploy Kafka with the staticForPortBasedRouting external access type:

    listeners:
      external:
        externalAccess:
          type: staticForPortBasedRouting
          staticForPortBasedRouting:
            portOffset:    --- [1]
            host:          --- [2]
    
    • [1] Required. The starting port number.

      If you change this value on a running cluster, you must roll the cluster.

    • [2] Required. The FQDN that will be used to configure all advertised listeners.

      If you change this value on a running cluster, you must roll the cluster.

    For example, the following are Kafka advertised listeners for three Kafka brokers with portOffset set to 9093 and host set to test.example.com:

    • test.example.com:9093
    • test.example.com:9094
    • test.example.com:9095
  2. Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.

    Specify the mappings between the TCP port and the Kafka service as shown in the example command below. Each broker should be mapped to the TCP port based on the portOffset you set in Step #1. Use the following command to see the Kafka clusterIP services and the ports:

    kubectl get services -n <namespace>
    

    The example below is a Helm command to install NGINX Ingress controller, mapping the ports, 9093, 9094, and 9095, to Kafka services and service ports (three brokers).

    helm upgrade --install ingress-nginx ingress-nginx \
      --repo https://kubernetes.github.io/ingress-nginx \
      --namespace <namespace> \
      --set controller.ingressClass=kafka \
      --set tcp.9093="operator/kafka-0-internal:9092" \
      --set tcp.9094="operator/kafka-1-internal:9092" \
      --set tcp.9095="operator/kafka-2-internal:9092"
    
  3. Verify that the Ingress controller is correctly configured. Refer to your specific Ingress controller documentation for details.

    For NGINX, run the following commands to get the configmap name and to verify the configmap that the Ingress controller created:

    kubectl get configmap -n <namespace>
    kubectl describe configmap <configmap name> -n <namespace>
    

    The output should have the namespace name, Kafka broker service name, and the port as in the above Helm command.

  4. Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.

    Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported. For detail on deploying the NGINX controller and configuring an Ingress resource, refer to this tutorial.

    The following example is to create an Ingress resource for NGINX Ingress controller to expose three Kafka brokers:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-without-sni
      namespace: confluent
      annotations:
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/rewrite-target: /
    spec:
      rules:
        - host: demo.example.com
          http:
            paths:
              - path:
                backend:
                  serviceName: kafka-0-internal
                  servicePort: 9092
              - path:
                backend:
                  serviceName: kafka-1-internal
                  servicePort: 9092
              - path:
                backend:
                  serviceName: kafka-2-internal
                  servicePort: 9092
    
  5. Create a DNS record with the host name you provided in Step #1 for Kafka and the external load balancer IP of the Ingress controller. The host name should resolve to the externalIP of the Ingress controller load balancer.

    You can retrieve the external IP using the following command:

    kubectl get services -n <namespace>
    

For a tutorial scenario on configuring external access using port-based static access, see the quickstart tutorial for port-based static access.