Configure Host-Based Static Access to Confluent Platform Components Using Confluent for Kubernetes

When you configure Kafka for host-based static access, the Kafka advertised listeners are set up with the broker prefix and the domain name.

This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using NGINX ingress controller.

This method requires:

  • Kafka is configured with TLS.
  • An Ingress controller that supports SSL passthrough is used.

To configure external access to Kafka using static host-based routing:

  1. Configure and deploy Kafka with the staticForHostBasedRouting access type.

    listeners:
      external:
        externalAccess:
          type: staticForHostBasedRouting
          staticForHostBasedRouting:
            port:            --- [1]
            domain:          --- [2]
            brokerPrefix:    --- [3]
    
    • [1] Required. The port to be used in the advertised listener for a broker. Set it to 443 to support SNI capabilities.

      If you change this value on a running cluster, you must roll the cluster.

    • [2] Required. domain will be configured as part of the Kafka advertised listener.

      If you change this value on a running cluster, you must roll the cluster.

    • [3] Optional. Use brokerPrefix to change the default Kafka broker prefix. The default Kafka broker prefix is b.

      These are used for DNS entries. The broker DNS names become <brokerPrefix>0.<domain>, <brokerPrefix>1.<domain>, and so on.

      If not set, the default broker DNS names are b0.<domain>, b1.<domain>, and so on.

      For example, the following are Kafka advertised listeners for three Kafka brokers with port: 443 and domain: example.com:

      • b0.example.com:443
      • b1.example.com:443
      • b2.example.com:443

      If you change this value on a running cluster, you must roll the cluster.

  2. Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.

    Your Ingress controller must support SSL passthrough that intercepts all traffic on the configured HTTPS port (default is 443) and hands it over to the Kafka TCP proxy.

    The below example is a Helm command to install NGINX Ingress controller with SSL passthrough enabled:

    helm upgrade --install ingress-nginx ingress-nginx \
      --repo https://kubernetes.github.io/ingress-nginx \
      --set controller.publishService.enabled=true \
      --set controller.extraArgs.enable-ssl-passthrough="true"
    
  3. Configure the DNS addresses for Kafka brokers to point to the Ingress controller. You need the following to derive Kafka DNS entries:

    • domain name you provided in the configuration file in Step #1

    • The external IP of the Ingress controller load balancer

      You can retrieve the external IP using the following command:

      kubectl get services -n <namespace>
      
    • Kafka brokerPrefix you provided in the configuration file in Step #1

    The following example shows the DNS table entries using:

    • Domain: example.com
    • Three broker replicas with the default prefix/replica numbers: b
    DNS name               ExternalIP
    b0.example.com       34.71.198.214
    b1.example.com       34.71.198.214
    b2.example.com       34.71.198.214
    
  4. Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.

    Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported.

    For detail on deploying the NGINX controller and configuring an Ingress resource, refer to this tutorial.

    The following example is to create an Ingress resource for NGINX Ingress controller. The resource exposes three Kafka brokers:

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: ingress-with-sni
      namespace: confluent
      annotations:
        nginx.ingress.kubernetes.io/ssl-passthrough: "true" ---[1]
        nginx.ingress.kubernetes.io/ssl-redirect: "false"   ---[2]
        nginx.ingress.kubernetes.io/backend-protocol: HTTPS ---[3]
        ingress.kubernetes.io/ssl-passthrough: "true"       ---[4]
        kubernetes.io/ingress.class: nginx                  ---[5]
    spec:
      tls:
        - hosts:
            - demo0.example.com
            - demo1.example.com
            - demo2.example.com
            - demo.example.com
      rules:
        - host: demo0.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-0-internal
                  servicePort: 9092
        - host: demo1.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-1-internal
                  servicePort: 9092
        - host: demo2.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-2-internal
                  servicePort: 9092
    
    • Annotation [1] instructs the controller to send TLS connections directly to the backend instead of letting NGINX decrypt the communication.
    • Annotation [2] disables the default value.
    • Annotation [3] indicates how NGINX should communicate with the backend service.
    • Annotation [4] ssl-passthrough is required.
    • Annotation [5] uses the NGINX controller.

For a tutorial scenario on configuring external access using host-based static access, see the quickstart tutorial for host-based static access.