Configure Host-Based Static Access

Host-based static access configures Apache Kafka® advertised listeners using a fixed broker prefix and domain name, without provisioning per-broker Kubernetes load balancers or node ports.

Set up Kafka for host-based static access to configure the advertised listeners with the broker prefix and the domain name.

This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using an NGINX Ingress controller.

For the port-based alternative, see Configure Port-Based Static Access. For an overview of networking options, see Configure Networking.

This method requires:

  • Kafka is configured with TLS (Transport Layer Security). For more information, see Configure Network Encryption.

  • An Ingress controller that supports SSL (Secure Sockets Layer) passthrough is used.

Limitations

Connect deployment through CFK doesn’t support GCP Application Load Balancer (ALB) or static host-based routing for external access. To expose Connect, use one of the other methods in Configure Networking.

For an example of host-based ingress for Confluent Platform components, see the ingress-service-hostbased-example.yaml in the CFK examples repository.

To configure external access to Kafka using host-based static access:

  1. Configure and deploy Kafka with the staticForHostBasedRouting access type.

    listeners:
      external:
        externalAccess:
          type: staticForHostBasedRouting
          staticForHostBasedRouting:
            port:            --- [1]
            domain:          --- [2]
            brokerPrefix:    --- [3]
    
    • [1] Required. The port to be used in the advertised listener for a broker. Set it to 443 to support Server Name Indication (SNI) capabilities.

      If you change this value on a running cluster, you must roll the cluster.

    • [2] Required. domain is configured as part of the Kafka advertised listener.

      If you change this value on a running cluster, you must roll the cluster.

    • [3] Optional. Use brokerPrefix to change the default Kafka broker prefix. The default Kafka broker prefix is b.

      These are used for DNS entries. The broker DNS names become <brokerPrefix>0.<domain>, <brokerPrefix>1.<domain>, and so on.

      If not set, the default broker DNS names are b0.<domain>, b1.<domain>, and so on.

      For example, the following are Kafka advertised listeners for three Kafka brokers with port: 443 and domain: example.com:

      • b0.example.com:443

      • b1.example.com:443

      • b2.example.com:443

      If you change this value on a running cluster, you must roll the cluster.

  2. Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.

    Your Ingress controller must support SSL passthrough that intercepts all traffic on the configured HTTPS port (default is 443) and hands it over to the Kafka TCP proxy.

    The following example is a Helm command to install an NGINX Ingress controller with SSL passthrough enabled:

    helm upgrade --install ingress-nginx ingress-nginx \
      --repo https://kubernetes.github.io/ingress-nginx \
      --set controller.publishService.enabled=true \
      --set controller.extraArgs.enable-ssl-passthrough="true"
    
  3. Configure the DNS addresses for Kafka brokers to point to the Ingress controller. You need the following to derive Kafka DNS entries:

    • domain name you provided in the configuration file in Step #1

    • The external IP of the Ingress controller load balancer

      You can retrieve the external IP using the following command:

      kubectl get services -n <namespace>
      
    • Kafka brokerPrefix you provided in the configuration file in Step #1

    The following example shows the DNS table entries using:

    • Domain: example.com

    • Three broker replicas with the default prefix and replica numbers: b

    DNS name

    External IP

    b0.example.com

    34.71.198.214

    b1.example.com

    34.71.198.214

    b2.example.com

    34.71.198.214

  4. Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.

    Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported.

    For detail on deploying the NGINX controller and configuring an Ingress resource, see the NGINX Ingress on GKE tutorial.

    The following example is to create an Ingress resource for the NGINX Ingress controller. The resource exposes three Kafka brokers:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-with-sni
      namespace: <namespace>
      annotations:
        nginx.ingress.kubernetes.io/ssl-passthrough: "true" ---[1]
        nginx.ingress.kubernetes.io/ssl-redirect: "false"   ---[2]
        nginx.ingress.kubernetes.io/backend-protocol: HTTPS ---[3]
        ingress.kubernetes.io/ssl-passthrough: "true"       ---[4]
        kubernetes.io/ingress.class: nginx                  ---[5]
    spec:
      tls:
        - hosts:
            - demo0.example.com
            - demo1.example.com
            - demo2.example.com
            - demo.example.com
      rules:
        - host: demo0.example.com
          http:
            paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-0-internal
                    port:
                      number: 9092
        - host: demo1.example.com
          http:
            paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-1-internal
                    port:
                      number: 9092
        - host: demo2.example.com
          http:
            paths:
              - path: /
                pathType: Prefix
                backend:
                  service:
                    name: kafka-2-internal
                    port:
                      number: 9092
    
    • Annotation [1] instructs the controller to send TLS connections directly to the backend instead of letting NGINX decrypt the communication.

    • Annotation [2] disables the default value.

    • Annotation [3] indicates how NGINX should communicate with the backend service.

    • Annotation [4] ssl-passthrough is required.

    • Annotation [5] uses the NGINX controller.

For a tutorial scenario on configuring external access using host-based static access, see the quickstart tutorial for host-based static access.