Configure Host-based Static Access to Confluent Components

When you configure Kafka for host-based static access, the Kafka advertised listeners are set up with the broker prefix and the domain name.

This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using NGINX ingress controller.

This method requires:

  • Kafka is configured with TLS.
  • An Ingress controller that supports SSL passthrough is used.

For the additional configuration steps required to allow external access to Metadata Service (MDS), see Configure Networking for RBAC.

To configure external access to Kafka using static host-based routing:

  1. Configure and deploy Kafka with the staticForHostBasedRouting access type.

    listeners:
      external:
        externalAccess:
          type: staticForHostBasedRouting
          staticForHostBasedRouting:
            port:            --- [1]
            domain:          --- [2]
            brokerPrefix:    --- [3]
    
    • [1] Required. The port to be used in the advertised listener for a broker. Set it to 443 to support SNI capabilities.

    • [2] Required. domain will be configured as part of the Kafka advertised listener.

    • [3] Optional. Use brokerPrefix to change the default Kafka broker prefix. The default Kafka broker prefix is b.

      These are used for DNS entries. The broker DNS names become <brokerPrefix>0.<domain>, <brokerPrefix>1.<domain>, and so on.

      If not set, the default broker DNS names are b0.<domain>, b1.<domain>, and so on.

      For example, the following are Kafka advertised listeners for three Kafka brokers with port: 443 and domain: example.com:

      • b0.example.com:443
      • b1.example.com:443
      • b2.example.com:443
  2. Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.

    Your Ingress controller must support SSL passthrough that intercepts all traffic on the configured HTTPS port (default is 443) and hands it over to the Kafka TCP proxy.

    The below example is a Helm command to install NGINX Ingress controller with SSL passthrough enabled:

    helm upgrade --install <release name> stable/nginx-ingress \
      --set controller.publishService.enabled=true \
      --set controller.extraArgs.enable-ssl-passthrough="true"
    
  3. Configure the DNS addresses for Kafka brokers to point to the Ingress controller. You need the following to derive Kafka DNS entries:

    • domain name you provided in the configuration file in Step #1

    • The external IP of the Ingress controller load balancer

      You can retrieve the external IP using the following command:

      kubectl get services -n <namespace>
      
    • Kafka brokerPrefix you provided in the configuration file in Step #1

    The following example shows the DNS table entries using:

    • Domain: example.com
    • Three broker replicas with the default prefix/replica numbers: b
    DNS name               ExternalIP
    b0.example.com       34.71.198.214
    b1.example.com       34.71.198.214
    b2.example.com       34.71.198.214
    
  4. Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.

    Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported.

    For detail on deploying the NGINX controller and configuring an Ingress resource, refer to this tutorial.

    The following example is to Ingress resource for NGINX Ingress controller. The resource is to expose three Kafka brokers:

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: ingress-with-sni
      annotations:
        nginx.ingress.kubernetes.io/ssl-passthrough: "true"
        kubernetes.io/ingress.class: nginx
        nginx.ingress.kubernetes.io/ssl-redirect: "false"
        ingress.kubernetes.io/ssl-passthrough: "true"
        nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    spec:
      tls:
        - hosts:
            - demo0.example.com
            - demo1.example.com
            - demo2.example.com
            - demo.example.com
      rules:
        - host: demo0.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-0-internal
                  servicePort: 9092
        - host: demo1.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-1-internal
                  servicePort: 9092
        - host: demo2.example.com
          http:
            paths:
              - backend:
                  serviceName: kafka-2-internal
                  servicePort: 9092
    

For a tutorial scenario on configuring external access using host-based static access, see the quickstart tutorial for host-based static access.