Configure Host-Based Static Access
Host-based static access configures Apache Kafka® advertised listeners using a fixed broker prefix and domain name, without provisioning per-broker Kubernetes load balancers or node ports.
Set up Kafka for host-based static access to configure the advertised listeners with the broker prefix and the domain name.
This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using an NGINX Ingress controller.
For the port-based alternative, see Configure Port-Based Static Access. For an overview of networking options, see Configure Networking.
This method requires:
Kafka is configured with TLS (Transport Layer Security). For more information, see Configure Network Encryption.
An Ingress controller that supports SSL (Secure Sockets Layer) passthrough is used.
Limitations
Connect deployment through CFK doesn’t support GCP Application Load Balancer (ALB) or static host-based routing for external access. To expose Connect, use one of the other methods in Configure Networking.
For an example of host-based ingress for Confluent Platform components, see the ingress-service-hostbased-example.yaml in the CFK examples repository.
To configure external access to Kafka using host-based static access:
Configure and deploy Kafka with the
staticForHostBasedRoutingaccess type.listeners: external: externalAccess: type: staticForHostBasedRouting staticForHostBasedRouting: port: --- [1] domain: --- [2] brokerPrefix: --- [3]
[1] Required. The
portto be used in the advertised listener for a broker. Set it to443to support Server Name Indication (SNI) capabilities.If you change this value on a running cluster, you must roll the cluster.
[2] Required.
domainis configured as part of the Kafka advertised listener.If you change this value on a running cluster, you must roll the cluster.
[3] Optional. Use
brokerPrefixto change the default Kafka broker prefix. The default Kafka broker prefix isb.These are used for DNS entries. The broker DNS names become
<brokerPrefix>0.<domain>,<brokerPrefix>1.<domain>, and so on.If not set, the default broker DNS names are
b0.<domain>,b1.<domain>, and so on.For example, the following are Kafka advertised listeners for three Kafka brokers with
port: 443anddomain: example.com:b0.example.com:443b1.example.com:443b2.example.com:443
If you change this value on a running cluster, you must roll the cluster.
Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.
Your Ingress controller must support SSL passthrough that intercepts all traffic on the configured HTTPS port (default is 443) and hands it over to the Kafka TCP proxy.
The following example is a Helm command to install an NGINX Ingress controller with SSL passthrough enabled:
helm upgrade --install ingress-nginx ingress-nginx \ --repo https://kubernetes.github.io/ingress-nginx \ --set controller.publishService.enabled=true \ --set controller.extraArgs.enable-ssl-passthrough="true"
Configure the DNS addresses for Kafka brokers to point to the Ingress controller. You need the following to derive Kafka DNS entries:
domainname you provided in the configuration file in Step #1The external IP of the Ingress controller load balancer
You can retrieve the external IP using the following command:
kubectl get services -n <namespace>
Kafka
brokerPrefixyou provided in the configuration file in Step #1
The following example shows the DNS table entries using:
Domain:
example.comThree broker replicas with the default prefix and replica numbers:
b
DNS name
External IP
b0.example.com
34.71.198.214
b1.example.com
34.71.198.214
b2.example.com
34.71.198.214
Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.
Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported.
For detail on deploying the NGINX controller and configuring an Ingress resource, see the NGINX Ingress on GKE tutorial.
The following example is to create an Ingress resource for the NGINX Ingress controller. The resource exposes three Kafka brokers:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-with-sni namespace: <namespace> annotations: nginx.ingress.kubernetes.io/ssl-passthrough: "true" ---[1] nginx.ingress.kubernetes.io/ssl-redirect: "false" ---[2] nginx.ingress.kubernetes.io/backend-protocol: HTTPS ---[3] ingress.kubernetes.io/ssl-passthrough: "true" ---[4] kubernetes.io/ingress.class: nginx ---[5] spec: tls: - hosts: - demo0.example.com - demo1.example.com - demo2.example.com - demo.example.com rules: - host: demo0.example.com http: paths: - path: / pathType: Prefix backend: service: name: kafka-0-internal port: number: 9092 - host: demo1.example.com http: paths: - path: / pathType: Prefix backend: service: name: kafka-1-internal port: number: 9092 - host: demo2.example.com http: paths: - path: / pathType: Prefix backend: service: name: kafka-2-internal port: number: 9092
Annotation [1] instructs the controller to send TLS connections directly to the backend instead of letting NGINX decrypt the communication.
Annotation [2] disables the default value.
Annotation [3] indicates how NGINX should communicate with the backend service.
Annotation [4]
ssl-passthroughis required.Annotation [5] uses the NGINX controller.
For a tutorial scenario on configuring external access using host-based static access, see the quickstart tutorial for host-based static access.