Configure Host-Based Static Access to Confluent Platform Components Using Confluent for Kubernetes¶
When you configure Kafka for host-based static access, the Kafka advertised listeners are set up with the broker prefix and the domain name.
This method does not create any Kubernetes resources, and you need to explicitly configure external access to Kafka, for example, using NGINX ingress controller.
This method requires:
- Kafka is configured with TLS.
- An Ingress controller that supports SSL passthrough is used.
To configure external access to Kafka using static host-based routing:
Configure and deploy Kafka with the
staticForHostBasedRouting
access type.listeners: external: externalAccess: type: staticForHostBasedRouting staticForHostBasedRouting: port: --- [1] domain: --- [2] brokerPrefix: --- [3]
[1] Required. The
port
to be used in the advertised listener for a broker. Set it to443
to support SNI capabilities.If you change this value on a running cluster, you must roll the cluster.
[2] Required.
domain
will be configured as part of the Kafka advertised listener.If you change this value on a running cluster, you must roll the cluster.
[3] Optional. Use
brokerPrefix
to change the default Kafka broker prefix. The default Kafka broker prefix isb
.These are used for DNS entries. The broker DNS names become
<brokerPrefix>0.<domain>
,<brokerPrefix>1.<domain>
, and so on.If not set, the default broker DNS names are
b0.<domain>
,b1.<domain>
, and so on.For example, the following are Kafka advertised listeners for three Kafka brokers with
port: 443
anddomain: example.com
:b0.example.com:443
b1.example.com:443
b2.example.com:443
If you change this value on a running cluster, you must roll the cluster.
Deploy an Ingress controller, such as ingress-nginx. For a list of available controllers, see Ingress controllers.
Your Ingress controller must support SSL passthrough that intercepts all traffic on the configured HTTPS port (default is 443) and hands it over to the Kafka TCP proxy.
The below example is a Helm command to install NGINX Ingress controller with SSL passthrough enabled:
helm upgrade --install ingress-nginx ingress-nginx \ --repo https://kubernetes.github.io/ingress-nginx \ --set controller.publishService.enabled=true \ --set controller.extraArgs.enable-ssl-passthrough="true"
Configure the DNS addresses for Kafka brokers to point to the Ingress controller. You need the following to derive Kafka DNS entries:
domain
name you provided in the configuration file in Step #1The external IP of the Ingress controller load balancer
You can retrieve the external IP using the following command:
kubectl get services -n <namespace>
Kafka
brokerPrefix
you provided in the configuration file in Step #1
The following example shows the DNS table entries using:
- Domain:
example.com
- Three broker replicas with the default prefix/replica numbers:
b
DNS name ExternalIP b0.example.com 34.71.198.214 b1.example.com 34.71.198.214 b2.example.com 34.71.198.214
Create an Ingress resource that includes a collection of rules the Ingress controller uses to route the inbound traffic to Kafka.
Ingress uses annotations to configure some options depending on the Ingress controller, an example of which is the rewrite-target annotation. Review the documentation for your Ingress controller to learn which annotations are supported.
For detail on deploying the NGINX controller and configuring an Ingress resource, refer to this tutorial.
The following example is to create an Ingress resource for NGINX Ingress controller. The resource exposes three Kafka brokers:
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: ingress-with-sni namespace: confluent annotations: nginx.ingress.kubernetes.io/ssl-passthrough: "true" ---[1] nginx.ingress.kubernetes.io/ssl-redirect: "false" ---[2] nginx.ingress.kubernetes.io/backend-protocol: HTTPS ---[3] ingress.kubernetes.io/ssl-passthrough: "true" ---[4] kubernetes.io/ingress.class: nginx ---[5] spec: tls: - hosts: - demo0.example.com - demo1.example.com - demo2.example.com - demo.example.com rules: - host: demo0.example.com http: paths: - backend: serviceName: kafka-0-internal servicePort: 9092 - host: demo1.example.com http: paths: - backend: serviceName: kafka-1-internal servicePort: 9092 - host: demo2.example.com http: paths: - backend: serviceName: kafka-2-internal servicePort: 9092
- Annotation [1] instructs the controller to send TLS connections directly to the backend instead of letting NGINX decrypt the communication.
- Annotation [2] disables the default value.
- Annotation [3] indicates how NGINX should communicate with the backend service.
- Annotation [4]
ssl-passthrough
is required. - Annotation [5] uses the NGINX controller.
For a tutorial scenario on configuring external access using host-based static access, see the quickstart tutorial for host-based static access.