Configuring the network

Refer to the following information to understand and configure internal, external, and route-based load balancers for your components and cluster.

Note

The terms external, internal, and route are used in this document to differentiate between the types of load balancers you can enable for a Confluent Operator and Confluent Platform cluster. Your platform provider may use different terms.

Load balancers

YAML files have a section that contains load balancer configuration parameters. The component-specific load balancer configuration parameters, options, and usage comments are provided in the component YAML files.

Note

Kafka brokers and ZooKeeper maintain a constant connection that can be broken if a load balancer is used for ZooKeeper. There are no configuration parameters provided for setting up a ZooKeeper load balancer. Do not configure a load balancer for ZooKeeper.

Kafka Load balancer configuration

The following snippet shows the default (and unmodified) load balancer parameters from a Kafka values.yaml file. All parameter modifications (or additions) must be made in the Kafka section of the <provider>.yaml.

loadBalancer:
## If use-case for Kafka-cluster is to access from out of kubernetes cluster then configured `enabled` to true.
enabled: false
## External will create public facing endpoints, setting this to internal will
## create a private-facing ELB with VPC peering
type: external
## Add other annotations on the ELB here
annotations: {}
##  Domain name will configure in Kafka's external listener
##
domain: ""
## If configured the bootstrap fqdn will be .bootstrapPrefix.domain (dots are not supported in the prefix)
## If not the bootstrapPrefix will be .name.domain
bootstrapPrefix: ""
## If not configured, the default value will be 'b' appended to the domain name as prefix (dots are not supported in the prefix)
##
brokerPrefix: ""
  • All load balancer component configurations default to enabled: false.
  • There are three load balancer types. External load balancers are external-facing and are used to enable access to components from outside the cluster. This is the default load balancer created if the type parameter is not specified. Internal load balancers are private-facing and are used to support VPC-to-VPC peering. Route load balancers are used for OpenShift deployments.
  • Annotations are used to add one or more key/value pairs (metadata) to the default component configuration. Typically, annotations are used to add application-specific or provider-specific settings. For an example of how annotations are used in this environment, see Configuring internal load balancers. For additional information, refer to your provider documentation and Kubernetes Concepts: Annotations.
  • Domain is the provider domain where your cluster is running.
  • The bootstrapPrefix and brokerPrefix parameters are used to change the default load balancer prefixes. The default bootstrap prefix is the component name (for example kafka.<mydomain>) and the default Kafka broker prefix is b (for example b0.<mydomain, b1.<mydomain>, etc.). These are used for DNS entries in your domain. As part of your network plan, you may want to change the default prefixes for each component to avoid DNS conflicts when running multiple Kafka clusters.

The following shows the default parameters provided for load balancers in the <provider>.yaml file.

loadBalancer:
enabled: false
domain: ""

To enable an external load balancer (using the example domain mydevplatform.gcp.cloud), you change the default parameters to the following:

loadBalancer:
enabled: true
domain: "mydevplatform.gcp.cloud"

Note that since you did not provide a load balancer type, the load balancer defaults to an external load balancer. To enable an internal load balancer, you add the type: internal parameter.

loadBalancer:
enabled: true
type: internal
domain: "mydevplatform.gcp.cloud"

You add additional parameters and annotations as needed. For example, if you are creating a load balancer for Amazon Web Services (AWS), you could add the following annotation to create an internal Network Load Balancer (NLB).

loadBalancer:
   annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb

For more information about Kubernetes annotations, see Annotations.

For information about Kubernetes load balancer annotations for AWS, see Load Balancers.

Component load balancer configuration

The following snippet shows the default (and unmodified) load balancer YAML file parameters for setting up access to other Confluent Platform components. All configuration parameters are identical to the Kafka load balancer configuration, with the exception of the prefix parameters. Instead of brokerPrefix and bootstrapPrefix, there is one prefix parameter. This allows you to set or update the default component DNS name, which defaults to <component-name>.<providerdomain>. If you add a prefix parameter, the component DNS name is configured as <myprefix>.<providermdomain>.

## External Access
##
loadBalancer:
  ## Create a LoadBalancer for external networking
  enabled: false
  ## External will create public facing endpoints, setting this to internal will
  ## create a private-facing ELB with VPC peering
  type: external
  ## Add other annotations here that you want on the ELB
  annotations: {}
  ## If external access is enable, the FQDN must be provided
  ##
  domain: ""
  ## If prefix is configured, external DNS name is configured as <prefix>.<domain>
  ## otherwise external DNS name is configured as <name>.<domain> where name is cluster-name.
  prefix: ""

Configuring internal load balancers

Internal load balancers add component support for VPC peering. When you select type: internal, the installation automatically creates an internal load balancer that works for the provider used. This is possible because the installer automatically configures the following provider-specific annotations at installation:

Azure:

 annotations:
   service.beta.kubernetes.io/azure-load-balancer-internal: "true"

AWS:

 annotations:
   service.beta.kubernetes.io/aws-load-balancer-internal: "0.0.0.0/0"

GCP:

 annotations:
   cloud.google.com/load-balancer-type = "Internal"

To create an internal load balancer, add the type: internal entry to the component section in the <provider>.yaml file.

loadBalancer:
enabled: true
type: internal
domain: "<mydomain>"

Once the internal load balancer is available, you can use it as an internal-only access point for VPC-to-VPC component peering connections.

Configuring external load balancers

External-facing load balancers are used for access to cluster components from outside the provider environment. External load balancers are configured in the provider.yaml file. For example:

loadBalancer:
  enabled: true
  domain: "<mydomain>"

If you set enabled: true and add your provider domain, the installer automatically creates an external load balancer for the component at installation. See the following section for an example of how an external load balancer is configured for your Kafka brokers.

Kafka access

The following example shows the parameters that enable external load balancers for the Kafka brokers (using the example domain mydevplatform.gcp.cloud). Note that the examples provided can be used to configure access for other Confluent Platform components.

## Kafka Cluster
##
kafka:
  name: kafka
  replicas: 3
  resources:
    requests:
      cpu: 200m
      memory: 1Gi
  loadBalancer:
    enabled: true
    domain: "mydevplatform.gcp.cloud"

Once the external load balancers are created, you add a DNS entry associated with the public IP for each Kafka broker load balancer and the bootstrap load balancer in your provider DNS table (or whatever method you use to get DNS entries recognized by your provider environment).

Note

See metricReporter for information about DNS entries that could affect Kafka brokers.

To get the information you need to add DNS entries, first list the services for components in the namespace.

kubectl get services -n <namespace>

The following example uses operator as the namespace:

kubectl get services -n operator

The list of Kafka services resembles the following:

kafka                         ClusterIP      None            <none>           9071/TCP,9072/TCP,9092/TCP,7203/TCP,7777/TCP   21h
kafka-0-internal              ClusterIP      10.47.247.181   <none>           9071/TCP,9072/TCP,9092/TCP,7203/TCP,7777/TCP   21h
kafka-0-lb                    LoadBalancer   10.47.245.192   192.50.14.35     9092:32183/TCP                                 21h
kafka-1-internal              ClusterIP      10.47.251.31    <none>           9071/TCP,9072/TCP,9092/TCP,7203/TCP,7777/TCP   21h
kafka-1-lb                    LoadBalancer   10.47.251.8     192.50.28.28     9092:31028/TCP                                 21h
kafka-2-internal              ClusterIP      10.47.242.124   <none>           9071/TCP,9072/TCP,9092/TCP,7203/TCP,7777/TCP   21h
kafka-2-lb                    LoadBalancer   10.47.250.236   192.50.64.18     9092:32636/TCP                                 21h
kafka-bootstrap-lb            LoadBalancer   10.47.250.151   192.50.34.20     9092:30840/TCP                                 21h

Kafka broker load balancer DNS entries are derived from the component prefix, replica number, and the provider domain. The Kafka bootstrap load balancer DNS entry is derived from the component name (for example. kafka) and provider domain. The default Kafka component prefix is b for broker.

For example, the brokers in a default three-replica configuration have the prefix/replica numbers b0, b1, and b2. The Kafka bootstrap load balancer takes the name of the component, which defaults to kafka. The following shows the DNS table entries you add (using the example domain mydevplatform.gcp.cloud):

DNS name                           Internal IP             External IP
b0.mydevplatform.gcp.cloud         10.47.245.57            192.50.14.35
b1.mydevplatform.gcp.cloud         10.47.240.85            192.50.28.28
b2.mydevplatform.gcp.cloud         10.35.186.46            192.50.64.18
kafka.mydevplatform.gcp.cloud      10.47.250.36            192.50.34.20

Important

As part of your network plan, change the default prefixes for each component. If you add multiple clusters to one domain you can avoid DNS conflicts.

You access Kafka using the bootstrap load balancer DNS/port as shown in the example below:

http://kafka.mydevplatform.gcp.cloud:9092

The following sections provide additional information about networking and connecting to other components. Note that the default load balancer configuration for all components follows the pattern below.

loadBalancer:
enabled: false
domain: ""

Connect access

Enable access to Connect by updating the loadBalancer parameters as shown by the example in Kafka access. Note that you set a prefix using the parameter described in Component load balancer configuration.

The following shows an example of the default services deployed for Connect:

connectors                    ClusterIP      None            <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h
connectors-0-internal         ClusterIP      10.47.255.235   <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h
connectors-1-internal         ClusterIP      10.47.244.122   <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h

To add external access to Connect after installation, you need to update the Connect configuration. The example below shows the upgrade command to add an external load balancer:

helm upgrade -f ./providers/gcp.yaml \
 --set connectors.enabled=true \
 --set connectors.loadBalancer.enabled=true \
 --set connectors.loadBalancer.domain=<provider-domain> connectors \
 ./confluent-operator

The following example shows the DNS table entry you add:

DNS name                               Internal IP             External IP
connectors.mydevplatform.gcp.cloud     10.47.250.36            192.50.98.3

You access Connect using the load balancer DNS/port as shown in the example below:

http://connectors.mydevplatform.gcp.cloud:8083

Replicator access

Enable access to Replicator by updating the loadBalancer parameters as shown by the example in Kafka access. Note that you set a prefix using the parameter described in Component load balancer configuration.

The following shows an example of the default services deployed for Replicator:

replicator                    ClusterIP      None            <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h
replicator-0-internal         ClusterIP      10.47.251.204   <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h
replicator-1-internal         ClusterIP      10.47.250.214   <none>           8083/TCP,8443/TCP,7203/TCP,7777/TCP            21h

To add external access to Replicator after installation, you need to update the Replicator configuration. The example below shows the upgrade command to add an external load balancer:

helm upgrade -f ./providers/gcp.yaml \
 --set replicator.enabled=true \
 --set replicator.loadBalancer.enabled=true \
 --set replicator.loadBalancer.domain=<provider-domain> replicator \
 ./confluent-operator

The following example shows the DNS table entry you add:

DNS name                                   Internal IP          External IP
replicator.mydevplatform.gcp.cloud         10.47.250.36         192.50.43.9

You access Replicator using the load balancer DNS/port as shown in the example below:

http://replicator.mydevplatform.gcp.cloud:8083

Schema Registry access

Enable access to Schema Registry by updating the loadBalancer parameters as shown by the example in Kafka access. Note that you set a prefix using the parameter described in Component load balancer configuration.

The following shows an example of the default services deployed for Schema Registry:

schemaregistry                ClusterIP      None            <none>           8443/TCP,8081/TCP,7203/TCP,7777/TCP            21h
schemaregistry-0-internal     ClusterIP      10.47.242.195   <none>           8443/TCP,8081/TCP,7203/TCP,7777/TCP            21h
schemaregistry-1-internal     ClusterIP      10.47.241.243   <none>           8443/TCP,8081/TCP,7203/TCP,7777/TCP            21h

To add external access to Schema Registry after installation, you need to update the Schema Registry configuration. The example below shows the upgrade command to add an external load balancer:

helm upgrade -f ./providers/gcp.yaml \
 --set schemaregistry.enabled=true \
 --set schemaregistry.loadBalancer.enabled=true \
 --set schemaregistry.loadBalancer.domain=<provider-domain> schemaregistry \
 ./confluent-operator

The following example shows the DNS table entry you add:

DNS name                                   Internal IP          External IP
schemaregistry.mydevplatform.gcp.cloud     10.47.250.36         192.50.75.32

You access Schema Registry using the load balancer DNS/port as shown in the example below:

http://schemaregistry.mydevplatform.gcp.cloud:8081

KSQL access

Enable access to KSQL by updating the loadBalancer parameters as shown by the example in Kafka access. Note that you set a prefix using the parameter described in Component load balancer configuration.

The following shows an example of the default services deployed for KSQL:

ksql                    ClusterIP      None          <none>           8088/TCP,9088/TCP,7203/TCP,7777/TCP            21h
ksql-0-internal         ClusterIP      10.3.254.15   <none>           8088/TCP,9088/TCP,7203/TCP,7777/TCP            21h
ksql-1-internal         ClusterIP      10.3.255.33   <none>           8088/TCP,9088/TCP,7203/TCP,7777/TCP             21h

To add external access to KSQL after installation, you need to update the KSQL configuration. The example below shows the upgrade command to add an external load balancer:

helm upgrade -f ./providers/gcp.yaml \
 --set ksql.enabled=true \
 --set ksql.loadBalancer.enabled=true \
 --set ksql.loadBalancer.domain=<provider-domain> ksql \
 ./confluent-operator

The following example shows the DNS table entry you add:

DNS name                                   Internal IP          External IP
ksql.mydevplatform.gcp.cloud               10.3.250.36          192.50.82.13

You access KSQL using the URL below (using the example domain):

http://ksql.mydevplatform.gcp.cloud:8088

Control Center access

Enable access to Confluent Control Center by updating the loadBalancer parameters as shown by the example in Kafka access. Note that you set a prefix using the parameter described in Component load balancer configuration.

The following shows an example of the default services deployed for Confluent Control Center:

controlcenter                 ClusterIP      None            <none>           9021/TCP
controlcenter-0-internal      ClusterIP      10.47.247.213   <none>           9021/TCP

To add external access to Confluent Control Center after installation, you need to update the Confluent Control Center configuration. The example below shows the upgrade command to add an external load balancer:

helm upgrade -f ./providers/gcp.yaml \
 --set controlcenter.enabled=true \
 --set controlcenter.loadBalancer.enabled=true \
 --set controlcenter.loadBalancer.domain=<provider-domain> controlcenter \
 ./confluent-operator

The following example shows the DNS table entry you add:

DNS name                               Internal IP             External IP
controlcenter.mydevplatform.gcp.cloud  10.47.250.36            192.50.34.22

You access Confluent Control Center using the load balancer DNS/port as shown in the example below:

http://controlcenter.mydevplatform.gcp.cloud:9021

OpenShift networking

OpenShift uses route-based load balancing. To enable this type of networking for OpenShift, you add the type: route parameter to the private.yaml file. This is shown in the following example:

## Kafka Cluster
##
kafka:
  name: kafka
  replicas: 3
  resources:
    requests:
      cpu: 200m
      memory: 1Gi
  loadBalancer:
    enabled: false
    type: route
    domain: "<OpenShift-Domain>"
  tls:
    enabled: false
    fullchain: |-
    privkey: |-
    cacerts: |-

Important

For OpenShift route-based load balacing, the Kafka cluster must have TLS enabled (either TLS, SASL_SSL or SSL). SASL/PLAIN security is not supported in OpenShift deployments. See Configuring security for information about how to set up the component YAML files with TLS enabled.

See the OpenShift documentation for additional information.

No-DNS development access

If you need immediate external access without going through your organization’s DNS process, you can edit the hosts file on your local workstation. For Linux distributions, this file is typically located at /etc/hosts). In the example deployment, the Kafka load balancer hosts file entries resemble the following:

192.50.14.35  b0.mydevplatform.gcp.cloud b0
192.50.28.28  b1.mydevplatform.gcp.cloud b1
192.168.0.192 b2.mydevplatform.gcp.cloud b2
192.50.64.18  kafka.mydevplatform.gcp.cloud kafka

Once the hosts file is configured, you should be able to get to the Kafka bootstrap load balancer from your local workstation.

Important

No-DNS access is for development and testing purposes only and should not be used in a production environment.