Deploy Confluent Platform Using Confluent for Kubernetes
Using Confluent for Kubernetes (CFK), install Confluent Platform components in the following steps:
Review and complete the required configuration tasks for Confluent Platform components.
There can be CFK-specific considerations for some components.
For example, in CFK, ksqlDB can only be deployed in the interactive mode. CFK does not support the headless mode deployment of ksqlDB.
For detailed configuration information, refer to specific configuration pages under Configure Confluent Platform.
Specify the Confluent Platform images and the version to deploy.
Specify the Confluent Control Center image and the version to deploy.
Specify the Confluent Platform images and the version to deploy
For the Confluent Platform components you want to deploy, specify the Docker image names and the version tags in the component custom resources (CRs).
For the list of Confluent images, see Confluent Docker Image Reference.
The tags are the version of Confluent Platform. For example, for Schema Registry version 8.3.0, set
application: confluentinc/cp-schema-registry:8.3.0in the Schema Registry CR.For details on how to specify different Confluent Platform versions in the component custom resources, see Confluent for Kubernetes example GitHub repo.
Specify the init container in
spec.image.init:. The tag of the init container is the version of CFK.To deploy the 6.2.0 and later versions of Confluent Platform, you must use the
confluent-init-containerinstead of the legacycp-init-container-operator.An example Schema Registry CR to deploy Schema Registry 8.3.0 with CFK 3.3.0:
kind: SchemaRegistry metadata: name: schemaregistry namespace: confluent spec: replicas: 1 image: application: confluentinc/cp-schema-registry:8.3.0 init: confluentinc/confluent-init-container:3.3.0
Specify the number of replicas for Confluent Platform components in the component CRs using the
spec.replicasproperty.This example deploys one Schema Registry replica.
You can deploy only one replica of Control Center in CFK.
If you are using CFK 3.x to deploy Confluent Platform 7.x, you need to set the following annotation in the component CR to use Log4j. CFK 3.x uses Log4j 2 by default. When you set the annotation to
true, CFK deploys the Confluent Platform component with Log4j.kind: <Component> metadata: annotations: platform.confluent.io/use-log4j1: "true"
Deploy Confluent Platform
Apply the Confluent Platform CR configurations. You can apply one configuration file for all Confluent Platform components, or apply multiple configuration files for individual Confluent Platform components.
kubectl apply -f <CR>
Expected: One
<kind>.platform.confluent.io/<name> createdline per resource in the CR file.Check that all Confluent Platform resources are successfully deployed:
kubectl get pods
Expected: Every pod shows a matching ready count, such as
1/1, underREADYandRunningunderSTATUS.
Specify the Confluent Control Center image and the version to deploy
You can deploy only one replica of Control Center (spec.replicas: 1) in CFK.
To deploy Control Center, specify the Docker image name and the version tags in the Control Center CR in
spec.image.application. The tag is the version of Control Center.For example:
kind: ControlCenter spec: image: application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag> init: confluentinc/confluent-init-container:3.3.0
Specify the init container in
spec.image.init:. The tag of the init container is the version of CFK.The init container tag should be at least
2.11.1to deploy Control Center.<c3_tag>is the Control Center release you are installing.An example Control Center CR:
kind: ControlCenter metadata: name: control-center-next-gen namespace: confluent spec: replicas: 1 image: application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag> init: confluentinc/confluent-init-container:3.3.0
Specify the Prometheus and Alertmanager images and tags:
kind: ControlCenter metadata: name: control-center-next-gen namespace: confluent spec: replicas: 1 image: application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag> init: confluentinc/confluent-init-container:3.3.0 services: prometheus: image: confluentinc/cp-enterprise-prometheus:<c3_tag> pvc: dataVolumeCapacity: 10Gi alertmanager: image: confluentinc/cp-enterprise-alertmanager:<c3_tag>
Deploy Confluent Control Center
After you configure Control Center and specify the image name and tag, apply the ControlCenter CR:
kubectl apply -f <CR>
Expected:
controlcenter.platform.confluent.io/control-center-next-gen created.Check whether any Control Center container is stuck or crashing. For example:
kubectl get pod control-center-next-gen-0 -n <namespace> \ -o jsonpath="{range .status.containerStatuses[*]}{.name}:{.state.*.reason}{'\n'}{end}"
Expected: A blank
<reason>for each container name means the container is running normally. This command only prints a reason (such asCrashLoopBackOff) for a container that is waiting or has terminated. To confirm the pod as a whole is ready, usekubectl get podsinstead.Use local port forwarding to load Control Center at localhost:9021:
kubectl port-forward pod/<pod_name> 9021:9021 -n <namespace>
Validate your Confluent Platform deployment
Validate your Confluent Platform deployment by checking the following:
Pods are running and healthy
Services are deployed
Control Center is accessible and monitors Apache Kafka® and other components
Client can create topics
For sample steps to validate a basic deployment, see the Confluent for Kubernetes example GitHub repo. You can also review the other scenarios for validating various deployment configurations, such as with various external access methods and security setups.