Deploy Confluent Platform Using Confluent for Kubernetes

Using Confluent for Kubernetes (CFK), install Confluent Platform components in the following steps:

  1. Review and complete the required configuration tasks for Confluent Platform components.

    There can be CFK-specific considerations for some components.

    For example, in CFK, ksqlDB can only be deployed in the interactive mode. CFK does not support the headless mode deployment of ksqlDB.

    For detailed configuration information, refer to specific configuration pages under Configure Confluent Platform.

  2. Specify the Confluent Platform images and the version to deploy.

  3. Deploy Confluent Platform.

  4. Specify the Confluent Control Center image and the version to deploy.

  5. Deploy Confluent Control Center.

  6. Validate your Confluent Platform deployment.

Specify the Confluent Platform images and the version to deploy

  1. For the Confluent Platform components you want to deploy, specify the Docker image names and the version tags in the component custom resources (CRs).

    For the list of Confluent images, see Confluent Docker Image Reference.

    The tags are the version of Confluent Platform. For example, for Schema Registry version 8.3.0, set application: confluentinc/cp-schema-registry:8.3.0 in the Schema Registry CR.

    For details on how to specify different Confluent Platform versions in the component custom resources, see Confluent for Kubernetes example GitHub repo.

  2. Specify the init container in spec.image.init:. The tag of the init container is the version of CFK.

    To deploy the 6.2.0 and later versions of Confluent Platform, you must use the confluent-init-container instead of the legacy cp-init-container-operator.

    An example Schema Registry CR to deploy Schema Registry 8.3.0 with CFK 3.3.0:

    kind: SchemaRegistry
    metadata:
      name: schemaregistry
      namespace: confluent
    spec:
      replicas: 1
      image:
        application: confluentinc/cp-schema-registry:8.3.0
        init: confluentinc/confluent-init-container:3.3.0
    
  3. Specify the number of replicas for Confluent Platform components in the component CRs using the spec.replicas property.

    This example deploys one Schema Registry replica.

    You can deploy only one replica of Control Center in CFK.

  4. If you are using CFK 3.x to deploy Confluent Platform 7.x, you need to set the following annotation in the component CR to use Log4j. CFK 3.x uses Log4j 2 by default. When you set the annotation to true, CFK deploys the Confluent Platform component with Log4j.

    kind: <Component>
    metadata:
      annotations:
        platform.confluent.io/use-log4j1: "true"
    

Deploy Confluent Platform

  1. Apply the Confluent Platform CR configurations. You can apply one configuration file for all Confluent Platform components, or apply multiple configuration files for individual Confluent Platform components.

    kubectl apply -f <CR>
    

    Expected: One <kind>.platform.confluent.io/<name> created line per resource in the CR file.

  2. Check that all Confluent Platform resources are successfully deployed:

    kubectl get pods
    

    Expected: Every pod shows a matching ready count, such as 1/1, under READY and Running under STATUS.

Specify the Confluent Control Center image and the version to deploy

You can deploy only one replica of Control Center (spec.replicas: 1) in CFK.

  1. To deploy Control Center, specify the Docker image name and the version tags in the Control Center CR in spec.image.application. The tag is the version of Control Center.

    For example:

    kind: ControlCenter
    spec:
      image:
        application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag>
        init: confluentinc/confluent-init-container:3.3.0
    
  2. Specify the init container in spec.image.init:. The tag of the init container is the version of CFK.

    The init container tag should be at least 2.11.1 to deploy Control Center.

    <c3_tag> is the Control Center release you are installing.

    An example Control Center CR:

    kind: ControlCenter
    metadata:
      name: control-center-next-gen
      namespace: confluent
    spec:
      replicas: 1
      image:
        application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag>
        init: confluentinc/confluent-init-container:3.3.0
    
  3. Specify the Prometheus and Alertmanager images and tags:

    kind: ControlCenter
    metadata:
      name: control-center-next-gen
      namespace: confluent
    spec:
      replicas: 1
      image:
        application: confluentinc/cp-enterprise-control-center-next-gen:<c3_tag>
        init: confluentinc/confluent-init-container:3.3.0
      services:
        prometheus:
          image: confluentinc/cp-enterprise-prometheus:<c3_tag>
          pvc:
            dataVolumeCapacity: 10Gi
        alertmanager:
          image: confluentinc/cp-enterprise-alertmanager:<c3_tag>
    

Deploy Confluent Control Center

  1. After you configure Control Center and specify the image name and tag, apply the ControlCenter CR:

    kubectl apply -f <CR>
    

    Expected: controlcenter.platform.confluent.io/control-center-next-gen created.

  2. Check whether any Control Center container is stuck or crashing. For example:

    kubectl get pod control-center-next-gen-0 -n <namespace> \
      -o jsonpath="{range .status.containerStatuses[*]}{.name}:{.state.*.reason}{'\n'}{end}"
    

    Expected: A blank <reason> for each container name means the container is running normally. This command only prints a reason (such as CrashLoopBackOff) for a container that is waiting or has terminated. To confirm the pod as a whole is ready, use kubectl get pods instead.

  3. Use local port forwarding to load Control Center at localhost:9021:

    kubectl port-forward pod/<pod_name> 9021:9021 -n <namespace>
    

Validate your Confluent Platform deployment

Validate your Confluent Platform deployment by checking the following:

  1. Pods are running and healthy

  2. Services are deployed

  3. Control Center is accessible and monitors Apache Kafka® and other components

  4. Client can create topics

For sample steps to validate a basic deployment, see the Confluent for Kubernetes example GitHub repo. You can also review the other scenarios for validating various deployment configurations, such as with various external access methods and security setups.