Confluent for Kubernetes Quick Start

This topic presents the quick start guides for Confluent Platform and Confluent for Kubernetes (CFK). Currently, the following options are available:

Prerequisites

  • A Kubernetes cluster conforming to one of the supported versions.

  • kubectl installed, initialized, with the context set. You also must have the kubeconfig file configured for your cluster.

  • Helm 3 installed.

  • Access to the Confluent for Kubernetes bundle.

  • For these quick start guides, your Kubernetes cluster is assumed to have the default dynamic storage provisioner.

Quick start using KRaft

Step 1: Set the tutorial home directory

Set an environment variable to point to the location of the tutorial custom resource (CR) files:

export TUTORIAL_HOME="https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/master/quickstart-deploy/kraft-quickstart"

Step 2: Create a namespace

  1. Create the namespace to use:

    kubectl create namespace confluent

  2. Set this namespace to be the default for your Kubernetes context:

    kubectl config set-context --current --namespace confluent

Step 3: Install Confluent for Kubernetes

  1. Add the Confluent for Kubernetes Helm repository:

    helm repo add confluentinc https://packages.confluent.io/helm

    helm repo update

  2. Install Confluent for Kubernetes:

    helm upgrade --install \
  confluent-operator confluentinc/confluent-for-kubernetes

  3. Check that the CFK pod comes up and is running:

    kubectl get pods

Step 4: Install Confluent Platform

  1. Deploy the KRaft controller and the Kafka brokers:

    kubectl apply -f $TUTORIAL_HOME/confluent-platform-c3++.yaml

  2. Install the sample producer app and topic:

    kubectl apply -f $TUTORIAL_HOME/producer-app-data.yaml

  3. Wait until all the Confluent Platform components are deployed and running:

    kubectl get pods

    In this tutorial, the following components are being deployed: KRaft controller, Kafka, Connect, Schema Registry, ksqlDB, REST Proxy, Control Center .

Step 5: View Control Center

Use Control Center to monitor Confluent Platform, and see the created topic and data.

  1. Set up port forwarding to Control Center web UI from your local machine:

    kubectl port-forward controlcenter-0 9021:9021

  2. Browse to Control Center at:

    http://localhost:9021

  3. Check that the elastic-0 topic was created and that messages are being produced to the topic.

  4. Optionally, you can try other tasks, such as adding or uploading your connectors. For details, see Install connector plugin.

Step 6: Tear down

  1. Shut down the sample producer app:

    kubectl delete -f $TUTORIAL_HOME/producer-app-data.yaml

  2. Shut down the Confluent Platform cluster:

    kubectl delete -f $TUTORIAL_HOME/confluent-platform-c3++.yaml

  3. Shut down the CFK when all the Confluent Platform components shut down:

    helm uninstall confluent-operator

  4. Delete the namespace:

    kubectl delete namespace confluent

Quick start using ZooKeeper

This quick start works with Confluent Platform 7.9 and earlier versions.

Step 1: Set the tutorial home directory

Set an environment variable to point to the location of the tutorial custom resource (CR) files:

export TUTORIAL_HOME="https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/master/quickstart-deploy"

Step 2: Create a namespace

  1. Create the namespace to use:

    kubectl create namespace confluent

  2. Set this namespace to default for your Kubernetes context:

    kubectl config set-context --current --namespace confluent

Step 3: Install Confluent for Kubernetes

  1. Add the Confluent for Kubernetes Helm repository:

    helm repo add confluentinc https://packages.confluent.io/helm

    helm repo update

  2. Install Confluent for Kubernetes:

    helm upgrade --install \
  confluent-operator confluentinc/confluent-for-kubernetes

Step 4: Install Confluent Platform

  1. Install all Confluent Platform components:

    kubectl apply -f $TUTORIAL_HOME/confluent-platform.yaml

  2. Install the sample producer app and topic:

    kubectl apply -f $TUTORIAL_HOME/producer-app-data.yaml

  3. Wait until all the Confluent Platform pods are deployed and running:

    kubectl get pods

    In this tutorial, the following components are being deployed: ZooKeeper, Kafka, Connect, Schema Registry, ksqlDB, REST Proxy, Confluent Control Center (Legacy) .

Step 5: View Control Center (Legacy)

Use Confluent Control Center (Legacy) to monitor the Confluent Platform, and see the created topic and data.

  1. Set up port forwarding to Control Center (Legacy) web UI from your local machine:

    kubectl port-forward controlcenter-0 9021:9021

  2. Browse to Control Center (Legacy) at:

    http://localhost:9021

  3. Check that the elastic-0 topic was created and that messages are being produced to the topic.

  4. Optionally, you can try other tasks, such as adding or uploading your connectors. For details, see Install connector plugin.

Step 6: Tear down

  1. Shut down the sample producer app:

    kubectl delete -f $TUTORIAL_HOME/producer-app-data.yaml

  2. Shut down the Confluent Platform clusters:

    kubectl delete -f $TUTORIAL_HOME/confluent-platform.yaml

  3. Shut down the CFK:

    helm uninstall confluent-operator

  4. Delete the namespace:

    kubectl delete namespace confluent

Bonus: Install and use Confluent kubectl plugin

The Confluent for Kubernetes (CFK) bundle contains a Confluent kubectl plugin for interacting with Confluent for Kubernetes.

  1. Download the CFK bundle using the following command, and unpack the downloaded bundle:

    curl -O https://packages.confluent.io/bundle/cfk/confluent-for-kubernetes-3.0.0.tar.gz

  2. Unpack the kubectl plugin that matches your client environment, Linux, Windows or Mac OS (Darwin), into your client environment local executables directory. On Mac OS and Linux, this would be /usr/local/bin/. This will allow the standard CLI kubectl to find the plugin.

    tar -xvf kubectl-plugin/kubectl-confluent-<environment>-amd64.tar.gz \
   -C <local directory for the plugin>

    For example, on Mac OS:

    tar -xvf kubectl-plugin/kubectl-confluent-darwin-amd64.tar.gz \
   -C /usr/local/bin/

  3. Open Control Center:

    kubectl confluent dashboard controlcenter

  4. View the Confluent Platform version:

    kubectl confluent version