Confluent for Kubernetes Quick Start

This topic presents the quick start guides for Confluent Platform and Confluent for Kubernetes (CFK). Currently, the following options are available:

Prerequisites

  • A Kubernetes cluster conforming to one of the supported versions.
  • kubectl installed, initialized, with the context set. You also must have the kubeconfig file configured for your cluster.
  • Helm 3 installed.
  • Access to the Confluent for Kubernetes bundle.
  • For these quick start guides, your Kubernetes cluster is assumed to have the default dynamic storage provisioner.

Quick start using KRaft

Step 1: Set the tutorial home directory

Set an environment variable to point to the location of the tutorial custom resource (CR) files:

export TUTORIAL_HOME="https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/master/quickstart-deploy/kraft-quickstart"
Copy

Step 2: Create a namespace

  1. Create the namespace to use:

    kubectl create namespace confluent
    
    Copy
  2. Set this namespace to default for your Kubernetes context:

    kubectl config set-context --current --namespace confluent
    
    Copy

Step 3: Install Confluent for Kubernetes

  1. Add the Confluent for Kubernetes Helm repository:

    helm repo add confluentinc https://packages.confluent.io/helm
    
    Copy
    helm repo update
    
    Copy
  2. Install Confluent for Kubernetes:

    helm upgrade --install \
      confluent-operator confluentinc/confluent-for-kubernetes
    
    Copy
  3. Check that the CFK pod comes up and is running:

    kubectl get pods
    
    Copy

Step 4: Install Confluent Platform

  1. Deploy the KRaft controller and the Kafka brokers:

    kubectl apply -f $TUTORIAL_HOME/confluent-platform.yaml
    
    Copy
  2. Install the sample producer app and topic:

    kubectl apply -f $TUTORIAL_HOME/producer-app-data.yaml
    
    Copy
  3. Wait until all the Confluent Platform components are deployed and running:

    kubectl get pods
    
    Copy

    In this tutorial, the following components are being deployed: KRaft controller, Kafka, Connect, Schema Registry, ksqlDB, REST Proxy, Confluent Control Center .

Step 5: View Control Center

Use Confluent Control Center to monitor the Confluent Platform, and see the created topic and data.

  1. Set up port forwarding to Control Center web UI from your local machine:

    kubectl port-forward controlcenter-0 9021:9021
    
    Copy
  2. Browse to Control Center at:

    http://localhost:9021

  3. Check that the elastic-0 topic was created and that messages are being produced to the topic.

  4. Optionally, you can try other tasks, such as adding or uploading your connectors. For details, see Install connector plugin.

Step 6: Tear down

  1. Shut down the sample producer app:

    kubectl delete -f $TUTORIAL_HOME/producer-app-data.yaml
    
    Copy
  2. Shut down the Confluent Platform cluster:

    kubectl delete -f $TUTORIAL_HOME/confluent-platform.yaml
    
    Copy
  3. Shut down the CFK when all the Confluent Platform components shut down:

    helm uninstall confluent-operator
    
    Copy
  4. Delete the namespace:

    kubectl delete namespace confluent
    
    Copy

Quick start using ZooKeeper

Step 1: Set the tutorial home directory

Set an environment variable to point to the location of the tutorial custom resource (CR) files:

export TUTORIAL_HOME="https://raw.githubusercontent.com/confluentinc/confluent-kubernetes-examples/master/quickstart-deploy"
Copy

Step 2: Create a namespace

  1. Create the namespace to use:

    kubectl create namespace confluent
    
    Copy
  2. Set this namespace to default for your Kubernetes context:

    kubectl config set-context --current --namespace confluent
    
    Copy

Step 3: Install Confluent for Kubernetes

  1. Add the Confluent for Kubernetes Helm repository:

    helm repo add confluentinc https://packages.confluent.io/helm
    
    Copy
    helm repo update
    
    Copy
  2. Install Confluent for Kubernetes:

    helm upgrade --install \
      confluent-operator confluentinc/confluent-for-kubernetes
    
    Copy

Step 4: Install Confluent Platform

  1. Install all Confluent Platform components:

    kubectl apply -f $TUTORIAL_HOME/confluent-platform.yaml
    
    Copy
  2. Install the sample producer app and topic:

    kubectl apply -f $TUTORIAL_HOME/producer-app-data.yaml
    
    Copy
  3. Wait until all the Confluent Platform pods are deployed and running:

    kubectl get pods
    
    Copy

    In this tutorial, the following components are being deployed: ZooKeeper, Kafka, Connect, Schema Registry, ksqlDB, REST Proxy, Confluent Control Center .

Step 5: View Control Center

Use Confluent Control Center to monitor the Confluent Platform, and see the created topic and data.

  1. Set up port forwarding to Control Center web UI from your local machine:

    kubectl port-forward controlcenter-0 9021:9021
    
    Copy
  2. Browse to Control Center at:

    http://localhost:9021

  3. Check that the elastic-0 topic was created and that messages are being produced to the topic.

  4. Optionally, you can try other tasks, such as adding or uploading your connectors. For details, see Install connector plugin.

Step 6: Tear down

  1. Shut down the sample producer app:

    kubectl delete -f $TUTORIAL_HOME/producer-app-data.yaml
    
    Copy
  2. Shut down the Confluent Platform clusters:

    kubectl delete -f $TUTORIAL_HOME/confluent-platform.yaml
    
    Copy
  3. Shut down the CFK:

    helm uninstall confluent-operator
    
    Copy
  4. Delete the namespace:

    kubectl delete namespace confluent
    
    Copy

Bonus: Install and use Confluent kubectl plugin

The Confluent for Kubernetes (CFK) bundle contains a Confluent kubectl plugin for interacting with Confluent for Kubernetes.

  1. Download the CFK bundle using the following command, and unpack the downloaded bundle:

    curl -O https://packages.confluent.io/bundle/cfk/confluent-for-kubernetes-2.11.0.tar.gz
    
    Copy
  2. Unpack the kubectl plugin that matches your client environment, Linux, Windows or Mac OS (Darwin), into your client environment local executables directory. On Mac OS and Linux, this would be /usr/local/bin/. This will allow the standard CLI kubectl to find the plugin.

    tar -xvf kubectl-plugin/kubectl-confluent-<environment>-amd64.tar.gz \
       -C <local directory for the plugin>
    
    Copy

    For example, on Mac OS:

    tar -xvf kubectl-plugin/kubectl-confluent-darwin-amd64.tar.gz \
       -C /usr/local/bin/
    
    Copy
  3. Open Confluent Control Center:

    kubectl confluent dashboard controlcenter
    
    Copy
  4. View the Confluent Platform version:

    kubectl confluent version
    
    Copy