Install Confluent Manager for Apache Flink with Helm

This topic walks you through how to install Confluent Manager for Apache Flink (CMF) with Helm.

Step 1: Confirm prerequisites​

  1. Confirm you have adequate hardware.

    The underlying processor architecture of your Kubernetes worker nodes must be a supported version for the Confluent Platform for Apache Flink version you plan to deploy.

    Currently, Confluent Platform for Apache Flink supports x86 and ARM64 hardware architecture.

    Component Nodes Storage Memory CPU
    Confluent Manager for Apache Flink 1 10GB (persistent storage as PVC) [1] 1GB RAM [1] 2 [1]
    [1](1, 2, 3)

    Storage, memory and CPU values are configurable through the Helm installation.

  2. Install the required tools.

    This installation guide assumes you have already installed Helm. CMF supports Helm 3 for installation. You should have already configured Helm using the Helm documentation. To verify that your environment is prepared, the following commands should complete without error:

    kubectl get pods
helm list

    Add the Confluent Platform for Apache Flink Helm repository.

    helm repo add confluentinc https://packages.confluent.io/helm
helm repo update

Step 2: Install the Confluent Platform for Apache Flink Kubernetes operator

You must install the Confluent Platform for Apache Flink Kubernetes operator before you install CMF because CMF uses the operator to manage the Flink clusters.

  1. Install the certificate manager.

    kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml

  2. Install the Flink Kubernetes operator.

    helm upgrade --install cp-flink-kubernetes-operator confluentinc/flink-kubernetes-operator

    You must ensure that the Kubernetes operator watches the Kubernetes namespaces you want to deploy Flink applications to with CMF.

Step 3: Deploy CMF from the Confluent Helm repository

Next you will install CMF using the Confluent-provided Helm chart. This Helm chart is the only supported way to install and update CMF. Out-of-band updates to the resources that the helm chart creates is not supported.

  1. Store your Confluent license in a Kubernetes secret.

    kubectl create secret generic <license-secret-name> --from-file=license.txt

  2. Install CMF using the default configuration:

    helm upgrade --install cmf \
confluent/confluent-manager-for-apache-flink \
--namespace <namespace>
--set license.secretRef=<license-secret-name>

    Warning

    If you do not specify a license, CMF will generate a trial license.

  3. Configure the Chart. Helm provides several options for setting and overriding values in a chart. For CMF, you should customize the chart by passing a values file with the --values flag.

    First, use helm to show the default values.yaml file for CMF.

    helm inspect values --version 1.0.0 confluent/confluent-manager-for-apache-flink

    Note the following about CMF default values:

    • CMF uses SQLite to store metadata about your deployments. The data is persisted on a persistent volume Persistent Volumes that is created during the installation.

    • The persistent volume is created with your Kubernetes cluster’s default storage class. Depending on your storage class (for example if the reclaim policy is Delete ), your metadata might not be retained (Storage Classes) if you uninstall CMF. Make sure to backup the data in the persistent volume regularly.

    • If you want to set your storage class, you can overwrite persistence.storageClassName during the installation.

    • By default, the chart uses the image hosted by Confluent on DockerHub. To specify your own registry, set the following configuration values:

      image:
repository: <image-registry>
name: cp-cmf
pullPolicy: IfNotPresent
tag: <tag>

    • By default, the chart creates a cluster role and service account that CMF can use to create and monitor Flink applications in all namespaces. If you want to keep your service account, you set the serviceAccount.name property during installation to the preferred service account.

Related content