Prepare Kubernetes Cluster for Confluent Platform

This topic describes the required tasks to prepare your Kubernetes cluster for Confluent Platform deployment. The user performing these tasks will need appropriate Kubernetes cluster-level permissions.

Create a namespace for Confluent Platform

  1. Create a Kubernetes namespace to deploy Confluent Platform into:

    kubectl create namespace <confluent-namespace>
    
  2. Set the new namespace as the current namespace.

    This step is not required, but is given here to simplify the example commands in the rest of the documents. The --namespace flag in the subsequent kubectl commands will be omitted as the commands assume the current namespace.

    kubectl config set-context --current --namespace=<confluent-namespace>
    

Configure Kubernetes RBAC

There are typically three personas involved in the deployment process of Confluent for Kubernetes (CFK). Two are human personas and one is a Service Account, as follows:

  • Kubernetes cluster admin

    The Kubernetes administrator provisions a namespace and specific permissions for the CFK user and the CFK Service Account.

  • CFK user

    The CFK user deploys CFK and Confluent Platform components.

  • CFK Service Account

    The CFK Service Account allows CFK to access the Kubernetes API and create StatefulSets, Services, Secrets, others that Confluent Platform needs.

To control the level of access that you want to allow the CFK Service Account has, you can limit its scope to a single Kubernetes namespace, and you can prevent it from installing cluster-level resources:

  • Namespaced deployment (Recommended)

    The CFK Service Account only manages resources within the namespace it is deployed to.

  • Cluster RBAC

    Using Kubernetes RBAC, the CFK Service Account can manages specific cluster-scoped Kubernetes resources such as Storage Classes, Custom Resource Definitions, Cluster Roles, and Cluster Role Bindings that Confluent Platform needs.

Here are questions to help select the right deployment scenario for you:

  • Do you want CFK to create and delete Confluent Platform clusters within only one specific namespace or in any namespace?
  • Do you want the CFK Service Account to be able to create cluster-level resources, or should your Kubernetes cluster admin pre-create the cluster level resources?

Kubernetes RBAC for namespaced deployment

To allow a user who does not have cluster-level access to deploy CFK and Confluent Platform in a namespace, perform the following tasks as a Kubernetes cluster admin before deploying CFK and Confluent Platform.

The snippets and the resource file referenced in this section use Helm repo and the confluent namespace for CFK install.

  1. Pull the Helm Chart contents to get the Confluent custom resource definitions (CRDs):

    mkdir -p <cfk-dir>
    
    helm pull confluentinc/confluent-for-kubernetes \
      --untar --untardir=<cfk-dir>
    
  2. Pre-install the Confluent CRDs with the following command:

    kubectl apply -f <cfk-dir>/crds \
      -namespace confluent
    
  3. Create the rolebinding.yaml file with the permissions required for a namespaced deployment, using Namespaced role bindings as the starting point. The content contains the minimum permissions required. Add any other resource permissions you might additionally require.

    The Role and RoleBinding must be in the same namespace as CFK.

    The subject in the RoleBinding must be the user/account existing in the given namespace.

  4. Create the Role and RoleBinding for the CFK Service Account:

    kubectl apply -f namespaced-rolebinding.yaml \
      -namespace confluent
    
  5. Deploy CFK with the rbac property set to false and the namespaced set to true. See Deploy Confluent for Kubernetes without permission for Kubernetes role binding and Deploy Confluent for Kubernetes with namespaced scope for the steps.

Kubernetes RBAC for cluster-wide deployment

To allow a user who does not have cluster-level access to deploy CFK and Confluent Platform cluster-wide, perform the following tasks as a Kubernetes cluster admin before deploying CFK and Confluent Platform.

The snippets and the resource file referenced in this section use Helm repo and the confluent namespace for CFK install.

  1. Pull the Helm Chart contents to get the Confluent CRDs:

    mkdir -p <cfk-dir>
    
    helm pull confluentinc/confluent-for-kubernetes \
      --untar --untardir=<cfk-dir>
    
  2. Pre-install the Confluent CRDs with the following command:

    kubectl apply -f <cfk-dir>/crds \
      -namespace confluent
    
  3. Create the cluster-role-rolebinding.yaml file with the permissions required for a cluster–wide deployment, using Cluster Role and Cluster RoleBinding as the starting point. The content contains the minimum permissions required. Add any other resource permissions you might additionally require.

    The ClusterRole and ClusterRoleBinding should be in the same namespace as CFK.

    The subject in the ClusterRoleBinding must be the user/account existing in the given namespace.

  4. Create the ClusterRole and ClusterRoleBinding for the CFK Service Account:

    kubectl apply -f cluster-role-rolebinding.yaml \
      -namespace confluent
    
  5. Deploy CFK with the rbac property set to false as described in Deploy Confluent for Kubernetes without permission for Kubernetes role binding.