Advanced Configuration Options in Confluent for Kubernetes Blueprints¶
This topic describes a few of the advanced configuration options Confluent for Kubernetes (CFK) Blueprints provides for Confluent Platform.
Mount custom volumes¶
Kubernetes provides storage abstraction through Kubernetes Volumes. These volumes can be ephemeral, where the volumes get destroyed when the pod dies, or persistent, where the lifetime of the volumes goes beyond the pod’s lifetime.
In CFK Blueprints, you can have a Confluent Platform pod configured to have multiple volumes of various types attached to the pod simultaneously and mounted at desired paths in the pod.
Mounting custom volumes does not support multiple PersistentVolumes for ZooKeeper and Kafka data. CFK Blueprints configures and manages one PersistentVolume for ZooKeeper and Kafka data.
The following are a few of the common use cases for custom volume mounts:
Third-party secrets providers
As an alternative to using Kubernetes secrets to secure sensitive information, you can use a vault product like Hashicorp Vault, AWS Secrets Manager, and Azure KeyVault.
You integrate a third-party secrets provider by configuring an ephemeral volume mount for the Confluent component pod that takes the credentials from the secrets provider.
Some Kafka connectors require JARs, that are outside of the Connect plugin but need to be available to the Connect pods. You can create persistent volumes with the connector JARs and mount them on the Connect worker pods.
Multiple custom partitions
For example, you could write logs to a separate persistent volume of your choice.
In CFK Blueprints, you mount custom volumes to Confluent component pods by defining custom volume mounts in the component class custom resources (CRs). The same volume will be mounted on all the pods in the component cluster in the specified paths.
To mount custom volumes to a Confluent Platform component:
Configure the volumes according to the driver specification.
Add the following to the Confluent Platform component CR:
kind: <component>ClusterClass spec: provisioner: cfk: k8s: mountedVolumes: - volumes: ---  - volumeMounts: --- 
 Required. An array of named volumes in a pod that may be accessed by any container in the pod.
For the supported volume types and the specific configuration properties required for each volume type, see Kubernetes Volume Types for the supported volume types.
 Required. An array of mounting paths of the
volumeswithin this container.
For the configuration properties for volume mount, see Kubernetes Pod volumeMounts.
Before the volumes and volume mounts are added to the component pod template, CFK Blueprints performs a validation to ensure that there is no conflict with internal volume mounts. Reconcile will fail, and the error will be added to the CFK Blueprints logs in the following cases:
A custom volume’s mount path conflicts with an internal mount path.
These are the internal mounts used by Confluent Platform components:
A custom volume’s mount path conflicts with a custom-mounted secret.
There is a conflict between the custom volume names or custom volume mount paths.
The below example is to mount an Azure file volume and HashiCorp vault with SecretProviderClass and a CSI driver:
apiVersion: platform.confluent.io/v1beta1 kind: Kafka spec: provisioner: cfk: k8s: mountedVolumes: volumes: - name: azure azureFile: secretName: azure-secret shareName: aksshare readOnly: true - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "vault-database" volumeMounts: - name: azure mountPath: /mnt/azurePath1 - name: azure mountPath: /mnt/azurePath2 - name: secrets-store-inline mountPath: "/mnt/secrets-store" readOnly: true
Customize Confluent pods with Pod Overlay¶
Confluent for Kubernetes (CFK) Blueprints supports a subset of Kubernetes PodTemplateSpec in its API. When you configure the stateful sets for Confluent Platform components, you can use the Pod Overlay feature to set and use additional Kubernetes features that the CFK Blueprints API does not natively support.
Example use cases for the Pod Overlay feature include:
Deploy a Confluent Platform cluster with a custom init container that runs alongside the CFK init container.
In this case, the custom init container runs before the CFK init container.
Use a newly introduced Kubernetes feature that has not been added to the CFK Blueprints API.
To use Pod Overlay:
Create a template file (
<template-file>) with the settings you want to add:
The template file must start with
spec.templateas above, and it has to follow the Kubernetes StatefulSetSpec API.
You can configure fields only inside
Fields specified outside of
spec.templatewill be considered as invalid.
The following example is for a custom init container:
spec: template: spec: initContainers: - name: busybox image: busybox:1.28 command: ["echo","I am a custom init-conatiner"] imagePullPolicy: IfNotPresent
Create a ConfigMap (
<configmap>) using the file created in the previous step (
<template-file>). You must use
pod-template.yamlas the key with the
kubectl create configmap <configmap> --from-file=pod-template.yaml=<template-file> -n <namespace>
Reference the above config map in the Confluent Platform component class custom resource (<component>ClusterClass CR):
kind: <component>ClusterClass spec: provisioner: cfk: podOverlayConfigMapRef: name: <configmap>
<configmap>>is the name of the config map that you created in the previous step to provide Pod Overlay configurations.
According to the Kubernetes convention, a ConfigMap can only be referenced by the pods residing in the same namespace. So CFK Blueprints will look for
<configmap>within the same namespace as the component ClusterClass CR object.
A ConfigMap can’t be shared across the Blueprints. But it can be shared within the Blueprint among multiple Confluent component classes.
Provide custom service account¶
Each Confluent component pod deployed by CFK Blueprints has an associated Kubernetes Service Account. These service accounts serve several functions in CFK Blueprints. For example, to deploy Confluent Docker images from a private Docker image registry, you can associate Docker registry credentials to the service account associated with your Confluent components.
If you do not specify a service account, the
default service account is
assigned to the component pod in the same namespace.
To provide a custom service account, set
serviceAccountName in the component
class custom resource (CR):
kind: <component>ClusterClass spec: provisioner: cfk: k8s: podTemplate serviceAccountName: