Confluent for Kubernetes¶
Confluent for Kubernetes (CFK) is a cloud-native control plane for deploying and managing Confluent in your private cloud environment. It provides a standard and simple interface to customize, deploy, and manage Confluent Platform through declarative API.
CFK runs on Kubernetes, the runtime for private cloud architectures.
Note
CFK is the next generation of Confluent Operator. For Confluent Operator 1.x documentation, see Confluent Operator 1, or use the version picker to browse to a specific version of the documentation.
For an overview of CFK, see Introducing Confluent for Kubernetes.
The following video provides an introduction to CFK.
The following shows the high-level architecture of CFK and Confluent Platform in Kubernetes. Note that CFK also supports Confluent REST Proxy that is not shown in the diagram.
Features¶
The following are summaries of the main, notable features of CFK.
- Cloud Native Declarative API
- Declarative Kubernetes-native API approach to configure, deploy, and manage Confluent Platform components (namely Apache Kafka®, Connect workers, ksqlDB, Schema Registry, Confluent Control Center, Confluent REST Proxy) and application resources (such as topics, rolebindings) through Infrastructure as Code (IaC).
- Provides built-in automation for cloud-native security best practices:
- Complete granular RBAC, authentication and TLS network encryption
- Auto-generated certificates
- Support for credential management systems, such as Hashicorp Vault, to inject sensitive configurations in memory to Confluent deployments
- Provides server properties, JVM, and Log4j configuration overrides for customization of all Confluent Platform components.
- Upgrades
- Provides automated rolling updates for configuration changes.
- Provides automated rolling upgrades with no impact to Kafka availability.
- Scaling
- Provides single command, automated scaling and reliability checks of Confluent Platform.
- Resiliency
- Restores a Kafka pod with the same Kafka broker ID, configuration, and persistent storage volumes if a failure occurs.
- Provides automated rack awareness to spread replicas of a partition across different racks (or zones), improving availability of Kafka brokers and limiting the risk of data loss.
- Scheduling
- Supports Kubernetes labels and annotations to provide useful context to DevOps teams and ecosystem tooling.
- Supports Kubernetes tolerations and pod/node affinity for efficient resource utilization and pod placement.
- Monitoring
- Supports metrics aggregation using JMX/Jolokia.
- Supports aggregated metrics export to Prometheus.
Documentation¶
A comprehensive set of documentation is available for CFK. The following are some of the documents you can start with CFK:
- To quickly set up a Confluent deployment on Kubernetes, follow the steps in Confluent for Kubernetes Quick Start.
- To understand how to plan your production deployment, review Plan for Confluent Platform Deployment using Confluent for Kubernetes.
- To understand what is supported across versions, read Confluent for Kubernetes Release Notes.
- To learn about the Kubernetes and Confluent concepts that are important for a production deployment, read through the various pages in the CFK documentation set.
- To review and start using example declarative specs, see the accompanying GitHub examples repository.
- To troubleshoot issues during deployment or operation, start from Troubleshoot Confluent for Kubernetes.