# Upgrade Confluent Platform Using Confluent for Kubernetes Upgrade a Confluent for Kubernetes (CFK) deployment using the following process. ## In this section - [Example of a multi-version upgrade](co-upgrade-example.md#co-upgrade-example): See the upgrade path and reasoning applied to a concrete scenario. - [Upgrade CFK](co-upgrade-cfk.md#co-upgrading): Upgrade the CFK Operator using Helm. - [Upgrade Confluent Platform](co-upgrade-cp.md#co-upgrade-cp): Upgrade Confluent Platform components for KRaft-based or ZooKeeper-based deployments. - [Migrate to and upgrade Control Center](co-upgrade-c3.md#co-upgrade-c3): Migrate Confluent Control Center (Legacy) alerts to Control Center and upgrade Control Center when moving to Confluent Platform 8.0. Before you start, review [Pre-upgrade tasks and known issues](co-upgrade-considerations.md#co-upgrade-additional-considerations). ## Why should you upgrade The following are some of the common reasons to upgrade: - To pick up security fixes, bug fixes, and new features in CFK and Confluent Platform. - To move to a CFK version that supports a newer Confluent Platform version. - To prepare for the architectural changes in Confluent Platform 8.0, which is the removal of ZooKeeper and the move to Control Center. For details, see the conditions in [Before you choose your upgrade path](#co-upgrade-before-choose-path) section. ## What you’re upgrading A CFK deployment has two independent layers that you upgrade separately: 1. You install and upgrade the CFK Operator with Helm. The Operator watches your Confluent Platform custom resources (CRs) and reconciles cluster state. The Operator package also includes the Custom Resource Definitions (CRDs) and the init container image. 2. The Confluent Platform components are Apache Kafka®, KRaft controllers, Schema Registry, Connect, ksqlDB, Control Center, and REST Proxy. Each runs from a container image whose tag is set in its CR. ## Check your current versions Verify your Kubernetes cluster is in the [supported environments](co-supported-environments.md#co-supported-environments) for the target CFK version, then identify your current CFK and Confluent Platform versions: - Check your current CFK version: ```bash helm list -n ``` Expected: `confluent-operator` with your current CFK version. - Check your current Confluent Platform version: ```bash kubectl get kafka -n -o jsonpath='{.status.currentClusterVersion}' ``` Expected: Your current Confluent Platform version in major.minor format, for example `7.9`. ## Before you choose your upgrade path Review the following conditions before you choose your upgrade path. For pre-upgrade tasks and known issues, see [Pre-upgrade tasks and known issues](co-upgrade-considerations.md#co-upgrade-additional-considerations). ### CFK and Confluent Platform compatibility Starting in CFK 2.9, CFK supports all supported Confluent Platform versions. For CFK versions earlier than 2.9, a CFK and Confluent Platform combination not listed in the [support matrix](co-supported-environments.md#co-operator-supported-versions) is still supported until you reach the final state where both are upgraded to a supported combination. For example, if you upgrade CFK from 2.6.1 to 3.3.0 along with Confluent Platform from 7.3.1 to 8.3.0, the combination of CFK 3.3.0 and Confluent Platform 7.3.1 is temporarily supported during the upgrade. #### Liveness probe change to exec in CFK 3.3.0 Starting in CFK 3.3.0, the liveness probe for the components that always run a liveness probe (Connect, Schema Registry, ksqlDB, Control Center, REST Proxy, Unified Stream Manager Agent, and Confluent Gateway) changes from an HTTP (`httpGet`) or TCP (`tcpSocket`) probe to an `exec` probe that runs the bundled `cfkprober` binary. After you upgrade, if you inspect a pod spec and see an `exec` probe in place of the previous `httpGet` or `tcpSocket` probe, this is expected and not a misconfiguration. The upgrade steps do not change. For Kafka and KRaft, the liveness probe remains opt-in, and when enabled, it also runs as an `exec` probe now. The probe runs in one of two modes: - By default, the probe runs the `cfkprober` binary, as the other components do. - If you set the `useProcNetPortCheck` flag, the probe reads the `/proc/net/tcp` file to find the port instead of communicating with the server. This helps avoid Transport Layer Security (TLS)-related errors. This flag is meaningful only for the Kafka and KRaft custom resources, where it avoids the TLS handshake issues described in the preceding section. The CFK init container delivers the `cfkprober` binary to the pod. When you upgrade, you must also update the init container image tag as described in the [CFK upgrade](co-upgrade-cfk.md#co-upgrade-init-container) process. If you skip the init container upgrade, the older init container does not deliver the `cfkprober` binary, so the new `exec` liveness probe fails open and always reports the component as live. #### Add RBAC permissions for custom roles before upgrading to 3.3 If you manage the CFK `ServiceAccount` permissions manually instead of letting Helm create them, add the following rules to your `Role` or `ClusterRole` before you upgrade to CFK 3.3 or later. Without these permissions, the CFK pod repeatedly crashes with an error like: ```text Failed to watch *v1.Job: failed to list *v1.Job: jobs.batch is forbidden: User "system:serviceaccount::confluent-for- kubernetes" cannot list resource "jobs" in API group "batch" in the namespace "" ``` ```yaml - apiGroups: - batch resources: - jobs verbs: - get - list - watch - create - update - patch - delete - apiGroups: - "" resources: - pods/log verbs: - get ``` For the full set of permissions required by CFK, see the reference role and role binding examples linked from [Configure Kubernetes RBAC and Custom Resource Definitions](co-prepare.md#co-k8s-rbac). #### ZooKeeper removal in Confluent Platform 8.0 Confluent Platform 8.0 removes ZooKeeper. Migrate your ZooKeeper-based deployment to KRaft before upgrading to 8.0. For migration steps, see [Migrate a Single Cluster from ZooKeeper to KRaft](co-migrate-kraft-procedure.md#co-migrate-kraft). #### Control Center replaces Control Center (Legacy) in Confluent Platform 8.0 Confluent Platform 8.0 or later doesn’t support Control Center (Legacy). Before you upgrade to Confluent Platform 8.x, migrate your alerts from Control Center (Legacy) to Control Center, and switch to Control Center. For migration steps, see [Migrate to and Upgrade Control Center](co-upgrade-c3.md#co-upgrade-c3). ## Choose your upgrade path Choose your path based on which layer you want to change. The CFK Operator, the Confluent Platform components, or both. Upgrading both is the most common case and keeps your CFK and Confluent Platform versions aligned with the [support matrix](co-supported-environments.md#co-operator-supported-versions). ![|co| Operator and |cp| components upgrade independently, with three supported upgrade paths](images/co-upgrade-cfk-cp-paths.svg) **Upgrade Confluent for Kubernetes only:** Make sure the CFK version you are upgrading to supports your current Confluent Platform version. If it does not, upgrade Confluent Platform into the new CFK’s supported range first. For instructions, see [Upgrade Confluent for Kubernetes](co-upgrade-cfk.md#co-upgrading). **Upgrade Confluent Platform only:** Make sure your current CFK version supports both the starting and ending Confluent Platform versions. For instructions, see [Upgrade Confluent Platform Using Confluent for Kubernetes](co-upgrade-cp.md#co-upgrade-cp). **Upgrade both Confluent for Kubernetes and Confluent Platform:** If your target Confluent Platform version is below 8.0, complete only steps 1 and 2 below. Steps 3 and 4 apply only when your target is Confluent Platform 8.0 or later. 1. [Upgrade Confluent for Kubernetes](co-upgrade-cfk.md#co-upgrading) 2. [Upgrade Confluent Platform Using Confluent for Kubernetes](co-upgrade-cp.md#co-upgrade-cp) 3. *(Confluent Platform 8.0 or later only.)* If your deployment is still ZooKeeper-based, [migrate to KRaft in your current version](co-migrate-kraft-procedure.md#co-migrate-kraft). 4. *(Confluent Platform 8.0 or later only.)* If you completed the preceding step, or your deployment was already KRaft-based, [upgrade to Confluent Platform 8.0](co-upgrade-cp.md#co-upgrade-kraft-based-cp). To upgrade from Confluent Operator 1.x to Confluent for Kubernetes, see [Migrate to Confluent for Kubernetes](co-migration.md#co-migration). ## Remove deprecated topic and broker configurations before Confluent Platform 8.0 upgrade Confluent Platform 8.0 removes or raises the minimum value of several broker and topic configurations. If your existing Kafka CR, KRaftController CR, or KafkaTopic CRs override these configurations with values that are no longer accepted, the brokers fail to start with errors similar to: ```text Invalid value 1000000 for configuration segment.bytes: Value must be at least 1048576 ``` or ```text Unknown topic config name: message.timestamp.difference.max.ms ``` Before you update the `cp-server` image tag to Confluent Platform 8.0 or later, update or remove the overrides described below in your Kafka CR, KRaftController CR, and KafkaTopic CRs, and any topic-level dynamic configurations that you set with the `kafka-configs` tool. ### Increase segment size to at least 1MiB Starting in Confluent Platform 8.0, [KIP-1030](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030%3A+Change+constraints+and+default+values+for+various+configurations) raises the minimum value of `segment.bytes` (topic-level) and `log.segment.bytes` (broker-level) from 14 bytes to 1048576 bytes (1MiB). The default value (1GiB) is unchanged. Topics or brokers configured below 1MiB cause broker startup to fail on Confluent Platform 8.0 and later. Update broker-level overrides in your Kafka CR: ```yaml kind: Kafka spec: configOverrides: server: - log.segment.bytes=1048576 ``` Update topic-level overrides in your KafkaTopic CRs: ```yaml kind: KafkaTopic spec: configs: segment.bytes: "1048576" ``` To find any topics with non-default `segment.bytes` overrides on a running cluster, run the `kafka-configs` tool against the Kafka brokers and update any value below 1048576 bytes. ### Remove deprecated broker and topic configurations Remove the following configurations. Leftover values prevent dynamic configuration updates and can return `Unknown topic config name` errors when CRs are reconciled: * `message.format.version` * `message.downconversion.enable` * `message.timestamp.difference.max.ms` ## Next steps - See a worked [example of a multi-version upgrade](co-upgrade-example.md#co-upgrade-example), then try the same reasoning against your own starting and target versions. - For pre-upgrade tasks and known issues, see [Pre-upgrade tasks and known issues](co-upgrade-considerations.md#co-upgrade-additional-considerations). - To upgrade the CFK Operator, see [Upgrade Confluent for Kubernetes](co-upgrade-cfk.md#co-upgrading). - To upgrade Confluent Platform components, see [Upgrade Confluent Platform Using Confluent for Kubernetes](co-upgrade-cp.md#co-upgrade-cp). - To migrate Confluent Control Center (Legacy) alerts to Control Center and upgrade Control Center, see [Migrate to and Upgrade Control Center](co-upgrade-c3.md#co-upgrade-c3).