Upgrade Confluent Platform Using Confluent for Kubernetes

Upgrade Confluent Platform components using Confluent for Kubernetes by choosing your upgrade path, preparing your clusters, and running the upgrade steps for KRaft-based or ZooKeeper-based deployments.

Before you start the upgrade, review the pre-upgrade tasks and known issues so the upgrade is not interrupted. For configurations that cannot be updated on deployed clusters, see Configurations that can’t change after deployment.

Known issues

Review the following known issues before you start the upgrade.

Warning

Before you update the cp-server image tag to Confluent Platform 8.0 or later, remove or update any deprecated broker and topic configuration overrides in your Apache Kafka® CR, KRaftController CR, and KafkaTopic CRs. Leftover values such as message.format.version or a segment.bytes value below 1MiB cause brokers to fail to start. For details, see Remove deprecated topic and broker configurations before Confluent Platform 8.0 upgrade.

A separate issue affects OAuth client assertion authentication starting in Confluent Platform 8.3.x.

Note

On Confluent Platform 8.3.x, components that authenticate to Kafka using OAuth client assertions (such as Schema Registry, Connect, ksqlDB, Control Center, and REST Proxy) fail to start. This happens because Kafka 4.3 rejects the configured private-key file under an empty, deny-by-default allowlist. Before you upgrade, set the -Dorg.apache.kafka.sasl.oauthbearer.allowed.files JVM system property on the affected component. If you configure issuer or JSON Web Key Set (JWKS) endpoint URLs, also set -Dorg.apache.kafka.sasl.oauthbearer.allowed.urls.

Before you begin

Identify your deployment mode

Your upgrade path depends on whether your Kafka cluster runs in KRaft mode or ZooKeeper mode. Check for a KRaftController resource in your namespace:

kubectl get kraftcontroller -n <namespace>

If the command returns a kraftcontroller resource, your deployment is KRaft-based. Use the KRaft upgrade path.
If the command returns No resources found, your deployment is ZooKeeper-based. Use the ZooKeeper upgrade path.

Get your current CR files

Each upgrade step updates the image tag in a Confluent Platform component CR. If you do not already have the CR YAML files locally, export them from the cluster.

For each component you plan to upgrade, save the current CR to a file:

kubectl get <CR kind> <CR name> -n <namespace> -o yaml > <CR name>.yaml

For example, to save the Kafka CR:

kubectl get kafka kafka -n <namespace> -o yaml > kafka.yaml

Edit the saved file when a step instructs you to update the image tag, then apply the file with kubectl apply -f <file>.

Choose your upgrade path

Decision tree for choosing the |kraft| or |zk| upgrade order, including the |c3-short++| 2.0/2.1 exception — Decision tree for choosing the KRaft or ZooKeeper upgrade order, including the Control Center 2.0/2.1 exception

If your deployment is KRaft-based (Confluent Platform 7.x or later), upgrade in this order:

KRaft controllers
Kafka brokers
Update metadata version
Other Confluent components (excluding Control Center and Control Center (Legacy)), in any order
Control Center

If your deployment is ZooKeeper-based (Confluent Platform 7.9 or earlier only), upgrade in this order:

ZooKeeper
Kafka
Other Confluent components (excluding Control Center and Control Center (Legacy)), in any order
Control Center

ZooKeeper, the other Confluent components, and Control Center all use the same generic procedure in Upgrade other Confluent Platform components below: repeat those steps for each component, in the order shown in the preceding list.

Upgrade a ZooKeeper-based deployment to Confluent Platform 8.0

Confluent Platform 8.0 removes ZooKeeper, so a ZooKeeper-based deployment must migrate to KRaft before upgrading to 8.0. The steps depend on your starting Confluent Platform version.

If you are on Confluent Platform 7.6.x or earlier:

Upgrade Kafka to 7.9 using the ZooKeeper-based upgrade path. Confluent Platform 7.7.0 and later support production-level KRaft migration, so you must first reach a supported version.
Migrate the ZooKeeper-based Confluent Platform to KRaft in 7.9.
Upgrade the KRaft-based Confluent Platform to 8.0.

If you are on Confluent Platform 7.7.x or later:

If you are upgrading to Confluent Platform 8.0 or later and already have Control Center 2.0 or 2.1 installed, upgrade in this order:

Confluent Platform 8.0 does not work with Control Center 2.0 or 2.1. When upgrading KRaft-based Confluent Platform to 8.0 and Control Center from 2.0 or 2.1 to 2.2, upgrade Control Center before upgrading Kafka:

Control Center.

For migration steps, see the Control Center Migration guide.
KRaft controllers
Kafka brokers
Other Confluent components (excluding Control Center and Control Center (Legacy)), in any order

Upgrade Kafka and KRaft in the KRaft mode

Warning

Known limitation: Upgrade from Confluent Platform 7.7.3 or 7.8.x to 7.9.x

When upgrading a KRaft-based cluster from Confluent Platform 7.7.3 or 7.8.x to 7.9.x, upgrade Kafka brokers before KRaft controllers. If a broker restarts while controllers are already on 7.9.x, the broker fails to register and enters a crash loop.

Decision tree for the broker-before-controller order required when upgrading |cp| 7.7.3 or 7.8.x to 7.9.0 through 7.9.2 — Decision tree for the broker-before-controller order required when upgrading Confluent Platform 7.7.3 or 7.8.x to 7.9.0 through 7.9.2

This occurs because Confluent Platform 7.9.x controllers require brokers to include the confluent.metadata.version field during registration. Older brokers remove this field from the registration request because they misidentify the 7.9.x controller as an Kafka controller rather than a Confluent Platform controller. This misidentification causes registration to fail.

To avoid this issue:

Upgrade directly to Confluent Platform 7.9.3 or later and follow the standard upgrade order.
If upgrading to Confluent Platform 7.9.0–7.9.2, upgrade Kafka brokers first and wait for them to become healthy before upgrading controllers. Avoid broker restarts while controllers are on 7.9.x.

Controller logs show this error when the issue occurs:

[ControllerServer id=9990] registerBroker: event failed with UnsupportedVersionException ...
Exception message: Unable to register broker 0 because it does not support confluent.metadata.version.
Please upgrade your broker to Confluent Platform, or downgrade your controller to Apache Kafka.
Brokers must be upgraded first in a rolling upgrade from Apache to CP.

Clusters starting on 7.7.2 or earlier, and clusters already on 7.9.3 or later, are not affected.

With that limitation in mind, review the following guidance for the steps below.

Important

Read the warning above before starting Step 1 of the |kraft| controller and |ak| broker upgrade below.

If you are upgrading from Confluent Platform 7.7.3 or 7.8.x to Confluent Platform 7.9.0, 7.9.1, or 7.9.2, do Step 2 before Step 1 (Kafka brokers before KRaft controllers). Following Steps 1 → 2 in the order shown causes a broker registration crash loop.

For all other upgrade paths (including upgrading directly to Confluent Platform 7.9.3 or later, or to Confluent Platform 8.x), follow Steps 1 → 2 → 3 in order.

Step 1: Upgrade KRaft controllers

If your KRaftController CR uses a name other than kraftcontroller, replace the pod and statefulset names in the commands below accordingly.

In the KRaftController custom resource (CR), update the image tag to the target Confluent Platform version:
```
kind: KRaftController
spec:
  image:
    application: confluentinc/cp-server:8.3.0
```
In the same CR, verify that the init container image tag is the current CFK version:
```
spec:
  image:
    init: confluentinc/confluent-init-container:3.3.0
```
Apply the updated CR:
```
kubectl apply -f <KRaftController CR> -n <namespace>
```
Expected: kraftcontroller.platform.confluent.io/kraftcontroller configured
Wait for the rolling restart to complete:
```
kubectl rollout status statefulset/kraftcontroller -n <namespace>
```
Expected: statefulset rolling update complete.

Verify the KRaft controllers are running on the target version:

kubectl get kraftcontroller -n <namespace>

Expected: STATUS shows RUNNING.

kubectl get pod kraftcontroller-0 -n <namespace> \
  -o jsonpath='{.spec.containers[*].image}'

Expected: image tag matches confluentinc/cp-server:8.3.0.

Step 2: Upgrade Kafka brokers

In the Kafka CR, update the image tag to the target Confluent Platform version:

kind: Kafka
spec:
  image:
    application: confluentinc/cp-server:8.3.0

In the same CR, verify that the init container image tag is the current CFK version:
```
spec:
  image:
    init: confluentinc/confluent-init-container:3.3.0
```
Apply the updated CR:
```
kubectl apply -f <Kafka CR> -n <namespace>
```
Expected: kafka.platform.confluent.io/kafka configured
Wait for the rolling restart to complete:
```
kubectl rollout status statefulset/kafka -n <namespace>
```
Expected: statefulset rolling update complete.
Verify the Kafka brokers are running on the target version:
```
kubectl get kafka -n <namespace>
```
Expected: STATUS shows RUNNING.
```
kubectl get pod kafka-0 -n <namespace> \
  -o jsonpath='{.spec.containers[*].image}'
```
Expected: image tag matches confluentinc/cp-server:8.3.0.

Step 3: Update metadata version

After verifying cluster health and performance, increment the metadata version for controllers and brokers. Run kafka-features from inside a Kafka broker pod. The binary is preinstalled, and the pod can reach the internal bootstrap listener (kafka:9071 by default):

kubectl exec kafka-0 -n <namespace> -- \
  kafka-features --bootstrap-server kafka:9071 \
    upgrade \
    --metadata 4.3

Expected: confluent.metadata.version was upgraded to <internal-numeric>. The numeric is an internal version code. You can see the effective version using the describe command shown below.

If your Kafka CR uses a name other than kafka, replace the pod name (kafka-0) and the bootstrap host accordingly.

The --metadata value must match the Apache Kafka® version that ships with your target Confluent Platform release. The example above uses 4.3 (the value for Confluent Platform 8.3.0). For other Confluent Platform versions, use:

Target Confluent Platform version	`--metadata` value
7.6	`3.6`
7.7	`3.7`
7.8	`3.8`
7.9	`3.9`
8.0	`4.0`
8.1	`4.1`
8.2	`4.2`
8.3	`4.3`

Verify the metadata version:

kubectl exec kafka-0 -n <namespace> -- \
  kafka-features --bootstrap-server kafka:9071 describe

Expected: The confluent.metadata.version row shows a FinalizedVersionLevel matching your target (for example, CP-7.9-IV0B after setting --metadata 3.9).

Warning

Do not use the --release-version option with the kafka-features command on clusters managed by CFK. This option upgrades kraft.version from 0 to 1, which is not part of the Confluent Platform upgrade process. CFK does not support upgrading kraft.version.

Use only the --metadata flag for CFK-managed clusters.

For more information about the metadata version, see the Confluent Platform documentation: Upgrade Kafka brokers in KRaft mode.

Upgrade Kafka in the ZooKeeper mode

This section only applies to Confluent Platform 7.9 or earlier.

Important

Starting with Confluent Platform version 8.0, ZooKeeper is no longer part of Confluent Platform.

You have two options to upgrade Kafka:

Automatic upgrade (recommended, CFK 2.8 or later)
Manual upgrade (CFK 2.7 or earlier, or for manual control)

Automatic upgrade of Kafka

Starting in CFK 2.8, CFK automates the Kafka upgrade process by automatically managing inter.broker.protocol.version and log.message.format.version during upgrades.

CFK enables this upgrade option by default. It applies to:

Standard Confluent Platform Docker images only (not custom image registries)
ZooKeeper-based Kafka clusters (Confluent Platform 7.9 or earlier only)

Note

CFK sets inter.broker.protocol.version in CP format. For example, when upgrading from Confluent Platform 7.3 to 7.6, CFK sets it as CP-7.3 rather than the Kafka format 3.3.

Step 1: (Optional) Override the default inter-broker version

CFK defaults to inter-broker version 2.6. To override this before running the auto upgrade (for example, when upgrading from Confluent Platform 7.3 to 7.6), set the bootstrap inter-broker protocol (IBP) version annotation:

kubectl annotate kafka kafka \
  platform.confluent.io/bootstrap-ibp-version="<inter-broker-version>" \
  -n <namespace>

Set <inter-broker-version> in CFK format. For example, use CP-7.3 when upgrading from Confluent Platform 7.3.

Verify the update:

kubectl get kafka kafka -n <namespace> -o yaml | grep previousClusterVersion

Note

Use this override only for a cluster that has already completed at least one automatic upgrade, so previousClusterVersion reflects a real prior Confluent Platform version. If this is the first automatic upgrade since the cluster was originally deployed, previousClusterVersion does not yet reflect a genuine prior version. As a result, the value you see here may not match what you set. In that case, this step does not apply to you: skip the override and continue to Step 2.

Remove the annotation after verifying:

kubectl annotate kafka kafka \
  platform.confluent.io/bootstrap-ibp-version- \
  -n <namespace>

Step 2: Verify the init container image tag

In the Kafka CR, verify that the init container image tag is the current CFK version:

spec:
  image:
    init: confluentinc/confluent-init-container:3.3.0

Step 3: Update the Kafka image tag

In the Kafka CR, update the image tag to the target Confluent Platform version:

spec:
  image:
    application: confluentinc/cp-server:8.3.0

Step 4: Apply the `Kafka` CR

The automatic upgrade triggers when the image tag is later than status.currentClusterVersion:

kubectl apply -f <Kafka CR> -n <namespace>

Expected: kafka.platform.confluent.io/kafka configured

Step 5: Verify the final cluster versions

To verify the final cluster versions, run the following command:

kubectl get kafka kafka -n <namespace> -o yaml | grep previousClusterVersion

kubectl get kafka kafka -n <namespace> -o yaml | grep currentClusterVersion

Expected: For example, after upgrading from 7.3 to 7.6:

status.previousClusterVersion: 7.3
status.currentClusterVersion: 7.6

Note

CFK does not set previousClusterVersion and currentClusterVersion when you manually upgrade Kafka.

The automatic upgrade always sets inter.broker.protocol.version to the previous Kafka version. In the above example, CFK sets inter.broker.protocol.version to 3.3. On the next upgrade from Confluent Platform 7.6 to 7.7, CFK sets it to 3.6.

Manual upgrade of Kafka

Use this method if you are on CFK 2.7 or earlier, or if you want manual control over the upgrade process.

Step 1: Update the Kafka image tag

In the Kafka CR, update the image tag to the target Confluent Platform version:

spec:
  image:
    application: confluentinc/cp-server:8.3.0

Step 2: Verify the init container image tag

Verify that the init container image tag is the current CFK version:

spec:
  image:
    init: confluentinc/confluent-init-container:3.3.0

Step 3: Disable the automatic upgrade feature

Add the following annotation to the Kafka CR to disable the automatic upgrade feature:

metadata:
  annotations:
    platform.confluent.io/automatic-ibp-upgrade: "disable"

Step 4: Set the current inter-broker protocol version

Set inter.broker.protocol.version to your current Confluent Platform version.

Look up the protocol version for your current Confluent Platform release in the Kafka upgrade documentation.

If upgrading from Confluent Platform 6.2.x or earlier, also set log.message.format.version. If upgrading from Confluent Platform 7.0 or later, set only inter.broker.protocol.version.

For example, when upgrading from Confluent Platform 7.4.1:

spec:
  configOverrides:
    server:
      - inter.broker.protocol.version=3.4

Step 5: Apply the `Kafka` CR

Apply the updated CR:

kubectl apply -f <Kafka CR> -n <namespace>

Expected: kafka.platform.confluent.io/kafka configured

Step 6: Set the target inter-broker protocol version

After the brokers have upgraded, set inter.broker.protocol.version to the target Confluent Platform version.

Look up the target protocol version in the Kafka upgrade documentation (the same table linked in Step 4). For example, when upgrading to 8.3, the target version is 4.3:

spec:
  configOverrides:
    server:
      - inter.broker.protocol.version=<target-broker-protocol-version>

Step 7: Apply the `Kafka` CR again

Apply the CR again:

kubectl apply -f <Kafka CR> -n <namespace>

Expected: kafka.platform.confluent.io/kafka configured

Note

CFK does not set the previousClusterVersion and currentClusterVersion properties in manual upgrades. These properties do not affect your cluster, but you can edit them manually if needed.

Upgrade other Confluent Platform components

Upgrade Confluent Platform components including ZooKeeper (Confluent Platform 7.9 and earlier) as follows:

Update the component image tag. In the component CR, set the image tag to the target Confluent Platform version:
```
spec:
  image:
    application: <component image>:8.3.0
```
Verify the init container image tag matches your Confluent for Kubernetes version. In the same CR, confirm the init container tag is set to your current CFK version (the same tag set in Step 7: Upgrade CFK init container):
```
spec:
  image:
    init: confluentinc/confluent-init-container:3.3.0
```

If upgrading Control Center, also update the Prometheus and Alertmanager image tags. Control Center uses independent versions, not Confluent Platform release versions. Set <c3-tag> to the Control Center release:

kind: ControlCenter
spec:
  image:
    application: confluentinc/cp-enterprise-control-center-next-gen:<c3-tag>
    init: confluentinc/confluent-init-container:3.3.0
  services:
    prometheus:
      image: confluentinc/cp-enterprise-prometheus:<c3-tag>
      pvc:
        dataVolumeCapacity: 10Gi
    alertmanager:
      image: confluentinc/cp-enterprise-alertmanager:<c3-tag>

Set dataVolumeCapacity to a capacity suitable for your environment. 10Gi is shown here only as an example.

If upgrading Control Center (Legacy) and you migrated from Operator 1.x to CFK 2.3.0 or earlier, and are now upgrading to CFK 2.4.0 or later with Confluent Platform 7.1 or earlier, set the following property:

Important

Starting with Confluent Platform version 8.0, Confluent Control Center (Legacy) is no longer supported with Confluent Platform. Use Control Center with Confluent Platform 8.0 and later.
```
spec:
  configOverrides:
    server:
      - confluent.controlcenter.internal.topics.partitions=4
```
Apply the component CR:
```
kubectl apply -f <component CR> -n <namespace>
```
Expected: <component>.platform.confluent.io/<name> configured
Verify the component is running:
```
kubectl get <component> -n <namespace>
```
Expected: STATUS shows RUNNING.

Next steps

After your Confluent Platform components are upgraded:

If you are migrating from Confluent Control Center (Legacy) to Control Center, or upgrading Control Center 2.0 or 2.1 to 2.2, see Migrate to and Upgrade Control Center.
For component-specific post-upgrade tasks and known issues, see Pre-upgrade tasks and known issues.

Upgrade Confluent Platform Using Confluent for Kubernetes

Known issues

Before you begin

Identify your deployment mode

Get your current CR files

Choose your upgrade path

Upgrade a ZooKeeper-based deployment to Confluent Platform 8.0

Upgrade Kafka and KRaft in the KRaft mode

Step 1: Upgrade KRaft controllers

Step 2: Upgrade Kafka brokers

Step 3: Update metadata version

Upgrade Kafka in the ZooKeeper mode

Automatic upgrade of Kafka

Step 1: (Optional) Override the default inter-broker version

Step 2: Verify the init container image tag

Step 3: Update the Kafka image tag

Step 4: Apply the Kafka CR

Step 5: Verify the final cluster versions

Manual upgrade of Kafka

Step 1: Update the Kafka image tag

Step 2: Verify the init container image tag

Step 3: Disable the automatic upgrade feature

Step 4: Set the current inter-broker protocol version

Step 5: Apply the Kafka CR

Step 6: Set the target inter-broker protocol version

Step 7: Apply the Kafka CR again

Upgrade other Confluent Platform components

Next steps

Step 4: Apply the `Kafka` CR

Step 5: Apply the `Kafka` CR

Step 7: Apply the `Kafka` CR again