Upgrade Confluent Platform Using Confluent for Kubernetes

A Confluent for Kubernetes deployment includes:

  • Confluent for Kubernetes (CFK) Operator
  • Confluent Platform component clusters

You can upgrade either CFK or Confluent Platform independently from each other.

When you are planning for an upgrade, review the following so that you can chart the appropriate upgrade path:

The following upgrade paths are supported:

To upgrade from Confluent Operator 1.x to Confluent for Kubernetes, see Migrate to Confluent for Kubernetes.

Upgrade considerations

CFK and Confluent Platform compatibility support

Starting in CFK 2.9, your upgrades get simplified from the CFK-Confluent Platform compatibility standpoint as CFK supports all supported Confluent Platform versions.

For the CFK versions that are older than 2.9, when you upgrade both CFK and Confluent Platform, a CFK and Confluent Platform combination that is not listed in the support matrix will be still supported until you reach the final state where CFK and Confluent Platform are both upgraded to a supported combination.

For example, suppose you are upgrading CFK from 2.6.1 to 2.9.3 along with Confluent Platform from 7.3.1 to 7.7.1. In that case, the combination of CFK 2.9.3 and Confluent Platform 7.3.1 will be temporarily supported during the upgrade even though it is not a supported combination in the support matrix table.

Disable resource reconciliation

There are configuration changes in CFK 2.9.3, and the changes will cause the existing Confluent Platform clusters to roll during the CFK upgrade process.

Before upgrading CFK, you must disable resource reconciliation as described in Upgrade CFK.

Upgrade default auto-generated certificates to configurable auto-generated certificates

By default, Confluent for Kubernetes (CFK) uses auto-generated certificates without configuration options.

Starting in CFK 2.4.0, you can use configurable certificates that CFK generates with the options you specify.

To upgrade the certificates that were auto-generated in CFK 2.3.x or earlier to configurable auto-managed certificates in 2.4.x or later, follow the below steps. For details, see Use auto-generated TLS certificates.

  1. Set managedCerts.enabled: true and managedCerts.caCertificate.secretRef=<secretName> in the CFK Helm values file.
  2. Set the other configuration settings under managedCerts as described in Use auto-generated TLS certificates.

CFK will renew the certificates based on the CA certificates and the key present in managedCerts.caCertificate.secretRefs, and will roll the corresponding Confluent clusters.

Change of CA is not supported while moving from the legacy to the configurable certificates mode.

Upgrade Kafka clusters involved in Cluster Linking

When you upgrade the KRaft-based Kafka clusters you use in Cluster Linking, upgrade the brokers on the destination cluster first, and then upgrade the metadata version on the source cluster.

Connect pods restart when upgrading to CFK 2.9 and later

In CFK 2.9, a configuration property in the Connect stateful set template was updated, and the change causes the Connect cluster to roll when you upgrade CFK to 2.9 and later.