Migrate to Confluent Cloud

You can migrate your existing Apache Kafka® workloads to Confluent Cloud from sources such as a self-managed Kafka deployment or Amazon Web Services (AWS) Managed Streaming for Kafka (MSK). A typical migration involves these steps:

Replicate your data to Confluent Cloud.
Migrate your schemas, if your clients use a Schema Registry, so that clients can serialize and deserialize data on the new cluster.
Migrate your connectors to Confluent Cloud managed connectors.
Switch over your clients to the new cluster.
Validate your migration and decommission the source cluster.

The following sections describe each step.

Tip

Before you migrate, plan your migration in the Confluent Cloud Console Migration hub. The Migration hub provides a step-by-step migration guide with a checklist to help you plan, run, and validate your migration. You must sign in to access this resource.

Replicate your data

Choose a data replication tool to copy your topic data to Confluent Cloud.

Cluster Linking (recommended for most migrations): Makes a byte-for-byte copy of your topics on Confluent Cloud and synchronizes consumer offsets between the clusters. It is fully managed and requires no extra infrastructure. For more information, see Migrate data with Cluster Linking on Confluent Cloud. To link a self-managed or on-premises cluster to Confluent Cloud, see Link Self-Managed and Confluent Cloud Clusters for Hybrid Cloud and Bridge-to-Cloud Deployments.
Confluent Replicator: Replicates topics and can repartition or reconfigure them during the migration. Use Replicator when Cluster Linking doesn’t support your source and destination cluster pairing. For more information, see Replicator Quick Start to Migrate Topic Data on Confluent Cloud and Example: Use Confluent Replicator to Copy Kafka Data to Confluent Cloud.
MirrorMaker 2: Replicates data, but has no managed option on Confluent Cloud, so use it only if you cannot use Cluster Linking or Replicator. For more information, see Kafka MirrorMaker.

Migrate your schemas

To migrate your schemas to Confluent Cloud, use Schema Linking, a fully managed service that copies schemas and keeps them in sync between two registries. This lets producers and consumers serialize and deserialize data on the new cluster. This step applies if your clients use a Schema Registry. For more information, see Schema Linking on Confluent Cloud.

Migrate your connectors

Move your self-managed connectors to Confluent Cloud managed connectors so that your data pipelines continue to run after the migration. For more information, see Migrate Self-Managed Connectors to Fully-Managed Connectors.

Switch over your clients

Choose a method to switch over your client applications to Confluent Cloud.

Important

Before you onboard clients to Confluent Gateway, make sure they use a supported authentication method. Clients that authenticate to MSK with AWS Identity and Access Management (IAM) must move to SASL/SCRAM or mutual TLS (mTLS) first.

Zero-cut migration (recommended for minimal downtime): Confluent Gateway routes traffic and maps authentication so clients keep their original endpoints and credentials and reconnect automatically at cutover, with no configuration changes. For MSK migrations, the kcp tool automates this process. For more information, see Migrate between Kafka Clusters using Confluent Cloud Gateway and Migrate Clusters with the Copy Paste (kcp) Tool.
Client restart: Update each client’s bootstrap server and security configuration and restart it against Confluent Cloud. Several migration patterns are available, such as restart all at once, stop-wait-restart, and stop-restart-repeat, depending on how much downtime and how many duplicate or missed messages your clients can tolerate. For the Cluster Linking cutover steps, see Migrate data with Cluster Linking on Confluent Cloud.

Validate and decommission

To validate your migration, confirm that your workloads run as expected on Confluent Cloud and meet your performance requirements after you switch over your clients. After validation, decommission the source cluster and any temporary infrastructure you created for the migration.