Confluent Developer Marketplace Get started
  1. Home
  2. Cloud
  3. Confluent Cloud Clusters
  4. Migrate to Confluent Cloud

Migrate Clusters with the Copy Paste (kcp) Tool

Copy Paste (kcp) is a command-line tool for migrating existing Apache Kafka® cluster deployments to Confluent Cloud. kcp provides a set of features that help you analyze, plan, and execute migrations efficiently.

kcp supports migrating from two source types:

  • Amazon Web Services (AWS) Managed Streaming for Kafka (MSK), including MSK Provisioned/Express and MSK Serverless clusters.

  • Kafka: any Kafka API-compatible source, reached directly through the Kafka Admin API.

Not every kcp command supports every source type or flavor. For the full list of which commands work with which source, see the Source Compatibility matrix in the kcp documentation.

To evaluate and plan your migration before you use kcp, you can start in the Migration hub, which you open from the navigation menu in the Confluent Cloud Console. The Migration hub provides resources to help you get started, including a step-by-step migration guide and free help from a migration expert. You must sign in to access this resource.

What kcp does

The kcp tool streamlines the migration process through these main functions:

  • Environment initialization: kcp helps initialize the local environment with required configuration variables, helping to simplify running subsequent commands.

  • Scanning and discovery: kcp scans and identifies resources within your existing Kafka deployments. This includes detailed information on clusters, brokers, topics, and consumer groups.

  • Report generation: kcp creates reports that provide migration planning and cost analysis. These reports detail cluster configurations, metrics, and other critical information.

  • Asset generation: kcp generates the necessary migration assets and infrastructure configurations, which can be provisioned using tools like Terraform.

  • Client migration: kcp migrates client applications to Confluent Cloud using Confluent Cloud Gateway (Confluent Gateway), redirecting traffic from the source Kafka cluster to Confluent Cloud without requiring changes to client configurations. For more information, see Migrate between Kafka Clusters using Confluent Cloud Gateway.

Set up kcp

To set up kcp, first determine whether your existing Kafka clusters are reachable from the internet or only from a private network, because this determines where you run kcp from.

  • Internet-accessible clusters: Run kcp commands directly from your local computer.

  • Private-network clusters: Run kcp from a bastion host or jump server in the same VPC as your clusters. If you don’t have one, use the create-asset command described in the Command Reference.

Next, install kcp. It’s available for macOS, Linux, and Windows. For installation instructions, including the recommended install script and manual download steps for each platform, see Installation in the kcp documentation.

Finally, authenticate kcp against your source cluster. For MSK migrations, kcp authenticates against AWS using the standard AWS credential chain. For Kafka migrations, kcp authenticates directly against the source cluster’s Kafka Admin API using an apache-kafka-credentials.yaml file. For the full list of supported AWS authentication methods and file formats, see Authentication in the kcp documentation.

kcp command reference

The kcp tool is structured around these central commands: help, discover, scan, report, create-asset, migration, and ui. For full command and flag details, see Command Reference in the kcp documentation.

Related content