Create a Kafka Cluster in Confluent Cloud

Create an Apache Kafka® cluster in Confluent Cloud to start streaming. Confluent Cloud clusters are fully managed: you choose a cluster type and a cloud provider and region, and Confluent handles provisioning, scaling, and fault tolerance.

Get Started for Free

Sign up for a Confluent Cloud trial and get $400 of free credit.

Requirements

• Operator or administrator access to a Confluent Cloud environment.

• API requests must include an authorization header with an API key and secret. For more information, see Authentication in the Confluent Cloud API reference.

You can create clusters using the Confluent Cloud Console, Confluent CLI, and REST API. For a description of cluster types, service level agreements (SLA), and resource quotas for clusters, see Kafka Cluster Types in Confluent Cloud and Service Quotas for Confluent Cloud.

Considerations:

• Cloud provider and region cannot be changed after provisioning a cluster.

• For data residency and sovereignty requirements, data you produce to a topic and send through Kafka does not move out of the geographic region you select. For more information, see the whitepaper (PDF) Confluent Cloud Security Controls.

• Cluster name (display_name) is a friendly name with the following requirements:

  • Use 64 characters or less

  • Use whitespace, Unicode letters, numbers, and the following special characters: Period (.), Comma (,), Ampersand (&), Underscore (_), Plus (+), Bar (|), Open square bracket ([), Close square bracket (]), Slash (/), Dash (-)

  Tip

  The business use of a cluster is a good choice for cluster name.

Cloud Console

Choose a cluster type.

Basic and Standard

1. Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Basic or Standard and click Begin configuration.

3. Select a cloud provider tile, Region, and Uptime SLA and click Continue.

4. Specify Cluster name.

5. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Enterprise

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Enterprise and click Begin configuration.

3. Select a cloud provider tile, Region, and Uptime SLA and click Continue.

4. For network configuration, select I will configure later and click Continue to setup networking later or select a cloud provider to set up private networking now:

   AWS

   AWS private network options:

   PrivateLink

   To create inbound or outbound connections

   1. Click Create configuration.

   2. Enter a gateway name, review your selected provider and region, and then click Submit.

   3. Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI. For more information, see Use AWS PrivateLink for Serverless Products on Confluent Cloud.

   Private Network Interface

   To create a Private Network Interface connection

   1. Click Create configuration.

   2. Enter a gateway name, review your selected provider and region, and select three zones for your gateway.

   3. Click Submit.

   4. Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI. For more information, see Use Private Network Interface on Confluent Cloud.

   Azure

   To create inbound or outbound connections

   1. Click Create configuration.

   2. Enter a gateway name, review your selected provider and region, and then click Submit.

   3. Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI. For more information, see Use Azure Private Link for Serverless Products on Confluent Cloud.

   GCP

   To create inbound connections

   1. Click Create configuration.

   2. Enter a gateway name, review your selected provider and region, and then click Submit.

   3. Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI. For more information, see Use Google Cloud Private Service Connect for Serverless Products on Confluent Cloud.

5. Specify Cluster name.

6. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Freight

You must contact Confluent to create Freight clusters. With Freight clusters, you first work with the sales team to determine a maximum eCKU for your cluster before you provision.

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Freight and click Begin configuration.

3. Select a cloud provider tile, Region, and click Continue.

4. Select I will configure later and click Continue, or click + Create configuration to configure a gateway.

   If you choose to configure, take one of the following options:

   • Click Continue if a gateway for the selected region is available.

   • Enter a gateway name, review your selected provider and region, and then click Submit.

     Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI.

5. Specify Cluster name.

6. Review your configuration and click Launch cluster.

   To verify or add a payment method, or use a promotional code, click Review payment method.

Dedicated

Considerations:

• The number of Confluent Unit for Kafka (CKU) determines the capacity of Dedicated clusters. You can manage CKUs after provisioning. For more information on CKUs and how to update them, see Fixed limits and recommended guidelines and Update Kafka clusters.

• To review network options, see Manage Networking on Confluent Cloud.

• To encrypt your at-rest Kafka cluster data to ensure only the appropriate entity or user can decrypt it, use the cluster creation steps in Protect Data at Rest Using Self-Managed Encryption Keys on Confluent Cloud.

To create a Dedicated cluster

1. Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. In Dedicated, specify a cluster size or accept the default size of one CKU and click Begin configuration.

   Multi-zone availability requires two CKUs.

3. Select a cloud provider tile, Region, and Availability and click Continue.

4. Select Internet and click Continue, or choose a cloud provider to specify a private network:

   AWS

   AWS private network options:

   Use existing private network

   1. Select Transit Gateway, VPC Peering, or PrivateLink.

   2. Choose Use existing, select the existing network, and click Continue.

   Create new private network

   VPC Peering

   1. Select VPC Peering.

   2. Choose Create new and enter Network name.

   3. Select your zones.

   4. Enter a CIDR block in CIDR for Confluent Cloud Network.

   5. Click Continue.

   Transit Gateway

   1. Select Transit Gateway.

   2. Choose Create new and enter Network name.

   3. Select your zones.

   4. Enter a CIDR block in CIDR for Confluent Cloud Network.

   5. Click Continue.

   PrivateLink

   1. Select PrivateLink.

   2. Choose Create new and enter Network name.

   3. Select your zones.

   4. (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.

   5. Click Continue.

   Azure

   Azure private network options:

   Use existing private network

   1. Select Private Link or VNet Peering.

   2. Choose Use existing, select the existing network, and click Continue.

   Create new private network

   Private Link

   1. Choose Private Link.

   2. Choose Create new and enter Network name.

   3. (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.

   4. Click Continue.

   VNet Peering

   1. Choose VNet Peering.

   2. Choose Create new and enter Network name.

   3. Enter a CIDR block in CIDR for Confluent Cloud Network.

   4. Click Continue.

   GCP

   GCP private network options:

   Use existing private network

   1. Select VPC Peering, or Private Service Connect.

   2. Choose Use existing, select the existing network, and click Continue.

   Create new private network

   VPC Peering

   1. Select VPC Peering.

   2. Choose Create new and enter Network name.

   3. Select your zones.

   4. Enter a CIDR block in CIDR for Confluent Cloud Network.

   5. Click Continue.

   Private Service Connect

   1. Select Private Service Connect.

   2. Choose Create new and enter Network name.

   3. Select your zones.

   4. (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.

   5. Click Continue.

   For more information, see Manage Networking on Confluent Cloud.

5. Specify Automatic security and click Continue.

   For self-managed options, follow the on-screen steps to register your key. For more information, see Protect Data at Rest Using Self-Managed Encryption Keys on Confluent Cloud.

6. Specify Cluster name.

7. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Confluent CLI

Create a cluster.

confluent kafka cluster create <name> [flags]

For example:

confluent kafka cluster create my_new_cluster --cloud "aws" --region "us-west-2"

It may take up to 5 minutes for the Kafka cluster to be ready.
+----------------------+---------------------------------------------------------+
| Current              | false                                                   |
| ID                   | lkc-123exa                                              |
| Name                 | my_new_cluster                                          |
| Type                 | BASIC                                                   |
| Ingress Limit (MB/s) |                                                     250 |
| Egress Limit (MB/s)  |                                                     750 |
| Storage              | 5 TB                                                    |
| Cloud                | aws                                                     |
| Region               | us-west-2                                               |
| Availability         | single-zone                                             |
| Status               | PROVISIONING                                            |
| Endpoint             | SASL_SSL://pkc-exa45.us-west-2.aws.confluent.cloud:9092 |
| REST Endpoint        | https://pkc-exa45.us-west-2.aws.confluent.cloud:443     |
+----------------------+---------------------------------------------------------+

Confluent Cloud APIs

To create a cluster, make a POST request with a JSON payload that specifies the requirements.

POST /cmk/v2/clusters

Request (all cluster types)

When you make the request, include a JSON payload that contains a spec object with the following:

Parameter	Required or Optional	Description
display_name	Required	The display name of the cluster.
availability	Required	Uptime for Basic, Standard, Enterprise, and Freight: Low (99.5%/99.9%) or High (99.99%) For Dedicated: SINGLE_ZONE or MULTI_ZONE Availability changes for Basic/Standard/Enterprise clusters Beginning April 16, 2024, the availability parameter for Basic, Standard, and Enterprise clusters will change to Low and High instead of SINGLE_ZONE or MULTI_ZONE. These changes are only applicable to Confluent Cloud organizations created on or after April 16, 2024. All organizations created before this date are not impacted and will continue to use their existing availability parameter. Legacy availability for Basic, Standard, and Enterprise clusters The cluster availability: ``SINGLE_ZONE`` or ``MULTI_ZONE``.
cloud	Required	GCP, AZURE or AWS
region	Required	A valid region for the cloud provider.
config	Required	Object that contains the cluster type: kind: Basic, Standard, Enterprise, Dedicated, or Freight. For Dedicated clusters, you must also specify a cku integer value, and you can specify an optional encryption_key for BYOK clusters. Clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic.
environment	Required	Object that contains the environment identifier: id: env-abc123.
network	Required for private network	Object that contains the network identifier: id: n-12345 and related environment name.

If the Dedicated cluster should be created in a private network, create the network first, then specify the network when you create the cluster.

Basic and Standard

The following example demonstrates the request format for Basic and Standard clusters:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"Low",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Basic"
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

Enterprise

The following example demonstrates the request format for an Enterprise cluster in a private network.

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"High",
      "cloud":"AWS",
      "region":"us-east-1",
      "config":{
         "kind":"Enterprise"
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

Dedicated

The following examples demonstrate the request format for Dedicated clusters.

The following example demonstrates the request on a secure public endpoint. You specify the number of CKUs in the config element:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

The following example demonstrates the request format for a private network. You specify the network in which to create the cluster in the network element.

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

Response

Successful calls return HTTP 202 ACCEPTED with a JSON payload that describes the cluster.

Responses include the following information:

• The cloud provider (AWS, GCP, AZURE) and region for the cluster.

• The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise, Dedicated, and Freight).

• Information about the environment that contains the cluster.

• Dedicated clusters only - the number of CKUs allocated to the cluster.

Basic and Standard

Example response for Basic and Standard clusters:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
      "api_version": "cmk/v2",
      "id": "abc-f3a90de",
      "kind": "Cluster",
      "metadata": {
         "created_at": "2022-04-22T20:45:26.657894Z",
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "updated_at": "2022-04-22T20:45:26.659364Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "Low",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
               "kind": "Basic"
         },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "api_version": "org/v2",
            "id": "env-a12b34",
            "kind":"Environment"
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         }
      },
      "status": {
         "phase": "PROVISIONING"
      }
}

Enterprise

Example response for Enterprise clusters in a private network

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2023-06-22T20:45:26.657894Z",
      "updated_at": "2023-06-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "High",
      "cloud": "AWS",
      "region": "us-east-1",
      "config": {
         "kind": "Enterprise"
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east-1.aws.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east-1.aws.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING"
   }
}

Dedicated

Example response on a secure public endpoint:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

Example response in a private network:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

Related content

• Learn how to use Confluent Cloud to create topics, produce and consume to a Kafka cluster with the Quick Start for Confluent Cloud

• For more information about Confluent CLI, see confluent kafka cluster in the Confluent CLI Command Reference

• For more information about Confluent Cloud APIs, see Cluster API reference

• For more information about creating a network using an API, see the Networking API reference

• For more information about supported providers and regions, see Confluent Cloud Regions and Availability by Cloud Provider

• For more information about BYOK encrypted clusters, see Protect Data at Rest Using Self-Managed Encryption Keys on Confluent Cloud

• How to use kafka-consumer-groups command with Confluent Cloud (Confluent Support)

• For cost estimates, see Confluent Cost Estimator

• For information about migrating from open source Kafka to Confluent Cloud, see the Migrating from Kafka services to Confluent PDF