Create a Kafka Cluster

Create an Apache Kafka® cluster in Confluent Cloud to start streaming. Confluent Cloud clusters are fully managed: you choose a cluster type and a cloud provider and region, and Confluent handles provisioning, scaling, and fault tolerance.

Get Started for Free

Sign up for a Confluent Cloud trial and get $400 of free credit.

You can create clusters using the Confluent Cloud Console, Confluent CLI, and REST API. For a description of cluster types, service level agreements (SLA), and resource quotas for clusters, see Kafka Cluster Types in Confluent Cloud and Service Quotas for Confluent Cloud.

Prerequisites:

• Operator or administrator access to a Confluent Cloud environment.

• For Freight clusters, you must contact Confluent and work with the sales team to determine a maximum eCKU for your cluster before you provision.

Considerations:

• Cloud provider and region cannot be changed after provisioning a cluster.

• To meet data residency and sovereignty requirements, data produced to a topic through Kafka stays within the geographic region you select. For more information, see the Confluent Cloud Security Controls whitepaper (PDF).

• For Dedicated clusters, the number of Confluent Unit for Kafka (CKU) determines the cluster’s capacity. You can manage CKUs after provisioning. For more information, see Fixed limits and recommended guidelines and Update Kafka clusters.

• Cluster name (display_name) is a friendly name with the following requirements:

  • Use 64 characters or less

  • Use whitespace, Unicode letters, numbers, and the following special characters: period (.), comma (,), ampersand (&), underscore (_), plus (+), bar (|), open square bracket ([), close square bracket (]), slash (/), dash (-)

Cloud Console

Select the tab that matches your cluster type.

Basic and Standard

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Basic or Standard and click Begin configuration.

3. Select a cloud provider tile, Region, and Uptime SLA and click Continue.

4. Specify Cluster name.

5. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Enterprise

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Enterprise and click Begin configuration.

3. Select a cloud provider tile, Region, and Uptime SLA and click Continue.

4. For network configuration, select I will configure later and click Continue to set up networking later or select a cloud provider to set up private networking now:

   1. Click Create configuration.

   2. On AWS, choose your mechanism: PrivateLink or Private Network Interface (PNI). On Azure and Google Cloud, PrivateLink is used automatically.

   3. Enter a gateway name, review your selected provider and region, and click Submit. If you chose Private Network Interface, also select three zones for your gateway before submitting.

   4. Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI.

      Provider	Mechanism	Connections
      AWS	PrivateLink	Inbound or outbound
      AWS	Private Network Interface	Inbound or outbound
      Azure	PrivateLink	Inbound or outbound
      Google Cloud	PrivateLink	Inbound only

      For more information, see the reference for your provider and mechanism:

      • For AWS PrivateLink, see Use AWS PrivateLink for Serverless Products on Confluent Cloud

      • For AWS Private Network Interface, see Use Private Network Interface on Confluent Cloud

      • For Azure PrivateLink, see Use Azure Private Link for Serverless Products on Confluent Cloud

      • For Google Cloud PrivateLink, see Use Google Cloud Private Service Connect for Serverless Products on Confluent Cloud

5. Specify Cluster name.

6. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Freight

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. Choose Freight and click Begin configuration.

3. Select a cloud provider tile, Region, and click Continue.

4. Select I will configure later and click Continue, or click + Create configuration to configure a gateway.

   If you choose to configure, take one of the following options:

   • Click Continue if a gateway for the selected region is available.

   • Enter a gateway name, review your selected provider and region, and then click Submit.

     Configure the networking using Terraform, the Confluent Cloud Console, or Confluent CLI.

5. Specify Cluster name.

6. Review your configuration and click Launch cluster.

   To verify or add a payment method, or use a promotional code, click Review payment method.

Dedicated

1. Navigate to the clusters page for your environment. If this is your first cluster, click Create cluster on my own. Otherwise, click + Add cluster.

2. In Dedicated, specify a cluster size or accept the default size of one CKU and click Begin configuration.

   Multi-zone availability requires two CKUs.

3. Select a cloud provider tile, Region, and Availability and click Continue.

4. Select Internet and click Continue, or choose a cloud provider to specify a private network:

   To use an existing private network

   1. Select your provider’s networking mechanism: AWS (Transit Gateway, VPC Peering, or PrivateLink), Azure (Private Link or VNet Peering), or Google Cloud (VPC Peering or Private Service Connect).

   2. Choose Use existing, select the existing network, and click Continue.

   To create a new private network

   1. Select your provider’s networking mechanism (see previous step).

   2. Choose Create new and enter Network name.

   3. If required for your mechanism, select your zones.

   4. Enter a CIDR block in CIDR for Confluent Cloud Network, or select Private DNS resolution to resolve your cluster endpoints using a private DNS zone, depending on your provider and mechanism:

      Provider	Mechanism	Zones	CIDR block	Private DNS resolution
      AWS	VPC Peering	Required	Required	Not supported
      AWS	Transit Gateway	Required	Required	Not supported
      AWS	PrivateLink	Required	Not applicable	Optional; otherwise public DNS resolution is used
      Azure	Private Link	Not applicable	Not applicable	Optional; otherwise public DNS resolution is used
      Azure	VNet Peering	Not applicable	Required	Not supported
      Google Cloud	VPC Peering	Required	Required	Not supported
      Google Cloud	Private Service Connect	Required	Not applicable	Optional; otherwise public DNS resolution is used

   5. Click Continue.

   For more information, see Manage Networking on Confluent Cloud.

5. Specify Automatic security and click Continue.

   For self-managed options, follow the on-screen steps to register your key. For more information, see Protect Data at Rest Using Self-Managed Encryption Keys on Confluent Cloud.

6. Specify Cluster name.

7. Review your configuration and click Launch cluster.

   

   To verify or add a payment method, or use a promotional code, click Review payment method.

Confluent CLI

To create a cluster using the CLI, run the following command in your terminal:

confluent kafka cluster create <name> [flags]

For example:

confluent kafka cluster create my_new_cluster --cloud "aws" --region "us-west-2"

Your output should resemble:

It may take up to 5 minutes for the Kafka cluster to be ready.
+----------------------+---------------------------------------------------------+
| Current              | false                                                   |
| ID                   | lkc-123exa                                              |
| Name                 | my_new_cluster                                          |
| Type                 | BASIC                                                   |
| Ingress Limit (MB/s) |                                                     250 |
| Egress Limit (MB/s)  |                                                     750 |
| Storage              | 5 TB                                                    |
| Cloud                | aws                                                     |
| Region               | us-west-2                                               |
| Availability         | single-zone                                             |
| Status               | PROVISIONING                                            |
| Endpoint             | SASL_SSL://pkc-exa45.us-west-2.aws.confluent.cloud:9092 |
| REST Endpoint        | https://pkc-exa45.us-west-2.aws.confluent.cloud:443     |
+----------------------+---------------------------------------------------------+

Confluent Cloud APIs

API requests must include an authorization header with an API key and secret. For more information, see Authentication in the Confluent Cloud API reference.

To create a cluster, make a POST request with a JSON payload that specifies the requirements.

POST /cmk/v2/clusters

Request

This request structure applies to all cluster types. When you make the request, include a JSON payload that contains a spec object with the following:

Parameter	Required or Optional	Description
display_name	Required	The display name of the cluster.
availability	Required	For Dedicated clusters: SINGLE_ZONE or MULTI_ZONE For Basic, Standard, or Enterprise clusters in organizations created on or after April 16, 2024: Low (99.5%/99.9%) or High (99.99%) For Basic, Standard, or Enterprise clusters in organizations created before April 16, 2024: SINGLE_ZONE or MULTI_ZONE
cloud	Required	GCP, AZURE or AWS
region	Required	A valid region for the cloud provider.
config	Required	Object that contains the cluster type: kind: Basic, Standard, Enterprise, Dedicated, or Freight. For Dedicated clusters, you must also specify a cku integer value, and you can specify an optional encryption_key for BYOK clusters. Clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic.
environment	Required	Object that contains the environment identifier: id: env-abc123.
network	Required for private network	Object that contains the network identifier: id: n-12345 and related environment name. To create a Dedicated cluster in a private network, first create the network, then specify that network when you create the cluster.

Request examples by cluster type

Example request for Basic and Standard clusters

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"Low",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Basic"
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

Example request for Enterprise clusters in a private network

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"High",
      "cloud":"AWS",
      "region":"us-east-1",
      "config":{
         "kind":"Enterprise"
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

Example requests for Dedicated clusters

On a secure public endpoint, specifying the number of CKUs in the config element:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

On a private network, specifying the network in the network element:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

Response

This response structure applies to all cluster types. Successful calls return HTTP 202 ACCEPTED with a JSON payload that describes the cluster.

Responses include the following:

• The cloud provider (AWS, GCP, AZURE) and region for the cluster.

• The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise, Dedicated, and Freight).

• Information about the environment that contains the cluster.

• For Dedicated clusters, the number of CKUs allocated to the cluster.

Response examples by cluster type

Example response for Basic and Standard clusters

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
      "api_version": "cmk/v2",
      "id": "abc-f3a90de",
      "kind": "Cluster",
      "metadata": {
         "created_at": "2022-04-22T20:45:26.657894Z",
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "updated_at": "2022-04-22T20:45:26.659364Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "Low",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
               "kind": "Basic"
         },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "api_version": "org/v2",
            "id": "env-a12b34",
            "kind":"Environment",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         }
      },
      "status": {
         "phase": "PROVISIONING"
      }
}

Example response for Enterprise clusters in a private network

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2023-06-22T20:45:26.657894Z",
      "updated_at": "2023-06-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "High",
      "cloud": "AWS",
      "region": "us-east-1",
      "config": {
         "kind": "Enterprise"
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east-1.aws.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east-1.aws.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment",
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING"
   }
}

Example responses for Dedicated clusters

On a secure public endpoint:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment",
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

In a private network:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment",
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

Terraform

To create a cluster using the Confluent Terraform provider, see confluent_kafka_cluster Resource.

Related content

• To create topics and produce and consume data, see Quick Start for Confluent Cloud

• For the CLI command reference, see confluent kafka cluster

• For the API reference, see Cluster API reference

• For the networking API reference, see Networking API reference

• For supported providers and regions, see Confluent Cloud Regions and Availability by Cloud Provider

• For BYOK encrypted clusters, see Protect Data at Rest Using Self-Managed Encryption Keys on Confluent Cloud

• For cost estimates, see Confluent Cost Estimator