Manage Kafka Clusters on Confluent Cloud¶

Use this topic to understand and manage clusters in Confluent Cloud, using the Confluent Cloud Console, Confluent CLI, and REST API.

Kafka cluster management concepts¶

Apache Kafka® clusters in Confluent Cloud are different from Kafka clusters in general in that Confluent offers a fully managed service. Use the following concepts to learn more.

Basics

You choose from different types of clusters for use in development, production, and high-traffic workloads. Confluent Cloud is available from all the major cloud providers in regions around the world and provides access to Kafka cluster and topic configuration settings. For more information, see the following topics:

Fault Tolerance

Confluent Cloud provides a centralized global control plane and a distributed collection of servers that operate as a cluster. Clusters can span data centers or cloud provider availability zones (AZs) and provide highly scalable and fault-tolerant systems that support mission-critical applications. For more information, see Resilience in Confluent Cloud.

Dedicated Clusters

Dedicated clusters in Confluent Cloud support options for private networking, self-managed keys, and client quotas for multi-tenant workloads. You provision and manage Dedicated clusters in terms of Confluent Unit for Kafka (CKU). Use CKUs to expand and shrink your cluster. For more information, see the following topics:

For more information about private networking, see Manage Networking on Confluent Cloud.

How to work with Kafka clusters¶

Use this section to create, delete, and update clusters.

Requirements

Operator or admin access to a Confluent Cloud environment.
API requests must include an authorization header with an API key and secret. For more information, see Authentication in the Confluent Cloud API reference.

Create Kafka clusters¶

You can create clusters using the Confluent Cloud Console, Confluent CLI, and REST API. For a description of cluster types and resource quotas for clusters, see Kafka Cluster Types in Confluent Cloud and Service Quotas for Confluent Cloud.

Considerations:

Cloud provider and region cannot be changed after provisioning a cluster.
For data residency and sovereignty requirements, data you produce to a topic and transmit through Kafka does not move out of the geographic region you select. For more information, see the white paper (PDF) Confluent Cloud Security Controls.
Cluster name (display_name) is a friendly name with the following requirements:
- Use 64 characters or less
- Use whitespace, Unicode letters, numbers, and the following special characters: Period (.), Comma (,), Ampersand (&), Underscore (_), Plus (+), Bar (|), Open square bracket ([), Close square bracket (]), Slash (/), Dash (-)
Tip

The business use of a cluster is a good choice for cluster name.

Choose a cluster type.

Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.
Choose Basic or Standard and click Begin configuration.
Select a cloud provider tile, Region, and Uptime SLA and click Continue.
Specify Cluster name.
Review your configuration and click Launch cluster.

To verify or add a payment method, or use a promotional code, click Review payment method.

Considerations:

The number of Confluent Unit for Kafka (CKU) determines the capacity of Dedicated clusters. You can manage CKUs after provisioning. For more information on CKUs and how to update them, see Limits per CKU and Update Kafka clusters.
To review network options, see Manage Networking on Confluent Cloud.
To encrypt your at-rest Kafka cluster data to ensure only the appropriate entity or user can decrypt it, use the cluster creation steps in Protect Data at Rest Using Self-managed Encryption Keys on Confluent Cloud.

To create a Dedicated cluster

Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.
In Dedicated, specify a cluster size or accept the default size of one CKU and click Begin configuration.

Multi-zone availability requires two CKUs.
Select a cloud provider tile, Region, and Availability and click Continue.
Select Internet and click Continue, or choose a cloud provider to specify a private network:
AWS private network options:
Select Transit Gateway, VPC Peering, or PrivateLink.

Choose Use existing, select the existing network, and click Continue.
Select VPC Peering.

Choose Create new and enter Network name.

Select your zones.

Enter a CIDR block in CIDR for Confluent Cloud Network.

Click Continue.
Azure private network options:
Select Private Link or VNet Peering.

Choose Use existing, select the existing network, and click Continue.
Choose Private Link.

Choose Create new and enter Network name.

(Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.

Click Continue.
GCP private network options:
Select VPC Peering, or Private Service Connect.

Choose Use existing, select the existing network, and click Continue.
Select VPC Peering.

Choose Create new and enter Network name.

Select your zones.

Enter a CIDR block in CIDR for Confluent Cloud Network.

Click Continue.
For more information, see Manage Networking on Confluent Cloud.
Specify Automatic security and click Continue.

For self-managed options, follow the on-screen steps to register your key. For more information, see Protect Data at Rest Using Self-managed Encryption Keys on Confluent Cloud.
Specify Cluster name.
Review your configuration and click Launch cluster.

To verify or add a payment method, or use a promotional code, click Review payment method.

Create a cluster.

confluent kafka cluster create <name> [flags]

For example:

confluent kafka cluster create my_new_cluster --cloud "aws" --region "us-west-2"

It may take up to 5 minutes for the Kafka cluster to be ready.
+----------------------+---------------------------------------------------------+
| Current              | false                                                   |
| ID                   | lkc-123exa                                              |
| Name                 | my_new_cluster                                          |
| Type                 | BASIC                                                   |
| Ingress Limit (MB/s) |                                                     250 |
| Egress Limit (MB/s)  |                                                     750 |
| Storage              | 5 TB                                                    |
| Cloud                | aws                                                     |
| Region               | us-west-2                                               |
| Availability         | single-zone                                             |
| Status               | PROVISIONING                                            |
| Endpoint             | SASL_SSL://pkc-exa45.us-west-2.aws.confluent.cloud:9092 |
| REST Endpoint        | https://pkc-exa45.us-west-2.aws.confluent.cloud:443     |
+----------------------+---------------------------------------------------------+

To create a cluster, make a POST request with a JSON payload that specifies the requirements.

POST /cmk/v2/clusters

Request (all cluster types)

When you make the request, include a JSON payload that contains a spec object with the following:

Parameter	Required or Optional	Description
`display_name`	Required	The display name of the cluster.
`availability`	Required	Uptime for Basic, Standard, and Enterprise: `Low` (99.5%/99.9%) or `High` (99.99%) For Dedicated: `SINGLE_ZONE` or `MULTI_ZONE` Availability changes for Basic/Standard/Enterprise clusters Beginning 4/16/2024, the `availability` parameter for Basic, Standard and Enterprise clusters will change to `Low` and `High` instead of `SINGLE_ZONE` or `MULTI_ZONE`. These changes are only applicable to Confluent Cloud organizations created on or after 4/16/2024. All organizations created before this date are not impacted and will continue to utilize their existing `availability` parameter. Legacy availability for Basic, Standard and Enterprise clusters The cluster availability: ``SINGLE_ZONE`` or ``MULTI_ZONE``.
`cloud`	Required	`GCP`, `AZURE` or `AWS`
`region`	Required	A valid region for the cloud provider.
`config`	Required	Object that contains the cluster type: `kind`: `Basic`, `Standard`, `Enterprise`, or `Dedicated`. For Dedicated clusters, you must also specify a `cku` integer value, and you can specify an optional `encryption_key` for BYOK clusters. Note that clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic.
`environment`	Required	Object that contains the environment identifier: `id`: `env-abc123`.
`network`	Required for private network	Object that contains the network identifier: `id`: `n-12345` and related environment name.

If the Dedicated cluster should be created in a private network, create the network first, then specify the network when you create the cluster.

The following example demonstrates the request format for Basic, Standard, and Enterprise clusters:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"Low",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Basic"
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

The following example demonstrates the request format for an Enterprise cluster in a private network.

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"High",
      "cloud":"AWS",
      "region":"us-east-1",
      "config":{
         "kind":"Enterprise"
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

The following examples demonstrate the request format for Dedicated clusters.

The example below demonstrates the request on a secure public endpoint. Note that you specify the number of CKUs in the config element:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      }
   }
}

The following example demonstrates the request format for a private network. Note that you specify the network in which to create the cluster in the network element.

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
      "display_name":"ProdKafkaCluster",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 2
      },
      "environment":{
         "id":"env-a12b34"
      },
      "network":{
         "id":"n-12345",
         "environment": "env-a12b34"
      }
   }
}

Response

Successful calls return HTTP 202 ACCEPTED with a JSON payload that describes the cluster.

Responses include the following information:

The cloud provider (AWS, GCP, AZURE) and region for the cluster.
The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise and Dedicated).
Information about the environment that contains the cluster.
Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Basic and Standard clusters:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
      "api_version": "cmk/v2",
      "id": "abc-f3a90de",
      "kind": "Cluster",
      "metadata": {
         "created_at": "2022-04-22T20:45:26.657894Z",
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "updated_at": "2022-04-22T20:45:26.659364Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "Low",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
               "kind": "Basic"
         },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "api_version": "org/v2",
            "id": "env-a12b34",
            "kind":"Environment"
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         }
      },
      "status": {
         "phase": "PROVISIONING"
      }
}

Example response for Enterprise clusters in a private network

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2023-06-22T20:45:26.657894Z",
      "updated_at": "2023-06-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "High",
      "cloud": "AWS",
      "region": "us-east-1",
      "config": {
         "kind": "Enterprise"
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east-1.aws.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east-1.aws.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING"
   }
}

Example response on a secure public endpoint:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

Example response in a private network:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "Cluster",
   "id": "abc-f3a90de",
   "metadata": {
      "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec": {
      "display_name": "ProdKafkaCluster",
      "availability": "SINGLE_ZONE",
      "cloud": "GCP",
      "region": "us-east4",
      "config": {
         "kind": "Dedicated",
         "cku": 2
      },
      "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind":"Environment"
         "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
         },
         "network": {
         "id": "n-00000",
         "environment": "env-a12b34",
         "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
         "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
         "api_version": "networking/v1",
         "kind": "Network"
         }
      },
   "status": {
      "phase": "PROVISIONING",
      "cku": 2
   }
}

View Kafka clusters¶

You can view details of a single cluster and lists multiple clusters using the Confluent Cloud Console, Confluent CLI, and REST API.

View a list of Kafka clusters¶

Use this procedure to list clusters.

To list clusters

From the Administration menu, select Environments.
Select an environment.

A list of clusters appears.

List clusters.

confluent kafka cluster list [flags]

For example:

confluent kafka cluster list

  Current |     ID     |        Name        |   Type   | Cloud    |  Region   | Availability | Status
----------+------------+--------------------+----------+----------+-----------+--------------+---------
          | lkc-123exc | cluster_1          | BASIC    | gcp      | us-west4  | single-zone  | UP
          | lkc-123exb | cluster_0          | STANDARD | gcp      | us-west4  | single-zone  | UP
          | lkc-123exa | my_new_cluster     | BASIC    | aws      | us-west-2 | single-zone  | UP

To list clusters, make a GET request that specifies the environment.

Request

GET /cmk/v2/clusters?environment={environment_id}

Your request can include the following parameters:

Parameter	Required or Optional	Description
`environment`	Required	Specifies the identifier for the environment that contains the clusters to list.
`page_size`	Optional	Specifies an integer page size for the returned payload. Max value is 100. Default: 10.
`spec.network`	Optional	A network identifier for the clusters to list. Your query can contain multiple network identifiers.
`page_token`	Optional	Contains the opaque pagination token string.

Response

Successful calls return HTTP 200 OK and a JSON payload that contains the list of clusters for the environment.

Responses include the following information:

The cloud provider (AWS, GCP, AZURE) and region for the cluster.
The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise and Dedicated).
Information about the environment that contains the cluster.
Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Basic and Standard clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "data": [
      {
      "api_version": "cmk/v2",
      "kind": "Cluster",
      "id": "abc-f3a90de",
      "metadata": {
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at": "2022-04-22T20:45:26.657894Z",
         "updated_at": "2022-04-22T21:13:55.742641944Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "Low",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
               "kind": "Basic"
         },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "id": "env-a12b34",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
            "api_version": "org/v2",
            "kind": "Environment"
         }
      },
      "status": {
         "phase": "PROVISIONED"
      }
   }
   ],
   "kind": "ClusterList",
   "metadata": {
   "first": "https://api.confluent.cloud/cmk/v2/clusters",
   "total_size": 1
   }
}

Example response for Enterprise clusters in a private network:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "data": [
      {
      "api_version": "cmk/v2",
      "kind": "Cluster",
      "id": "abc-f3a90de",
      "metadata": {
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at": "2022-04-22T20:45:26.657894Z",
         "updated_at": "2022-04-22T21:13:55.742641944Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "High",
         "cloud": "AWS",
         "region": "us-east-1",
         "config": {
            "kind": "Enterprise"
            },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "id": "env-a12b34",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
            "api_version": "org/v2",
            "kind": "Environment"
            },
         "network": {
            "id": "n-00000",
            "environment": "env-a12b34",
            "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34,
            "api_version": "networking/v1",
            "kind": "Network"
            }
      },
      "status": {
         "phase": "PROVISIONED"
      }
   }
   ],
   "kind": "ClusterList",
   "metadata": {
   "first": "https://api.confluent.cloud/cmk/v2/clusters",
   "total_size": 1
   }
}

Example response on a secure public endpoint:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "data": [
      {
      "api_version": "cmk/v2",
      "kind": "Cluster",
      "id": "abc-f3a90de",
      "metadata": {
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at": "2022-04-22T20:45:26.657894Z",
         "updated_at": "2022-04-22T21:13:55.742641944Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "SINGLE_ZONE",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
               "kind": "Basic"
         },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "id": "env-a12b34",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
            "api_version": "org/v2",
            "kind": "Environment"
         }
      },
      "status": {
         "phase": "PROVISIONED",
         "cku": 2
      }
   }
   ],
   "kind": "ClusterList",
   "metadata": {
   "first": "https://api.confluent.cloud/cmk/v2/clusters",
   "total_size": 1
   }
}

Example response in a private network:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "data": [
      {
      "api_version": "cmk/v2",
      "kind": "Cluster",
      "id": "abc-f3a90de",
      "metadata": {
         "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at": "2022-04-22T20:45:26.657894Z",
         "updated_at": "2022-04-22T21:13:55.742641944Z"
      },
      "spec": {
         "display_name": "ProdKafkaCluster",
         "availability": "SINGLE_ZONE",
         "cloud": "GCP",
         "region": "us-east4",
         "config": {
            "kind": "Basic"
            },
         "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment": {
            "id": "env-a12b34",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34",
            "api_version": "org/v2",
            "kind": "Environment"
            },
         "network": {
            "id": "n-00000",
            "environment": "env-a12b34",
            "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34,
            "api_version": "networking/v1",
            "kind": "Network"
            }
      },
      "status": {
         "phase": "PROVISIONED",
         "cku": 2
      }
   }
   ],
   "kind": "ClusterList",
   "metadata": {
   "first": "https://api.confluent.cloud/cmk/v2/clusters",
   "total_size": 1
   }
}

View a Kafka cluster¶

Use this procedure to get details about a cluster.

To list a cluster

Navigate to the clusters page for your environment.
Select Cluster Settings to see details about your cluster. To see capacity details, click Capacity.

Describe a cluster.

confluent kafka cluster describe [id] [flags]

For example:

confluent kafka cluster describe lkc-123exa

+----------------------+---------------------------------------------------------+
| Current              | true                                                    |
| ID                   | lkc-123exa                                              |
| Name                 | my_new_cluster                                          |
| Type                 | BASIC                                                   |
| Ingress Limit (MB/s) |                                                     250 |
| Egress Limit (MB/s)  |                                                     750 |
| Storage              | 5 TB                                                    |
| Cloud                | aws                                                     |
| Region               | us-west-2                                               |
| Availability         | single-zone                                             |
| Status               | UP                                                      |
| Endpoint             | SASL_SSL://pkc-exa45.us-west-2.aws.confluent.cloud:9092 |
| REST Endpoint        | https://pkc-exa45.us-west-2.aws.confluent.cloud:443     |
+----------------------+---------------------------------------------------------+

To list a cluster, make a GET request that specifies the cluster and environment.

Request (all cluster types)

GET /cmk/v2/clusters/{id}?environment={environment_id}

Your request can include the following parameters:

Parameter	Required or Optional	Description
`id`	Required	Path parameter specifying the identifier for the cluster to read.
`environment`	Required	Query parameter specifying the identifier for the environment that contains the cluster to read.

Response

Successful calls return HTTP 200 OK and a JSON payload that describes the cluster.

Responses include the following information:

The cloud provider (AWS, GCP, AZURE) and region for the cluster.
The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise, Dedicated).
Info about the environment that contains the cluster.
Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Basic and Standard clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
      "api_version":"cmk/v2",
      "kind":"Cluster",
      "id":"abc-f3a90de",
      "metadata":{
         "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at":"2022-04-22T20:45:26.657894Z",
         "updated_at":"2022-04-22T20:45:28.045579Z"
      },
      "spec":{
         "display_name":"ProdKafkaCluster",
         "availability":"Low",
         "cloud":"GCP",
         "region":"us-east4",
         "config":{
               "kind":"Standard"
         },
         "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment":{
               "api_version": "org/v2",
               "id": "env-a12b34",
               "kind": "Environment",
               "related":"https://api.confluent.cloud/v2/environments/env-a12b34",
               "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         }
      },
      "status":{
         "phase":"PROVISIONED"
      }
}

Example response for Enterprise clusters on a private network:

HTTP/1.1 200 OK
Content-Type: application/json

{
      "api_version":"cmk/v2",
      "kind":"Cluster",
      "id":"abc-f3a90de",
      "metadata":{
         "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at":"2006-01-02T15:04:05-07:00",
         "updated_at":"2006-01-02T15:04:05-07:00"
      },
      "spec":{
         "display_name":"ProdKafkaCluster",
         "availability":"High",
         "cloud":"AWS",
         "region":"us-east-1",
         "config":{
               "kind":"Enterprise"
         },
            "kafka_bootstrap_endpoint": "abc-00000-00000.us-east-1.aws.glb.confluent.cloud:9092",
            "http_endpoint": "https://abc-00000-00000.us-east-1.aws.glb.confluent.cloud",
            "environment": {
            "api_version": "org/v2",
            "id": "env-a12b34",
            "kind": "Environment",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
            },
            "network": {
            "id": "n-00000",
            "environment": "test",
            "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
            "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
            "api_version": "networking/v1",
            "kind": "Network"
         }
      },
      "status":{
         "phase":"PROVISIONED"
      }
}

Example response on a secure public endpoint:

HTTP/1.1 200 OK
Content-Type: application/json

{
      "api_version":"cmk/v2",
      "kind":"Cluster",
      "id":"abc-f3a90de",
      "metadata":{
         "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at":"2006-01-02T15:04:05-07:00",
         "updated_at":"2006-01-02T15:04:05-07:00"
      },
      "spec":{
         "display_name":"ProdKafkaCluster",
         "availability":"SINGLE_ZONE",
         "cloud":"GCP",
         "region":"us-east4",
         "config":{
               "kind":"Dedicated",
               "cku": 2
         },
         "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
         "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
         "environment":{
               "api_version": "org/v2",
               "id": "env-a12b34",
               "kind": "Environment",
               "related":"https://api.confluent.cloud/v2/environments/env-a12b34",
               "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
         }
      },
      "status":{
         "phase":"PROVISIONED",
         "cku": 2
      }
}

Example response on a private network:

HTTP/1.1 200 OK
Content-Type: application/json

{
      "api_version":"cmk/v2",
      "kind":"Cluster",
      "id":"abc-f3a90de",
      "metadata":{
         "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
         "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
         "created_at":"2006-01-02T15:04:05-07:00",
         "updated_at":"2006-01-02T15:04:05-07:00"
      },
      "spec":{
         "display_name":"ProdKafkaCluster",
         "availability":"SINGLE_ZONE",
         "cloud":"GCP",
         "region":"us-east4",
         "config":{
               "kind":"Dedicated",
               "cku": 2
         },
            "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
            "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
            "environment": {
            "api_version": "org/v2",
            "id": "env-a12b34",
            "kind": "Environment",
            "related": "https://api.confluent.cloud/v2/environments/env-a12b34",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
            },
            "network": {
            "id": "n-00000",
            "environment": "test",
            "related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
            "resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
            "api_version": "networking/v1",
            "kind": "Network"
         }
      },
      "status":{
         "phase":"PROVISIONED",
         "cku": 2
      }
}

Update Kafka clusters¶

You can update a cluster using the Confluent Cloud Console, Confluent CLI, and REST API.

Considerations:

You can update cluster names for all cluster types.
For Basic clusters, you can upgrade cluster type to Standard (cluster types cannot be downgraded).
Upgrades from Basic to Standard must be made on their own, with no other requests.
For Standard clusters, you can upgrade uptime (SLA downgrades not supported).
For Dedicated clusters, you can update number of CKUs.
Shrink operations can reduce the capacity of the cluster one (1) CKU at a time to the minimum number of CKU supported by the cluster.

Note

Support for shrink operations by more than one CKU at a time is in Limited Availability. To be considered for access before General Availability, contact your account team.

For more information about resizing clusters, see Expand and shrink.

Navigate to the clusters page for your environment.
Select Cluster Settings.
In Identification, click the edit icon.
Enter the new name in Cluster name and click Save changes.

To resize Dedicated clusters with Cloud Console, see Expand a Dedicated Kafka Cluster and Shrink a Dedicated Kafka Cluster.

Update a cluster.

confluent kafka cluster update <id> [flags]

For example:

confluent kafka cluster update lkc-123exa --name "my-new-name"

+----------------------+---------------------------------------------------------+
| Current              | true                                                    |
| ID                   | lkc-123exa                                              |
| Name                 | my-new-name                                             |
| Type                 | BASIC                                                   |
| Ingress Limit (MB/s) |                                                     250 |
| Egress Limit (MB/s)  |                                                     750 |
| Storage              | 5 TB                                                    |
| Cloud                | aws                                                     |
| Region               | us-west-2                                               |
| Availability         | single-zone                                             |
| Status               | UP                                                      |
| Endpoint             | SASL_SSL://pkc-exa45.us-west-2.aws.confluent.cloud:9092 |
| REST Endpoint        | https://pkc-exa45.us-west-2.aws.confluent.cloud:443     |
+----------------------+---------------------------------------------------------+

To update the cluster, make a PATCH call to the cluster endpoint.

You specify the environment and cluster identifiers in the request, and include a payload that contains details of the change.

Request (all cluster types)

PATCH /cmk/v2/clusters/{id}

Parameter	Required or Optional	Description
`id`	Required	Path parameter specifying the identifier for the cluster to update.

The JSON payload must contain a spec object with the following:

environment object with an id property that identifies the environment

And one or more of the following:

display_name property with a new cluster name value
config object that contains a kind property specifying the desired cluster type: Standard or Dedicated
config object that contains a cku property with an integer value (Dedicated clusters only)

The following example demonstrates the request format for a name change:

PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
         "display_name": "updated_name",
         "environment": {
               "id": "env-a12b34""
         }
   }
}

The following example demonstrates the request format to change the cluster type:

PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
         "config": {
               "kind": "Standard"
         },
         "environment": {
               "id": "env-a12b34""
         }
   }
}

The following example demonstrates the request format to change the number of CKUs:

PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
         "config": {
               "kind": "Dedicated",
               "cku": 3
         },
         "environment": {
               "id": "env-a12b34""
         }
   }
}

Response

Success returns a 200 OK and JSON payload that describes the updated cluster.

Responses include the following information:

The cloud provider (AWS, GCP, AZURE) and region for the cluster.
The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Enterprise and Dedicated).
Information about the environment that contains the cluster.
Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Basic, Standard, and Enterprise clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version":"cmk/v2",
   "kind":"Cluster",
   "id":"abc-f3a90de",
   "metadata":{
      "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec":{
      "display_name":"updated_name",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Basic"
      },
      "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment":{
         "id":"env-a12b34",
         "related":"https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status":{
      "phase":"PROVISIONED"
   }
}

Example response for Standard clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version":"cmk/v2",
   "kind":"Cluster",
   "id":"abc-f3a90de",
   "metadata":{
      "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec":{
      "display_name":"updated_name",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Standard"
      },
      "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment":{
         "id":"env-a12b34",
         "related":"https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status":{
      "phase":"PROVISIONED"
   }
}

Example response for Dedicated clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version":"cmk/v2",
   "kind":"Cluster",
   "id":"abc-f3a90de",
   "metadata":{
      "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
      "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
      "created_at": "2022-04-22T20:45:26.657894Z",
      "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec":{
      "display_name":"updated_name",
      "availability":"SINGLE_ZONE",
      "cloud":"GCP",
      "region":"us-east4",
      "config":{
         "kind":"Dedicated",
         "cku": 3
      },
      "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
      "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
      "environment":{
         "id":"env-a12b34",
         "related":"https://api.confluent.cloud/v2/environments/env-a12b34",
         "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-a12b34"
      }
   },
   "status":{
      "phase":"PROVISIONED"
      "cku": 3
   }
}

Delete Kafka clusters¶

You can delete a cluster using the Confluent Cloud Console, Confluent CLI, and REST API. You cannot restore a deleted cluster.

To delete a cluster

Navigate to the clusters page for your environment.
Select Cluster Settings.
Click Delete cluster, enter the cluster name, and click Continue.

Delete a cluster.

confluent kafka cluster delete <id> [flags]

For example (text you must enter is highlighted):

confluent kafka cluster delete lkc-123exa

Are you sure you want to delete Kafka cluster "lkc-123exa"?
To confirm, type "my-new-name". To cancel, press Ctrl-C:

my-new-name

Deleted Kafka cluster "lkc-123exa".

To delete a cluster, make a DELETE request specifying the environment and cluster.

Request (all cluster types)

DELETE /cmk/v2/clusters/{id}?environment={environment_id}

Parameter	Required or Optional	Description
`id`	Required	Path parameter specifying the identifier for the cluster to delete.
`environment`	Required	Query parameter specifying the identifier for the environment that contains the cluster to delete.

Response

Success returns an HTTP 204 No Content.

Confluent retains deleted clusters for a brief period after you delete them. For more information, contact Confluent Cloud Support.

Manage Kafka Clusters on Confluent Cloud¶

Kafka cluster management concepts¶

How to work with Kafka clusters¶

Create Kafka clusters¶

View Kafka clusters¶

View a list of Kafka clusters¶

View a Kafka cluster¶

Update Kafka clusters¶

Delete Kafka clusters¶

Related content¶