Manage Kafka Clusters on Confluent Cloud¶
Use this topic to understand and mange clusters in Confluent Cloud, using the Confluent Cloud Console, Confluent CLI, and REST API.
Cluster management concepts¶
Apache Kafka® clusters in Confluent Cloud are different from Kafka clusters in general in that Confluent offers a fully managed service. Use the following concepts to learn more.
- Basics
You choose from different types of clusters for use in development, production, and high-traffic workloads. Confluent Cloud is available from all the major cloud providers in regions around the world and provides access to Kafka cluster and topic configuration settings. For more information, see the following topics:
- Fault Tolerance
- Confluent Cloud provides a centralized global control plane and a distributed collection of servers that operate as a cluster. Clusters can span data centers or cloud provider availability zones (AZs) and provide highly scalable and fault-tolerant systems that support mission-critical applications. For more information, see Resilience in Confluent Cloud.
- Dedicated Clusters
Dedicated clusters in Confluent Cloud support single-tenant deployments with a 99.95% uptime SLA for Single-Zone, and 99.99% for Multi-Zone, options for private networking and self-managed keys. Use Confluent Unit for Kafka (CKU) to expand and shrink your Dedicated cluster and client quotas to support multi-tenant workloads. For more information, see the following topics:
- Resize a Dedicated Kafka Cluster in Confluent Cloud
- Encrypt Confluent Cloud Kafka Clusters using Self-Managed Keys
- Multi-tenancy and Client Quotas on Confluent Cloud
For more information about private networking, see Networking in Confluent Cloud.
How to work with clusters¶
Use this section to create, delete, and update clusters.
Requirements
- Operator or admin access to a Confluent Cloud environment.
- API requests must include an authorization header with an API key and secret. For more information, see Authentication in the Confluent Cloud API reference.
Create clusters¶
You can create clusters using the Confluent Cloud Console, Confluent CLI, and REST API. For a description of cluster types and resource quotas for clusters, see Confluent Cloud Cluster Types and Service Quotas for Confluent Cloud.
Considerations:
Cloud provider, region, and availability cannot be changed after provisioning a cluster.
Cluster name (
display_name
) is a friendly name with the following requirements:- Use 64 characters or less
- Use whitespace, Unicode letters, numbers, and the following special characters: Period (
.
), Comma (,
), Ampersand (&
), Underscore (_
), Plus (+
), Bar (|
), Open square bracket ([
), Close square bracket (]
), Slash (/
), Dash (-
)
Tip
The business use of a cluster is a good choice for cluster name.
Choose a cluster type.
Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.
Select Standard or Basic and click Begin configuration.
Select a cloud provider tile, Region, and Availability and click Continue.
Specify Cluster name.
Review your configuration and click Launch cluster.
To verify or add a payment method, or use a promotional code, click Review payment method.
Considerations:
- The number of Confluent Unit for Kafka (CKU) determines the capacity of Dedicated clusters. You can manage CKUs after provisioning. For more information on CKUs and how to update them, see Limits per CKU and Update clusters.
- To review network options, see Networking in Confluent Cloud.
- To encrypt your at-rest Kafka cluster data to ensure only the appropriate entity or user can decrypt it, use the cluster creation steps in Encrypt Confluent Cloud Kafka Clusters using Self-Managed Keys.
To create a Dedicated cluster
Navigate to the clusters page for your environment. If this is your fist cluster, click Create cluster on my own. Otherwise, click + Add cluster.
Select Dedicated and click Begin configuration.
Specify a cluster size or accept the default size of one CKU and click Continue.
Multi-zone availability requires two CKUs.
Select a cloud provider tile, Region, and Availability and click Continue.
Select Internet and click Continue, or choose a cloud provider to specify a private network:
AWS private network options:
- Select Transit Gateway, VPC Peering, or PrivateLink.
- Choose Use existing, select the existing network, and click Continue.
- Select VPC Peering.
- Choose Create new and enter Network name.
- Select your zones.
- Enter a CIDR block in CIDR for Confluent Cloud Network.
- Click Continue.
- Select Transit Gateway.
- Choose Create new and enter Network name.
- Select your zones.
- Enter a CIDR block in CIDR for Confluent Cloud Network.
- Click Continue.
- Select PrivateLink.
- Choose Create new and enter Network name.
- Select your zones.
- (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.
- Click Continue.
Azure private network options:
- Select Private Link or VNet Peering.
- Choose Use existing, select the existing network, and click Continue.
- Choose Private Link.
- Choose Create new and enter Network name.
- (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.
- Click Continue.
- Choose VNet Peering.
- Choose Create new and enter Network name.
- Enter a CIDR block in CIDR for Confluent Cloud Network.
- Click Continue.
GPC private network options:
- Select VPC Peering, or Private Service Connect.
- Choose Use existing, select the existing network, and click Continue.
- Select VPC Peering.
- Choose Create new and enter Network name.
- Select your zones.
- Enter a CIDR block in CIDR for Confluent Cloud Network.
- Click Continue.
- Select Private Service Connect.
- Choose Create new and enter Network name.
- Select your zones.
- (Optional) Select Private DNS resolution to resolve your cluster endpoints using a private DNS zone. Otherwise, public DNS resolution is used.
- Click Continue.
For more information, see Networking in Confluent Cloud.
Specify Automatic security and click Continue.
For self-managed options, follow the on-screen steps to register your key. For more information, see Encrypt Confluent Cloud Kafka Clusters using Self-Managed Keys.
Specify Cluster name.
Review your configuration and click Launch cluster.
To verify or add a payment method, or use a promotional code, click Review payment method.
Create a cluster.
confluent kafka cluster create <name> [flags]
For example:
confluent kafka cluster create my_new_cluster --cloud "aws" --region "us-west-2"
To create a cluster, make a POST
request with a JSON payload that specifies the requirements.
POST /cmk/v2/clusters
Request (all cluster types)
When you make the request, include a JSON payload that contains a spec
object with the following:
Parameter | Required or Optional | Description |
---|---|---|
display_name |
Required | The display name of the cluster. |
availability |
Required | The cluster availability: SINGLE_ZONE or MULTI_ZONE . |
cloud |
Required | GCP , AZURE or AWS |
region |
Required | A valid region for the cloud provider. |
config |
Required | Object that contains the cluster type: kind : Basic , Standard or Dedicated .
For Dedicated clusters, you must also specify a cku integer value, and
you can specify an optional encryption_key for BYOK clusters.
Note that clusters can be upgraded from Basic to Standard, but cannot be
downgraded from Standard to Basic. |
environment |
Required | Object that contains the environment identifier: id : env-1234 . |
network |
Required for private network | Object that contains the network identifier: id : n-12345 and related environment name. |
If the Dedicated cluster should be created in a private network, create the network first, then specify the network when you create the cluster.
The following example demonstrates the request format for Basic and Standard clusters:
POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud
{
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Basic"
},
"environment":{
"id":"env-00000"
}
}
}
The following example demonstrates the request format for Dedicated clusters.
Note that you specify the number of CKUs in the config
element:
POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud
{
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Dedicated",
"cku": 2
},
"environment":{
"id":"env-00000"
}
}
}
The following example demonstrates the request format for a Dedicated cluster in a private network.
Note that you specify the network in which to create the cluster in the network
element.
POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud
{
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Dedicated",
"cku": 2
},
"environment":{
"id":"env-00000"
},
"network":{
"id":"n-12345",
"environment": "env-00000"
}
}
}
Response
Successful calls return HTTP 202 ACCEPTED
with a JSON payload that describes the cluster.
Responses include the following information:
- The cloud provider (AWS, GCP, AZURE) and region for the cluster.
- The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard and Dedicated).
- Info about the environment that contains the cluster.
- Dedicated clusters only - the number of CKUs allocated to the cluster, and the network identifier, if in a private network.
Example response for Basic and Standard clusters:
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"api_version": "cmk/v2",
"id": "abc-f3a90de",
"kind": "Cluster",
"metadata": {
"created_at": "2022-04-22T20:45:26.657894Z",
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"updated_at": "2022-04-22T20:45:26.659364Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Basic"
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"api_version": "org/v2",
"id": "env-00000",
"kind":"Environment"
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status": {
"phase": "PROVISIONING"
}
}
Example response for Dedicated clusters on the public internet:
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"api_version": "cmk/v2",
"kind": "Cluster",
"id": "abc-f3a90de",
"metadata": {
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Dedicated",
"cku": 2
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"api_version": "org/v2",
"id": "env-00000",
"kind":"Environment"
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status": {
"phase": "PROVISIONING",
"cku": 2
}
}
Example response for Dedicated clusters in a private network
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"api_version": "cmk/v2",
"kind": "Cluster",
"id": "abc-f3a90de",
"metadata": {
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Dedicated",
"cku": 2
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"api_version": "org/v2",
"id": "env-00000",
"kind":"Environment"
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000",
},
"network": {
"id": "n-00000",
"environment": "env-00000",
"related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
"resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
"api_version": "networking/v1",
"kind": "Network"
}
},
"status": {
"phase": "PROVISIONING",
"cku": 2
}
}
View clusters¶
You can view details of a single cluster and lists multiple clusters using the Confluent Cloud Console, Confluent CLI, and REST API.
View a list of clusters¶
Use this procedure to list clusters.
To list clusters
From the Administration menu, select Environments.
Select an environment.
A list of clusters appears.
List clusters.
confluent kafka cluster list [flags]
For example:
confluent kafka cluster list
To list clusters, make a GET
request that specifies the environment.
Request
GET /cmk/v2/clusters?environment={environment_id}
Your request can include the following parameters:
Parameter | Required or Optional | Description |
---|---|---|
environment |
Required | Specifies the identifier for the environment that contains the clusters to list. |
page_size |
Optional | Specifies an integer page size for the returned payload. Max value is 100. Default: 10. |
spec.network |
Optional | A network identifier for the clusters to list. Your query can contain multiple network identifiers. |
page_token |
Optional | Contains the opaque pagination token string. |
Response
Successful calls return HTTP 200 OK
and a JSON payload that contains the list of clusters for the environment.
Responses include the following information:
- The cloud provider (AWS, GCP, AZURE) and region for the cluster.
- The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard and Dedicated).
- Info about the environment that contains the cluster.
- Dedicated clusters only - the number of CKUs allocated to the cluster.
Example response for Basic and Standard clusters:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version": "cmk/v2",
"data": [
{
"api_version": "cmk/v2",
"kind": "Cluster",
"id": "abc-f3a90de",
"metadata": {
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Basic"
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"id": "env-00000",
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000",
"api_version": "org/v2",
"kind": "Environment"
}
},
"status": {
"phase": "PROVISIONED"
}
}
],
"kind": "ClusterList",
"metadata": {
"first": "https://api.confluent.cloud/cmk/v2/clusters",
"total_size": 1
}
}
Example response for Dedicated clusters on the public internet:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version": "cmk/v2",
"data": [
{
"api_version": "cmk/v2",
"kind": "Cluster",
"id": "abc-f3a90de",
"metadata": {
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Basic"
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"id": "env-00000",
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000",
"api_version": "org/v2",
"kind": "Environment"
}
},
"status": {
"phase": "PROVISIONED",
"cku": 2
}
}
],
"kind": "ClusterList",
"metadata": {
"first": "https://api.confluent.cloud/cmk/v2/clusters",
"total_size": 1
}
}
Example response for Dedicated clusters in a private network:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version": "cmk/v2",
"data": [
{
"api_version": "cmk/v2",
"kind": "Cluster",
"id": "abc-f3a90de",
"metadata": {
"self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec": {
"display_name": "ProdKafkaCluster",
"availability": "SINGLE_ZONE",
"cloud": "GCP",
"region": "us-east4",
"config": {
"kind": "Basic"
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"id": "env-00000",
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000",
"api_version": "org/v2",
"kind": "Environment"
},
"network": {
"id": "n-00000",
"environment": "env-00000",
"related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000,
"api_version": "networking/v1",
"kind": "Network"
}
},
"status": {
"phase": "PROVISIONED",
"cku": 2
}
}
],
"kind": "ClusterList",
"metadata": {
"first": "https://api.confluent.cloud/cmk/v2/clusters",
"total_size": 1
}
}
View a cluster¶
Use this procedure to get details about a cluster.
To list a cluster
- Navigate to the clusters page for your environment.
- Select Cluster Settings to see details about your cluster. To see capacity details, click Capacity.
Describe a cluster.
confluent kafka cluster describe [id] [flags]
For example:
confluent kafka cluster describe lkc-000000
To list a cluster, make a GET
request that specifies the cluster and environment.
Request (all cluster types)
GET /cmk/v2/clusters/{id}?environment={environment_id}
Your request can include the following parameters:
Parameter | Required or Optional | Description |
---|---|---|
id |
Required | Path parameter specifying the identifier for the cluster to read. |
environment |
Required | Query parameter specifying the identifier for the environment that contains the cluster to read. |
Response
Successful calls return HTTP 200 OK
and a JSON payload that describes the cluster.
Responses include the following information:
- The cloud provider (AWS, GCP, AZURE) and region for the cluster.
- The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
- Info about the environment that contains the cluster.
- Dedicated clusters only - the number of CKUs allocated to the cluster.
Example response for Basic and Standard clusters:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at":"2022-04-22T20:45:26.657894Z",
"updated_at":"2022-04-22T20:45:28.045579Z"
},
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Standard"
},
"kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment":{
"api_version": "org/v2",
"id": "env-00000",
"kind": "Environment",
"related":"https://api.confluent.cloud/v2/environments/env-00000",
"resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status":{
"phase":"PROVISIONED"
}
}
Example response for Dedicated clusters on the public internet:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at":"2006-01-02T15:04:05-07:00",
"updated_at":"2006-01-02T15:04:05-07:00"
},
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Dedicated",
"cku": 2
},
"kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment":{
"api_version": "org/v2",
"id": "env-00000",
"kind": "Environment",
"related":"https://api.confluent.cloud/v2/environments/env-00000",
"resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status":{
"phase":"PROVISIONED",
"cku": 2
}
}
Example response for Dedicated clusters on a private network:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at":"2006-01-02T15:04:05-07:00",
"updated_at":"2006-01-02T15:04:05-07:00"
},
"spec":{
"display_name":"ProdKafkaCluster",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Dedicated",
"cku": 2
},
"kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment": {
"api_version": "org/v2",
"id": "env-00000",
"kind": "Environment",
"related": "https://api.confluent.cloud/v2/environments/env-00000",
"resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
},
"network": {
"id": "n-00000",
"environment": "test",
"related": "https://api.confluent.cloud/networking/v1/networks/n-00000",
"resource_name": "https://api.confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/network=n-00000",
"api_version": "networking/v1",
"kind": "Network"
}
},
"status":{
"phase":"PROVISIONED",
"cku": 2
}
}
Update clusters¶
You can update a cluster using the Confluent Cloud Console, Confluent CLI, and REST API.
Considerations:
- You can update cluster type, cluster names and number of CKUs (Dedicated clusters only)
- Clusters cannot be downgraded from Standard to Basic
- Upgrades from Basic to Standard must be made on their own, with no other requests
- Shrink operations can reduce the capacity of the cluster one (1) CKU at a time
For more information about resizing clusters, see Expand and shrink.
- Navigate to the clusters page for your environment.
- Select Cluster Settings.
- In Identification, click the edit icon.
- Enter the new name in Cluster name and click Save changes.
- Navigate to the clusters page for your environment.
- Select Cluster Settings.
- In Cluster type for a Basic cluster, click the edit icon.
- In Standard cluster, click Upgrade, acknowledge that you understand cluster downgrades are not supported, and click Instant upgrade.
To resize Dedicated clusters with Cloud Console, see Expand a Dedicated Kafka Cluster and Shrink a Dedicated Kafka Cluster.
Update a cluster.
confluent kafka cluster update <id> [flags]
For example:
confluent kafka cluster update lkc-000000 --name "my-new-name"
To update the cluster, make a PATCH
call to the cluster endpoint.
You specify the environment and cluster identifiers in the request, and include a payload that contains details of the change.
Request (all cluster types)
PATCH /cmk/v2/clusters/{id}
Parameter | Required or Optional | Description |
---|---|---|
id |
Required | Path parameter specifying the identifier for the cluster to update. |
The JSON payload must contain a spec
object with the following:
environment
object with anid
property that identifies the environment
And one or more of the following:
display_name
property with a new cluster name valueconfig
object that contains akind
property specifying the desired cluster type: Standard or Dedicatedconfig
object that contains acku
property with an integer value (Dedicated clusters only)
The following example demonstrates the request format for a name change:
PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud
{
"spec": {
"display_name": "updated_name",
"environment": {
"id": "env-00000""
}
}
}
The following example demonstrates the request format to change the cluster type:
PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud
{
"spec": {
"config": {
"kind": "Standard"
},
"environment": {
"id": "env-00000""
}
}
}
The following example demonstrates the request format to change the number of CKUs:
PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud
{
"spec": {
"config": {
"kind": "Dedicated",
"cku": 3
},
"environment": {
"id": "env-00000""
}
}
}
Response
Success returns a 200 OK
and JSON payload that describes the updated cluster.
Responses include the following information:
- The cloud provider (AWS, GCP, AZURE) and region for the cluster.
- The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard and Dedicated).
- Info about the environment that contains the cluster.
- Dedicated clusters only - the number of CKUs allocated to the cluster.
Example response for Basic and Standard clusters:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec":{
"display_name":"updated_name",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Basic"
},
"kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment":{
"id":"env-00000",
"related":"https://api.confluent.cloud/v2/environments/env-00000",
"resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status":{
"phase":"PROVISIONED"
}
}
Example response for Standard clusters:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec":{
"display_name":"updated_name",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Standard"
},
"kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment":{
"id":"env-00000",
"related":"https://api.confluent.cloud/v2/environments/env-00000",
"resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status":{
"phase":"PROVISIONED"
}
}
Example response for Dedicated clusters:
HTTP/1.1 200 OK
Content-Type: application/json
{
"api_version":"cmk/v2",
"kind":"Cluster",
"id":"abc-f3a90de",
"metadata":{
"self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
"resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
"created_at": "2022-04-22T20:45:26.657894Z",
"updated_at": "2022-04-22T21:13:55.742641944Z"
},
"spec":{
"display_name":"updated_name",
"availability":"SINGLE_ZONE",
"cloud":"GCP",
"region":"us-east4",
"config":{
"kind":"Dedicated",
"cku": 3
},
"kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
"http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
"environment":{
"id":"env-00000",
"related":"https://api.confluent.cloud/v2/environments/env-00000",
"resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
}
},
"status":{
"phase":"PROVISIONED"
"cku": 3
}
}
Delete clusters¶
You can delete a cluster using the Confluent Cloud Console, Confluent CLI, and REST API. You cannot restore a deleted cluster.
To delete a cluster
- Navigate to the clusters page for your environment.
- Select Cluster Settings.
- Click Delete cluster, enter the cluster name, and click Continue.
Delete a cluster.
confluent kafka cluster delete <id> [flags]
For example:
confluent kafka cluster delete lkc-000000
To delete a cluster, make a DELETE
request specifying the environment and cluster.
Request (all cluster types)
DELETE /cmk/v2/clusters/{id}?environment={environment_id}
Parameter | Required or Optional | Description |
---|---|---|
id |
Required | Path parameter specifying the identifier for the cluster to delete. |
environment |
Required | Query parameter specifying the identifier for the environment that contains the cluster to delete. |
Response
Success returns an HTTP 204 No Content
.
Confluent retains deleted clusters for a brief period after you delete them. For more information, contact Confluent Cloud Support.