Important

Cluster management APIs are generally available for use with Standard and Basic clusters.

For Dedicated clusters, the cluster management APIs are available on Confluent Cloud in an Early Access Program to a limited set of early adopters. An early access feature is a component of Confluent Cloud introduced to gain feedback. This feature should be used only for evaluation and non-production testing purposes or to provide feedback to Confluent, particularly as it becomes more widely available in follow-on preview editions. If you would like to participate in the Early Access Program, please email ccloud-api-access+v2-early-access@confluent.io.

Early Access Program features are intended for evaluation use in development and testing environments only, and not for production use. The warranty, SLA, and Support Services provisions of your agreement with Confluent do not apply to Early Access Program features. Early Access Program features are considered to be a Proof of Concept as defined in the Confluent Cloud Terms of Service. Confluent may discontinue providing preview releases of the Early Access Program features at any time in Confluent’s sole discretion.

Confluent Cloud Cluster Management API Overview

In addition to managing Apache Kafka® clusters with the Confluent Cloud Console, you can use a collection of REST APIs to perform operations on Kafka clusters.

  • For Standard and Basic clusters, these APIs are available to all customers
  • For Dedicated clusters, these APIs are available as Early Access (EA)

These APIs enable you to automate and streamline your cluster operations. Like other Confluent Cloud APIs, the APIs have predictable resource-oriented URLs, transport data using JSON, and use standard HTTP verbs, response codes, authentication, and design principles.

You can perform the following operations with this API:

This topic shows you how to use each API and provides example request and responses for Basic, Standard and Dedicated clusters. Note that these APIs are subject to the limits described in Resource Limits for Confluent Cloud.

For more details about call parameters and examples in multiple languages, see Cluster API reference.

Base URL

The base URL for all cluster management calls is:

https://api.confluent.cloud/

Object model

All Confluent objects share a set of common properties:

  • api_version: Indicates the API version for the object. The version for these APIs is cmk/v2.
  • kind: Indicates what kind of object it is.
  • id: An identifier for the object.

Authentication

Each request to the cluster endpoint must include an Authorization: Basic {key} header that contains a Confluent Cloud API key. For more detail on creating or accessing your Confluent Cloud API keys, see Confluent Cloud API keys. API requests without a correct authentication header will fail. A key grants access to a single Confluent cluster.

An API key is owned by a User or Service Account and inherits the permissions granted to the owner. API keys grant many privileges and should not be shared publicly to GitHub, in client-side code, and so forth.

HTTP Basic authorization requires the key to be colon-separated and base-64 encoded.

For example, if your API Key is abcp7DEFGH123456789 and the corresponding Secret is XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO, generate a base-64 encoded header with the following command on MacOS or Linux:

echo -n "abcp7DEFGH123456789:XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO" | base64

The resulting authorization header will be:

Authorization: Basic QUJDREVGR0gxMjM0NTY3ODk6WE5DSVc5M0kyTDFTUVBKU0o4MjNLMUxTOTAyS0xERk1DWlBXRU8=

Alternatively, if testing using a tool such as Postman, specify the API Key as the Username and the API Secret as the password and Postman will provide the encoding.

List clusters

If you have operator or admin access to an Confluent Cloud environment, you can list all of the Kafka clusters with a GET call to the cluster endpoint.

In the request, you must specify the environment ID for the environment that contains the clusters, and optionally specify additional parameters. This call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

GET /cmk/v2/clusters?environment={environment_id}&page_size={page_size}&page_token={token}

Your request should include the following parameters:

Parameter Required or Optional Description
environment_id Required Query parameter specifying the ID for the environment that contains the clusters to list.
page_size Optional Query parameter that specifies an integer page size for the returned payload. Max value is 100. Default: 10.
token Optional Query parameter that contains the opaque pagination token string.

Example response

A successful call returns an HTTP 200 OK and a JSON payload that contains a list of clusters for the environment. The response describes each cluster including the following information:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only (EA) - the number of CKUs allocated to the cluster.
HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "kind": "ClusterList",
   "metadata": {
       "first": "https://api.confluent.cloud/v2/resourcekinds",
       "last": "https://api.confluent.cloud/v2/resourcekinds?page_token=bcAOehAY8F16YD84Z1wT",
       "prev": "https://api.confluent.cloud/v2/resourcekinds?page_token=YIXRY97wWYmwzrax4dld",
       "next": "https://api.confluent.cloud/v2/resourcekinds?page_token=UvmDWOB1iwfAIBPj6EYb",
       "total_size": 123
   },
   "data": [
       {
       "api_version": "cmk/v2",
       "kind": "Cluster",
       "id": "abc-f3a90de",
       "metadata": {
           "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
           "created_at": "2006-01-02T15:04:05-07:00",
           "updated_at": "2006-01-02T15:04:05-07:00"
       },
       "spec": {
           "display_name": "ProdKafkaCluster",
           "availability": "SINGLE_ZONE",
           "cloud": "GCP",
           "region": "us-east4",
           "config": {
               "kind": "Basic"
           },
           "kafka_bootstrap_endpoint": "abc-00000-00000.us-central1.gcp.glb.confluent.cloud:9092",
           "http_endpoint": "https://abc-00000-00000.us-central1.gcp.glb.confluent.cloud",
           "environment": {
           "id": "env-00000",
           "related": "https://api.confluent.cloud/v2/environments/env-00000",
           "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
           }
       },
       "status": {
           "phase": "PROVISIONED"
       }
     }
   ]
}

For details on the types of responses you can expect when a call fails, see Failures.

Create a cluster

If you have operator or admin access to a Confluent Cloud environment, you can create a Kafka cluster with a POST call to the cluster endpoint. Note that you must adhere to the Resource Limits for Confluent Cloud. Also, make sure you include the authorization header with your API key and API secret.

POST /cmk/v2/clusters

Request (all cluster types)

When you make the request, include a JSON payload that contains a spec object with the following:

Parameter Required or Optional Description
display_name Required The display name of the cluster.
availability Required The cluster availability: SINGLE_ZONE or MULTI_ZONE.
cloud Required GCP, AZURE or AWS
region Required A valid region for the cloud provider. See Cloud Providers and Regions for a full list.
config Required Object that contains the cluster type: kind: Basic, Standard or Dedicated (EA only). For Dedicated clusters, you should also specify a cku integer value. Note that clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic.
environment Required Object that contains the environment ID: id: env-1234.

The following example demonstrates the request format for Standard and Basic clusters:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
       "display_name":"ProdKafkaCluster",
       "availability":"SINGLE_ZONE",
       "cloud":"GCP",
       "region":"us-east4",
       "config":{
           "kind":"Basic"
       },
       "environment":{
           "id":"env-00000"
       }
   }
}

Example response

Success returns an HTTP 202 ACCEPTED with a JSON payload that describes the cluster. The cluster description includes the following:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only (EA) - the number of CKUs allocated to the cluster.

Example response for Standard and Basic clusters:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
       "api_version": "cmk/v2",
       "kind": "Cluster",
       "id": "abc-f3a90de",
       "metadata": {
           "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
           "created_at": "2006-01-02T15:04:05-07:00",
           "updated_at": "2006-01-02T15:04:05-07:00"
       },
       "spec": {
           "display_name": "ProdKafkaCluster",
           "availability": "SINGLE_ZONE",
           "cloud": "GCP",
           "region": "us-east4",
           "config": {
               "kind": "Basic"
           },
           "kafka_bootstrap_endpoint": "abc-00000-00000.us-central1.gcp.glb.confluent.cloud:9092",
           "http_endpoint": "https://abc-00000-00000.us-central1.gcp.glb.confluent.cloud",
           "environment": {
           "id": "env-00000",
           "related": "https://api.confluent.cloud/v2/environments/env-00000",
           "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
           }
       },
       "status": {
           "phase": "PROVISIONED"
       }
}

For details on the types of responses you can expect when a call fails, see Failures.

Read a cluster

If you have operator or admin access to a Confluent Cloud environment, you can make a GET call to the cluster endpoint to retrieve details about a specified cluster. You must specify the environment and cluster IDs in the request. This call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

GET /cmk/v2/clusters/{id}?environment={environment_id}

Your request should include the following parameters:

Parameter Required or Optional Description
id Required Path parameter specifying the ID for the cluster to read.
environment_id Required Query parameter specifying the ID for the environment that contains the cluster to read.

Example response

Success returns an HTTP 200 OK and the following JSON payload that describes the cluster, including the following:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only (EA) - the number of CKUs allocated to the cluster.

Example response for Standard and Basic clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
       "api_version":"cmk/v2",
       "kind":"Cluster",
       "id":"abc-f3a90de",
       "metadata":{
           "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
           "created_at":"2006-01-02T15:04:05-07:00",
           "updated_at":"2006-01-02T15:04:05-07:00"
       },
       "spec":{
           "display_name":"ProdKafkaCluster",
           "availability":"SINGLE_ZONE",
           "cloud":"GCP",
           "region":"us-east4",
           "config":{
               "kind":"Standard"
           },
           "kafka_bootstrap_endpoint":"abc-00000-00000.us-central1.gcp.glb.confluent.cloud:9092",
           "http_endpoint":"https://abc-00000-00000.us-central1.gcp.glb.confluent.cloud",
           "environment":{
               "id":"env-00000",
               "related":"https://api.confluent.cloud/v2/environments/env-00000",
               "resource_name":"crn://confluent.cloud/organization=9bb441c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
           }
       },
       "status":{
           "phase":"PROVISIONED"
       }
}

For details on the types of responses you can expect when a call fails, see Failures.

Update a cluster name or type

If you have operator or admin access to a Confluent Cloud environment, you can update the cluster type or change the name of Kafka cluster with an HTTP PATCH call to the cluster endpoint. You specify the environment and cluster IDs in the request, and include a payload that contains details of the change. You can specify one change in the request; either change the cluster type, update the name or adjust the CKUs (Dedicated clusters only). Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

PATCH /cmk/v2/clusters/{id}?environment={environment_id}
Parameter Required or Optional Description
id Required Path parameter specifying the ID for the cluster to update.
environment_id Required Query parameter specifying the ID for the environment that contains the cluster to update.

The JSON payload contains a spec object that contains:

  • An environment object that contains the ID for the environment

One of the following:

  • display_name property that provides the new name value for the cluster.
  • config object that contains a kind property . specifying the desired cluster type: Basic, Standard or Dedicated. Note that clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic. If you specify Basic for a Standard cluster, an error will occur.
  • For Dedicated clusters only (EA): A config object specifying a cku value. Note that you can increase CKUs for a cluster, but you cannot decrease the value.

Include either a display_name or a config object, but not both.

PATCH /cmk/v2/clusters/abcp7-12345?environment=env-test1 HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
           "display_name": "updated_name",
           "environment": {
               "id": "env-00000""
           }
   }
 }

Example response

Success returns a 200 OK and JSON payload that describes the updated cluster. Similiar to the other calls, the payload describes:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only (EA) - the number of CKUs allocated to the cluster.

Example response for Standard and Basic clusters:

 HTTP/1.1 200 OK
 Content-Type: application/json

{
    "api_version":"cmk/v2",
    "kind":"Cluster",
    "id":"abc-f3a90de",
    "metadata":{
        "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
        "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
        "created_at":"2006-01-02T15:04:05-07:00",
        "updated_at":"2006-01-02T15:04:05-07:00"
    },
    "spec":{
        "display_name":"ProdKafkaCluster",
        "availability":"SINGLE_ZONE",
        "cloud":"GCP",
        "region":"us-east4",
        "config":{
            "kind":"Standard"
        },
        "kafka_bootstrap_endpoint":"abc-00000-00000.us-central1.gcp.glb.confluent.cloud:9092",
        "http_endpoint":"https://abc-00000-00000.us-central1.gcp.glb.confluent.cloud",
        "environment":{
            "id":"env-00000",
            "related":"https://api.confluent.cloud/v2/environments/env-00000",
            "resource_name":"crn://confluent.cloud/organization=9bb441c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
        }
    },
    "status":{
        "phase":"PROVISIONED"
    }
}

For details on the types of responses you can expect when a call fails, see Failures.

Delete a cluster

If you have operator or admin access to Confluent Cloud environment, you can delete a Kafka cluster with a DELETE call to the cluster endpoint. You specify the environment and cluster IDs in the request, and the call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

DELETE /cmk/v2/clusters/{id}?environment={environment_id}
Parameter Required or Optional Description
id Required Path parameter specifying the ID for the cluster to delete.
environment_id Required Query parameter specifying the ID for the environment that contains the cluster to read.

Example response

Success returns an HTTP 204 No Content.

For details on the types of responses you can expect when a call fails, see Failures.

Failures

When a failure occurs, you could receive one of the following statuses:

  • 400 Bad Request - Typically a malformed or missing parameter
  • 401 Unauthorized - Typically invalid or missing authentication credentials
  • 403 Forbidden - Indicates the credentials provided are not valid for the resource
  • 404 Not Found - Indicates the resource could not be found at the specified URL
  • 429 Rate Limit Exceeded - You have sent too many requests
  • 500 Oops something went wrong - An issue not covered by other errors

The failure message body will provide more detail about the failure reason and an ID to use with Confluent support if you need help solving the problem. The following example shows a 400 Bad Request error with details about why the call failed.

HTTP/1.1 400 BAD REQUEST
Content-Type: application/json

{
    "errors": [
      {
           "id": "c9cb03c2878ca172bbd0072bd99a9aaa",
            "status": "400",
            "code": "parameter_missing",
            "detail": "Missing Parameter",
            "source": {
               "parameter": "environment"
             }
      }
   ]
 }