Confluent Cloud Cluster Management API Quick Start

In addition to managing Apache Kafka® clusters with the Confluent Cloud Console and Confluent CLI , you can use a collection of REST APIs to perform operations on Kafka clusters.

These APIs enable you to automate and streamline your cluster operations. Like other Confluent Cloud APIs, the APIs have predictable resource-oriented URLs, transport data using JSON, and use standard HTTP verbs, response codes, authentication, and design principles.

You can perform the following operations on Basic, Standard and Dedicated clusters with this API:

This topic shows you how to get started with each API and provides example request and responses for Basic, Standard and Dedicated clusters. Note that these APIs are subject to the limits described in Service Quotas for Confluent Cloud.

For details about call parameters and examples in multiple languages, see Cluster API reference.

Base URL

The base URL for all cluster management calls is:

https://api.confluent.cloud/

Object model

All Confluent objects share a set of common properties:

  • api_version: Indicates the API version for the object. The version for these APIs is cmk/v2.
  • kind: Indicates what kind of object it is.
  • id: An identifier for the object.

Authentication

Each request to the server must include an Authorization: Basic {key} header that contains a Confluent Cloud API key. For more details on creating or accessing your Confluent Cloud API keys, see Cloud API keys. API requests without a correct authentication header will fail. An API key grants access to a single Confluent cluster.

These keys grant API access to many resources, and should not be shared publicly to GitHub, in client-side code, and so forth.

HTTP Basic authorization requires the key to be colon-separated and base64-encoded.

For example, if your API Key is abcp7DEFGH123456789 and the corresponding Secret is XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO, generate a base64-encoded header with the following command on MacOS or Linux:

echo -n "abcp7DEFGH123456789:XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO" | base64

The resulting authorization header will be:

Authorization: Basic QUJDREVGR0gxMjM0NTY3ODk6WE5DSVc5M0kyTDFTUVBKU0o4MjNLMUxTOTAyS0xERk1DWlBXRU8=

Alternatively, if testing using Postman, specify Basic Auth and provide the API Key as the Username and the API Secret as the password and Postman will provide the encoding.

List clusters

If you have operator or admin access to an Confluent Cloud environment, you can list all of the Kafka clusters with a GET call to the cluster endpoint.

In the request, you must specify the environment identifier for the environment that contains the clusters, and optionally specify additional parameters. For calls to clusters in a private network, you must specify the network identifier. For more details on creating a network using an API, see the Networking API reference. This call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Request example

The following examples show how to list clusters. Your request can include the following query parameters

GET /cmk/v2/clusters?environment={environment_id}&page_size={page_size}&page_token={token}

Your request can include the following query parameters:

Parameter Required or Optional Description
environment Required Specifies the identifier for the environment that contains the clusters to list.
page_size Optional Specifies an integer page size for the returned payload. Max value is 100. Default: 10.
spec.network Required For private networks A network identifier for the clusters to list. Your query can contain multiple network identifiers.
page_token Optional Contains the opaque pagination token string.

Example response

A successful call returns an HTTP 200 OK and a JSON payload that contains a list of clusters for the environment. The response describes each cluster including the following information:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for Confluent Cloud for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only - the number of CKUs allocated to the cluster.
HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version": "cmk/v2",
   "data": [
       {
       "api_version": "cmk/v2",
       "kind": "Cluster",
       "id": "abc-f3a90de",
       "metadata": {
           "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
           "created_at": "2022-04-22T20:45:26.657894Z",
           "updated_at": "2022-04-22T21:13:55.742641944Z"
       },
       "spec": {
           "display_name": "ProdKafkaCluster",
           "availability": "SINGLE_ZONE",
           "cloud": "GCP",
           "region": "us-east4",
           "config": {
               "kind": "Basic"
           },
           "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
           "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
           "environment": {
            "id": "env-00000",
            "related": "https://api.confluent.cloud/v2/environments/env-00000",
            "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000",
            "api_version": "org/v2",
            "kind": "Environment"
           }
       },
       "status": {
           "phase": "PROVISIONED"
       }
     }
   ],
   "kind": "ClusterList",
   "metadata": {
     "first": "https://api.confluent.cloud/cmk/v2/clusters",
     "total_size": 1
   }
}

For details on the types of responses you can expect when a call fails, see Failures.

Create a cluster

If you have operator or admin access to a Confluent Cloud environment, you can create a Kafka cluster with a POST call to the cluster endpoint. Note that you must adhere to the Service Quotas for Confluent Cloud. Also, make sure you include the authorization header with your API key and API secret.

For Dedicated clusters, there are two extra options:

POST /cmk/v2/clusters

Request (all cluster types)

When you make the request, include a JSON payload that contains a spec object with the following:

Parameter Required or Optional Description
display_name Required

The display name of the cluster. The following requirements apply to the cluster name:

  • Cannot exceed 64 characters
  • Can contain whitespace, Unicode letters, numbers and marks, and the following special characters: . , & _ + | [] / - . Note that using dashes instead of spaces makes it easier to reference the cluster in a programmatic way.
availability Required The cluster availability: SINGLE_ZONE or MULTI_ZONE.
cloud Required GCP, AZURE or AWS
region Required A valid region for the cloud provider. See Cloud Providers and Regions for Confluent Cloud for a full list.
config Required Object that contains the cluster type: kind: Basic, Standard or Dedicated. For Dedicated clusters, you must also specify a cku integer value, and you can specify an optional encryption_key for BYOK clusters. See Encrypt Confluent Cloud Clusters using Self-Managed Keys. Note that clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic.
environment Required Object that contains the environment identifier: id: env-1234.
network Required for private network Object that contains the network identifier: id: n-12345 and related environment name.

The following example demonstrates the request format for Standard and Basic clusters:

POST /cmk/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
   "spec":{
       "display_name":"ProdKafkaCluster",
       "availability":"SINGLE_ZONE",
       "cloud":"GCP",
       "region":"us-east4",
       "config":{
           "kind":"Basic"
       },
       "environment":{
           "id":"env-00000"
       }
   }
}

Example response

Success returns an HTTP 202 ACCEPTED with a JSON payload that describes the cluster. The cluster description includes the following:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for Confluent Cloud for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only - the number of CKUs allocated to the cluster, and the network identifier, if in a private network.

Example response for Standard and Basic clusters:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
       "api_version": "cmk/v2",
       "id": "abc-f3a90de",
       "kind": "Cluster",
       "metadata": {
           "created_at": "2022-04-22T20:45:26.657894Z",
           "self": "https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name": "crn://confluent.cloud/kafka=abc-f3a90de",
           "updated_at": "2022-04-22T20:45:26.659364Z"
       },
       "spec": {
           "display_name": "ProdKafkaCluster",
           "availability": "SINGLE_ZONE",
           "cloud": "GCP",
           "region": "us-east4",
           "config": {
               "kind": "Basic"
           },
           "kafka_bootstrap_endpoint": "abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
           "http_endpoint": "https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
           "environment": {
             "api_version": "org/v2",
             "id": "env-00000",
             "kind":"Environment"
             "related": "https://api.confluent.cloud/v2/environments/env-00000",
             "resource_name": "crn://confluent.cloud/organization=1234abcd-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
           }
       },
       "status": {
           "phase": "PROVISIONING"
       }
}

For details on the types of responses you can expect when a call fails, see Failures.

Read a cluster

If you have operator or admin access to a Confluent Cloud environment, you can make a GET call to the cluster endpoint to retrieve details about a specified cluster. You must specify the environment and cluster identifiers in the request. This call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

GET /cmk/v2/clusters/{id}?environment={environment_id}

Your request should include the following parameters:

Parameter Required or Optional Description
id Required Path parameter specifying the identifier for the cluster to read.
environment Required Query parameter specifying the identifier for the environment that contains the cluster to read.

Example response

Success returns an HTTP 200 OK and the following JSON payload that describes the cluster, including the following:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for Confluent Cloud for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Standard and Basic clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
       "api_version":"cmk/v2",
       "kind":"Cluster",
       "id":"abc-f3a90de",
       "metadata":{
           "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
           "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
           "created_at":"2022-04-22T20:45:26.657894Z",
           "updated_at":"2022-04-22T20:45:28.045579Z"
       },
       "spec":{
           "display_name":"ProdKafkaCluster",
           "availability":"SINGLE_ZONE",
           "cloud":"GCP",
           "region":"us-east4",
           "config":{
               "kind":"Standard"
           },
           "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
           "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
           "environment":{
               "api_version": "org/v2",
               "id": "env-00000",
               "kind": "Environment",
               "related":"https://api.confluent.cloud/v2/environments/env-00000",
               "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
           }
       },
       "status":{
           "phase":"PROVISIONED"
       }
}

For details on the types of responses you can expect when a call fails, see Failures.

Update a cluster

If you have operator or admin access to a Confluent Cloud environment, you can update the cluster with an HTTP PATCH call to the cluster endpoint. You specify the environment and cluster identifiers in the request, and include a payload that contains details of the change. You can specify one change in the request:

  • Change the cluster type
  • Update the cluster name
  • Expand or shrink the number of CKUs for the cluster (Dedicated clusters only)

Make sure you include the authorization header with your API key and API secret.

Request example (all cluster types)

PATCH /cmk/v2/clusters/{id}
Parameter Required or Optional Description
id Required Path parameter specifying the identifier for the cluster to update.

The JSON payload contains a spec object that contains:

  • An environment object that contains the indentifier for the environment

One of the following:

  • display_name property that provides the new name value for the cluster.
  • config object that contains a kind property . specifying the desired cluster type: Basic, Standard or Dedicated. Note that clusters can be upgraded from Basic to Standard, but cannot be downgraded from Standard to Basic. If you specify Basic for a Standard cluster, an error will occur.
  • For Dedicated clusters only : A config object specifying a cku value. Note that for a shrink operation, you can reduce the capacity of the cluster one (1) CKU at a time.

Include either a display_name or a config object, but not both.

PATCH /cmk/v2/clusters/abc-f3a90de HTTP/1.1
Host: api.confluent.cloud

{
   "spec": {
           "display_name": "updated_name",
           "environment": {
               "id": "env-00000""
           }
   }
 }

Example response

Success returns a 200 OK and JSON payload that describes the updated cluster. Similiar to the other calls, the payload describes:

  • The cloud provider (AWS, GCP, AZURE) and region for the cluster. See Cloud Providers and Regions for Confluent Cloud for a full list of regions.
  • The cluster status (PROVISIONED, PROVISIONING, FAILED) and kind (Basic, Standard, Dedicated).
  • Info about the environment that contains the cluster.
  • Dedicated clusters only - the number of CKUs allocated to the cluster.

Example response for Standard and Basic clusters:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "api_version":"cmk/v2",
   "kind":"Cluster",
   "id":"abc-f3a90de",
   "metadata":{
       "self":"https://api.confluent.cloud/v2/kafka-clusters/abc-f3a90de",
       "resource_name":"crn://confluent.cloud/kafka=abc-f3a90de",
       "created_at": "2022-04-22T20:45:26.657894Z",
       "updated_at": "2022-04-22T21:13:55.742641944Z"
   },
   "spec":{
       "display_name":"updated_name",
       "availability":"SINGLE_ZONE",
       "cloud":"GCP",
       "region":"us-east4",
       "config":{
           "kind":"Standard"
       },
       "kafka_bootstrap_endpoint":"abc-00000-00000.us-east4.gcp.glb.confluent.cloud:9092",
       "http_endpoint":"https://abc-00000-00000.us-east4.gcp.glb.confluent.cloud",
       "environment":{
           "id":"env-00000",
           "related":"https://api.confluent.cloud/v2/environments/env-00000",
           "resource_name":"crn://confluent.cloud/organization=abcd41c4-edef-46ac-8a41-c49e44a3fd9a/environment=env-00000"
       }
   },
   "status":{
       "phase":"PROVISIONED"
   }
}

For details on the types of responses you can expect when a call fails, see Failures.

Delete a cluster

If you have operator or admin access to Confluent Cloud environment, you can delete a Kafka cluster with a DELETE call to the cluster endpoint. You specify the environment and cluster identifiers in the request, and the call does not require a payload. Make sure you include the authorization header with your API key and API secret.

Note that once a cluster has been deleted, it cannot be restored.

Request example (all cluster types)

DELETE /cmk/v2/clusters/{id}?environment={environment_id}
Parameter Required or Optional Description
id Required Path parameter specifying the identifier for the cluster to delete.
environment Required Query parameter specifying the identifier for the environment that contains the cluster to delete.

Example response

Success returns an HTTP 204 No Content.

For details on the types of responses you can expect when a call fails, see Failures.

Failures

When a failure occurs, you should receive one of the following statuses:

400 Bad Request
Typically a malformed or missing parameter or a parameter value that is not allowed.
401 Unauthorized
Typically invalid or missing authentication credentials.
403 Forbidden
Indicates the credentials provided are not valid for the resource. Make sure your cluster and environment identifiers are correct and match the credentials provided.
404 Not Found
Indicates the resource could not be found at the specified URL.
429 Rate Limit Exceeded
You have sent too many requests.
500 Oops something went wrong
An issue not covered by other errors.

The failure message body will provide more detail about the failure reason and an identifier to use with Confluent support if you need help solving the problem. The following example shows a 400 Bad Request error with details about why the call failed.

HTTP/1.1 400 BAD REQUEST
Content-Type: application/json

{
    "errors": [
      {
           "id": "c9cb03c2878ca172bbd0072bd99a9aaa",
            "status": "400",
            "code": "parameter_missing",
            "detail": "Missing Parameter",
            "source": {
               "parameter": "environment"
             }
      }
   ]
 }