Quick Start for ksqlDB Cluster API on Confluent Cloud¶

In addition to managing ksqlDB clusters with the Confluent Cloud Console and Confluent CLI , you can use a collection of REST APIs to perform some basic control operations on ksqlDB clusters.

Like other Confluent Cloud APIs, the APIs have predictable resource-oriented URLs, transport data using JSON, and use standard HTTP verbs, response codes, authentication, and design principles.

You can perform the following operations with this API:

List ksqlDB clusters
Create a ksqlDB cluster
Read a ksqlDB cluster
Delete a ksqlDB cluster

This topic shows you how to get started with each API and provides example request and responses.

For details about call parameters and examples in multiple languages, see kdqlDB Cluster API reference.

Base URL¶

The base URL for all ksqlDB cluster management calls is:

https://api.confluent.cloud/

Object model¶

All Confluent objects share a set of common properties:

api_version: Indicates the API version for the object. The version for these APIs is ksqldbcm/v2.
kind: Indicates what kind of object it is.
id: An identifier for the object.

Authentication¶

Each request to the server must include an Authorization: Basic {key} header that contains a Confluent Cloud API key. For more details on creating or accessing your Confluent Cloud API keys, see Cloud API keys. API requests without a correct authentication header will fail. An API key grants access to a single Confluent cluster.

These keys grant API access to many resources, and should not be shared publicly to GitHub, in client-side code, and so forth.

HTTP Basic authorization requires the key to be colon-separated and base64-encoded.

For example, if your API Key is abcp7DEFGH123456789 and the corresponding Secret is XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO, generate a base64-encoded header with the following command on MacOS or Linux:

echo -n "abcp7DEFGH123456789:XNCIW93I2L1SQPJSJ823K1LS902KLDFMCZPWEO" | base64

The resulting authorization header will be:

Authorization: Basic QUJDREVGR0gxMjM0NTY3ODk6WE5DSVc5M0kyTDFTUVBKU0o4MjNLMUxTOTAyS0xERk1DWlBXRU8=

Alternatively, if testing using Postman, specify Basic Auth and provide the API Key as the Username and the API Secret as the password and Postman will provide the encoding.

Quotas for ksqlDB resources¶

ksqlDB clusters are subject to limits on how many ksqlDB clusters are allowed per environment, and how many CSUs per cluster. For more information, see Environment and ksqlDB cluster.

List ksqlDB clusters¶

If you have operator or admin access to an Confluent Cloud environment, you can list all of the ksqlDB clusters with a GET call to the ksqlDB cluster endpoint.

The request:

Must include the authorization header with your API key and API secret, 64-bit encoded.
Must specify the environment identifier for the environment that contains the clusters as a query parameter, and optionally specify additional query parameters.
Does not require a payload.

Request example¶

The following example shows how to list ksqlDB clusters.

GET /ksqldbcm/v2/clusters?environment={environment_id}&page_size={page_size}&page_token={token}

Your request can include the following query parameters:

Parameter	Required (Yes/No)	Description
`environment`	Yes	Specifies the identifier for the environment that contains the clusters to list.
`page_size`	No	Specifies an integer page size for the returned payload. Max value is 100. Default: 10.
`page_token`	Yes	Contains the opaque pagination token string.

Response example¶

A successful call returns an HTTP 200 OK and a JSON payload that contains a list of clusters for the environment. The response contains a data array. Each entry in the array describes a ksqlDB cluster in the environment, including the following:

A spec object with the following info:
- The ksqlDB identifier and display name.
- Info about the Kafka cluster associated with the ksqlDB cluster.
- Info about the environment that contains the ksqlDB cluster.
The ksqlDB cluster status (PROVISIONED, PROVISIONING, FAILED).

Following is an example of the response format:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "api_version": "ksqldbcm/v2",
    "data": [
        {
            "api_version": "ksqldbcm/v2",
            "id": "lksqlc-w1234",
            "kind": "Cluster",
            "metadata": {
                "created_at": "2022-07-28T23:45:39.180207Z",
                "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lksqlc-w1234",
                "self": "https://api.confluent.cloud/ksqldbcm/v2/clusters/lksqlc-w1234"
            },
            "spec": {
                "credential_identity": {
                    "api_version": "iam/v2",
                    "id": "",
                    "kind": "User",
                    "related": "https://api.confluent.cloud/iam/v2/users",
                    "resource_name": "crn://confluent.cloud/organization=guid-1234/user="
                },
                "csu": 4,
                "display_name": "ksqlDB_api",
                "environment": {
                    "api_version": "org/v2",
                    "id": "env-a12b34",
                    "kind": "Environment",
                    "related": "https://api.confluent.cloud/org/v2/environments/env-a12b34",
                    "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34"
                },
                "kafka_cluster": {
                    "api_version": "cmk/v2",
                    "environment": "env-a12b34",
                    "id": "lkc-21012",
                    "kind": "Cluster",
                    "related": "https://api.confluent.cloud/cmk/v2/clusters/lkc-21012",
                    "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lkc-21012"
                },
                "use_detailed_processing_log": true
            },
            "status": {
                "http_endpoint": "https://pksqlc-nq123.us-central1.gcp.confluent.cloud:443",
                "is_paused": false,
                "phase": "PROVISIONED",
                "storage": 100,
                "topic_prefix": "pksqlc-nq123"
            }
        }
    ],
    "kind": "ClusterList",
    "metadata": {
        "first": "https://api.confluent.cloud/ksqldbcm/v2/clusters",
        "total_size": 1
    }
}

For details on the types of responses you can expect when a call fails, see Failures.

Create a ksqlDB cluster¶

If you have operator or admin access to a Confluent Cloud environment, you can create a ksqlDB cluster with a POST call to the cluster endpoint. Note that you must adhere to the Service Quotas for Confluent Cloud. The request:

Must include the authorization header with your API key and API secret.
Must include a payload that contains a spec object that describes the cluster you want to create.

POST /ksqldbcm/v2/clusters

Request example¶

When you make the request, include a JSON payload that contains a spec object with the following:

Parameter	Required (Yes/No)	Description
`display_name`	Yes	The display name of the cluster. The following requirements apply to the cluster name: Cannot exceed 32 characters Can contain Unicode letters, numbers and underscores
`csu`	Yes	An integer, either 1, 2, 4, 8, 12 that specifies the number of CSUs for the ksqlDB cluster.
`kafka_cluster`	Yes	The Kafka cluster to associate with the ksqlDB cluster. id: The cluster ID environment: Optional. The environment that contains the cluster, if scoped to an environment.
`credential_identity`	Yes	The user associated with the ksqlDB cluster. id: The user ID to associate with the ksqlDB cluster environment: Optional. The environment associated with the user, if scoped to an environment.
`environment`	Yes	Object that contains the environment identifier: `id`: `env-y1234q`.

The following example demonstrates the request format:

POST /ksqldbcm/v2/clusters HTTP/1.1
Host: api.confluent.cloud

{
  "spec": {
    "display_name": "ksqlDB_api",
    "csu": 4,
    "kafka_cluster": {
      "id": "lkc-21012",
      "environment": "string"
    },
    "credential_identity": {
      "id": "u-prr8qk"
    },
    "environment": {
      "id": "env-a12b34"
    }
  }
}

Response example¶

Success returns an HTTP 202 ACCEPTED with a JSON payload that describes the ksqlDB cluster. The cluster description includes the following:

A spec object that describes the ksqlDB cluster, including the following:
- The ksqlDB identifier and display name
- Information about the Kafka cluster associated with the ksqlDB cluster
- Information about the environment that contains the ksqlDB cluster
The ksqlDB cluster status (PROVISIONED, PROVISIONING, FAILED)

Following is an example of the response format:

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
 "api_version": "ksqldbcm/v2",
 "id": "lksqlc-012345",
 "kind": "Cluster",
 "metadata": {
     "created_at": "2022-07-29T17:42:53.018344Z",
     "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lksqlc-012345",
     "self": "https://api.confluent.cloud/ksqldbcm/v2/clusters/lksqlc-012345"
 },
 "spec": {
     "credential_identity": {
         "api_version": "iam/v2",
         "id": "",
         "kind": "User",
         "related": "https://api.confluent.cloud/iam/v2/users",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/user="
     },
     "csu": 4,
     "display_name": "ksqlDB_api",
     "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind": "Environment",
         "related": "https://api.confluent.cloud/org/v2/environments/env-a12b34",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34"
     },
     "kafka_cluster": {
         "api_version": "cmk/v2",
         "environment": "env-a12b34",
         "id": "lkc-21012",
         "kind": "Cluster",
         "related": "https://api.confluent.cloud/cmk/v2/clusters/lkc-21012",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lkc-21012"
     },
     "use_detailed_processing_log": true
 },
 "status": {
     "http_endpoint": "",
     "is_paused": false,
     "phase": "PROVISIONING",
     "storage": 100,
     "topic_prefix": "pksqlc-12wko"
 }
}

For details on the types of responses you can expect when a call fails, see Failures.

Read a ksqlDB cluster¶

If you have operator or admin access to a Confluent Cloud environment, you can make a GET call to the ksqlDB cluster endpoint to retrieve details about a specified cluster. The request:

Does not require a payload.
Must include an authorization header with your API key and API secret.
Must specify the cluster and environment identifiers as path and query parameters, respectively.

Request example¶

GET /ksqldbcm/v2/clusters/{id}?environment={environment_id}

Your request should include the following parameters:

Parameter	Required (Yes/No)	Description
`id`	Yes	Path parameter specifying the identifier for the cluster to read.
`environment`	Yes	Query parameter specifying the identifier for the environment that contains the cluster to read.

Response example¶

Success returns an HTTP 200 OK and a JSON payload that describes the cluster, including the following:

A spec object that describes the ksqlDB cluster, including the following:
- The ksqlDB identifier and display name
- Info about the Kafka cluster associated with the ksqlDB cluster
- Info about the environment that contains the ksqlDB cluster
The ksqlDB cluster status (PROVISIONED, PROVISIONING, FAILED)

Following is an example of the response format:

HTTP/1.1 200 OK
Content-Type: application/json

{
 "api_version": "ksqldbcm/v2",
 "id": "lksqlc-012345",
 "kind": "Cluster",
 "metadata": {
     "created_at": "2022-07-29T17:42:53.018344Z",
     "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lksqlc-012345",
     "self": "https://api.confluent.cloud/ksqldbcm/v2/clusters/lksqlc-012345"
 },
 "spec": {
     "credential_identity": {
         "api_version": "iam/v2",
         "id": "",
         "kind": "User",
         "related": "https://api.confluent.cloud/iam/v2/users",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/user="
     },
     "csu": 4,
     "display_name": "ksqlDB_api",
     "environment": {
         "api_version": "org/v2",
         "id": "env-a12b34",
         "kind": "Environment",
         "related": "https://api.confluent.cloud/org/v2/environments/env-a12b34",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34"
     },
     "kafka_cluster": {
         "api_version": "cmk/v2",
         "environment": "env-a12b34",
         "id": "lkc-21012",
         "kind": "Cluster",
         "related": "https://api.confluent.cloud/cmk/v2/clusters/lkc-21012",
         "resource_name": "crn://confluent.cloud/organization=guid-1234/environment=env-a12b34/cloud-cluster=lkc-21012"
     },
     "use_detailed_processing_log": true
 },
 "status": {
     "http_endpoint": "https://pksqlc-12wko.us-central1.gcp.confluent.cloud:443",
     "is_paused": false,
     "phase": "PROVISIONING",
     "storage": 100,
     "topic_prefix": "pksqlc-12wko"
 }
}

For details on the types of responses you can expect when a call fails, see Failures.

Delete a ksqlDB cluster¶

If you have operator or admin access to Confluent Cloud environment, you can delete a ksqlDB cluster with a DELETE call to the cluster endpoint. The request:

Must include the cluster and environment identifiers in the request as path and query parameters, respectively.
Must include an authorization header with your API key and API secret.
Does not require a payload.

Request example¶

DELETE /ksqldbcm/v2/clusters/{id}?environment={environment_id}

Parameter	Required (Yes/No)	Description
`id`	Yes	Path parameter specifying the identifier for the cluster to delete.
`environment`	Yes	Query parameter specifying the identifier for the environment that contains the cluster to delete.

Response example¶

Success returns an HTTP 204 No Content.

For details on the types of responses you can expect when a call fails, see Failures.

Failures¶

When a failure occurs, you should receive one of the following statuses:

400 Bad Request: Typically a malformed or missing parameter or a parameter value that is not allowed.
401 Unauthorized: Typically invalid or missing authentication credentials.
403 Forbidden: Indicates the credentials provided are not valid for the resource. Make sure your cluster and environment identifiers are correct and match the credentials provided.
404 Not Found: Indicates the resource could not be found at the specified URL.
429 Rate Limit Exceeded: You have sent too many requests.
500 Oops something went wrong: An issue not covered by other errors.

The failure message body will provide more detail about the failure reason and an identifier to use with Confluent support if you need help solving the problem. The following example shows a 400 Bad Request error with details about why the call failed.

HTTP/1.1 400 BAD REQUEST
Content-Type: application/json

{
    "errors": [
      {
           "id": "c9cb03c2878ca172bbd0072bd99a9aaa",
            "status": "400",
            "code": "parameter_missing",
            "detail": "Missing Parameter",
            "source": {
               "parameter": "environment"
             }
      }
   ]
 }

Quick Start for ksqlDB Cluster API on Confluent Cloud¶

Base URL¶

Object model¶

Authentication¶

Quotas for ksqlDB resources¶

List ksqlDB clusters¶

Request example¶

Response example¶

Create a ksqlDB cluster¶

Request example¶

Response example¶

Read a ksqlDB cluster¶

Request example¶

Response example¶

Delete a ksqlDB cluster¶

Request example¶

Response example¶

Failures¶

Related content¶