ksqlDB REST API in Confluent Platform

The ksqlDB REST API enables you to interact with your ksqlDB applications programmatically. You can use it to execute ksqlDB commands, run queries, and manage your ksqlDB clusters.

There are a number key aspects to using the ksqlDB REST API:

  • Endpoints: The /ksql endpoint is used for running commands like SHOW STREAMS and SHOW TABLES, as well as for creating streams and tables. The /query endpoint was previously used for running queries, but the /query-stream endpoint is now preferred, especially for continuous queries that emit changes. The /query-stream endpoint is essential for consuming continuous results from push queries.

  • Authentication: When using Confluent Cloud, you use Basic Authentication with an API key and secret. You can create these by using the Confluent CLI with the command confluent api-key create --resource <id>, replacing <id> with the ID of your ksqlDB cluster.

  • Request Format: Requests to the /ksql endpoint are POST requests with a JSON body. The body contains a ksql field with the SQL statement and a properties field (previously sometimes referred to as streamProperties) for any additional properties. For the /query-stream endpoint, the JSON body should contain a sql field with the SQL statement and a properties field for any additional properties. The Content-Type header should be set to application/vnd.ksql.v1+json. For continuous queries using the /query-stream endpoint, the Accept header should also be set to application/vnd.ksql.v1+json.

  • Response Format: The response is in JSON format. For the /query-stream endpoint, the response is a continuous stream of data, which can be consumed line-by-line.

  • Tools: You can use tools like curl, httpx, requests, or Postman to interact with the ksqlDB REST API. If using httpx for continuous queries, ensure the http2=True parameter is set in the httpx.AsyncClient. When using the requests library, the stream argument is crucial for handling continuous responses.

  • Pagination: For list operations, like listing clusters, the API uses pagination. The page_size parameter controls the number of items per page, and the page_token is used to navigate through the result set.

For more information, see the REST API reference.

Available endpoints:

  • /clusterStatus: Get the status of ksqlDB servers

  • /info: Get information about the status of a ksqlDB server

  • /is_valid_property: Determine whether a property is prohibited from setting

  • /ksql: Run a statement

  • /query: Run a query and stream back the output

  • /status: Get the status of a statement

  • /query-stream: Send requests to the /query-stream endpoint.

  • /terminate: Terminate a ksqlDB cluster