Manage Lightning Tables in Confluent Cloud
Enable, view, query, and disable Lightning Tables, and check their status.
Prerequisites
Before you begin, make sure that you have the following:
A Basic, Standard, Enterprise, or Dedicated cluster on AWS in a supported region.
An Apache Kafka® topic.
A schema for the topic (Avro, Protobuf, or JSON Schema). If the topic doesn’t have one, the Confluent Cloud Console prompts you to create one when you enable the Lightning Table. For more information, see Enable a Lightning Table for a topic without a schema.
At minimum, the CloudClusterAdmin role or the ResourceOwner role on the topic.
A Cloud API key for CLI access.
A Cloud API key or Global API key for Terraform access.
For full details on roles and permissions, see Manage Access to Lightning Tables in Confluent Cloud.
Open your cluster
The Confluent Cloud Console procedures in the following sections start in your cluster and use the cluster’s navigation menu. To open your cluster, complete the following steps.
Sign in to the Confluent Cloud Console at https://confluent.cloud/.
From the navigation menu, select Environments.
Select your environment.
From the navigation menu, select Clusters.
Select your cluster.
Enable a Lightning Table for a topic
When you enable a Lightning Table for a topic, Confluent Cloud makes the topic data available for low-latency SQL queries.
You can enable a Lightning Table from the Lightning Tables page, the Topics page, or a topic’s Settings tab. Each one opens the same confirmation dialog.
After you confirm, the dialog shows the setup progress. Enabling a Lightning Table takes several minutes. You can close the dialog and check back later. When setup completes, a dialog opens that confirms the Lightning Table is enabled. From this dialog, you can query the Lightning Table with SQL.
To enable a Lightning Table from the cluster’s Lightning Tables page, see Get Started with Lightning Tables in Confluent Cloud.
To enable a Lightning Table from the Topics page, complete the following steps.
From the cluster’s navigation menu, select Topics.
In the Lightning Tables column for your topic, click + Enable.
You might need to scroll right to see the column.
In the confirmation dialog, click Enable.
To enable a Lightning Table from a topic’s settings, complete the following steps.
From the cluster’s navigation menu, select Topics.
Select your topic.
Click the Settings tab.
In the Lightning Table section, click Enable Lightning Table.
In the confirmation dialog, click Enable.
Use the following command to enable a Lightning Table for an existing topic:
confluent rtce rtce-topic create \
--cloud <cloud> \
--region <region> \
--topic-name <topic_name> \
--description "<topic_description>"
The command accepts the following flags:
--cloud: Required. The cloud provider. Set toaws.--region: Required. The AWS region of the cluster.--topic-name: Required. The name of the Kafka topic to enable.--description: Optional. A description of the topic data.
The following example enables a Lightning Table for a topic:
confluent rtce rtce-topic create \
--cloud aws \
--region us-west-2 \
--topic-name orders_topic \
--description "Customer orders table"
Send a POST request to enable a Lightning Table for a topic:
curl -X POST \
https://api.confluent.cloud/rtce/v1/rtce-topics \
-H "Content-Type: application/json" \
-u "<api_key>:<api_secret>" \
-d '{
"spec": {
"cloud": "<cloud>",
"description": "<topic_description>",
"environment": {"id": "<env_id>"},
"kafka_cluster": {"id": "<lkc_id>"},
"region": "<region>",
"topic_name": "<topic_name>"
}
}'
The request body accepts the following parameters:
cloud: Required. The cloud provider. Set toAWS.region: Required. The AWS region of the cluster.topic_name: Required. The name of the Kafka topic to enable.environment.id: Required. The ID of the Confluent Cloud environment.kafka_cluster.id: Required. The ID of the Kafka cluster.description: Optional. A description of the topic data.
The following example enables a Lightning Table for a topic:
curl -X POST \
https://api.confluent.cloud/rtce/v1/rtce-topics \
-H "Content-Type: application/json" \
-u "$KEY:$SECRET" \
-d '{
"spec": {
"cloud": "AWS",
"description": "Customer orders table",
"environment": {"id": "env-abc123"},
"kafka_cluster": {"id": "lkc-abc123"},
"region": "us-west-2",
"topic_name": "orders_topic"
}
}'
Use the confluent_rtce_topic resource to enable a Lightning Table for a topic:
resource "confluent_rtce_topic" "example" {
cloud = "<cloud>"
description = "<topic_description>"
environment {
id = "<env_id>"
}
kafka_cluster {
id = "<lkc_id>"
}
region = "<region>"
topic_name = "<topic_name>"
}
The resource accepts the following arguments:
cloud: Required. The cloud provider. Set to"AWS".region: Required. The AWS region of the cluster.topic_name: Required. The name of the Kafka topic to enable.environment.id: Required. The ID of the Confluent Cloud environment.kafka_cluster.id: Required. The ID of the Kafka cluster.description: Optional. A description of the topic data.
The following example enables a Lightning Table for a topic:
resource "confluent_rtce_topic" "example" {
cloud = "AWS"
description = "Customer orders table"
environment {
id = "env-abc123"
}
kafka_cluster {
id = "lkc-abc123"
}
region = "us-west-2"
topic_name = "orders_topic"
}
If enabling fails, verify that you meet the prerequisites.
Enable a Lightning Table for a topic without a schema
A Lightning Table needs a schema for its topic. If the topic doesn’t have one, the Cloud Console prompts you to create one before it enables the Lightning Table. When you apply the schema, Confluent Cloud registers it with Schema Registry.
To enable a Lightning Table for a topic without a schema, complete the following steps.
From the cluster’s navigation menu, select Lightning Tables.
Click Add Lightning Table.
If the cluster doesn’t have any Lightning Tables yet, click Enable Lightning Table instead.
A dialog opens that lists the topics in your cluster.
Clear the Has schema checkbox.
The list now includes topics without a schema.
In the Lightning Table column for your topic, click Enable.
In the Create a topic schema before you enable the Lightning Table dialog, paste a sample JSON message or write your own schema.
For Format, select the schema format.
Click Continue.
In the confirmation dialog, click Enable.
View Lightning Tables
The Lightning Tables page lists the Lightning Tables in a cluster.
To view the Lightning Tables in a cluster, select Lightning Tables from the cluster’s navigation menu.
The Lightning Tables page shows the following columns for each Lightning Table:
Topic name: The topic that the Lightning Table is enabled for.
Lightning Table status: The status of the Lightning Table. For more information, see View Lightning Table status.
Size: The size of the Lightning Table.
Scanned bytes: The bytes scanned across all queries on the Lightning Table in the last 24 hours.
Preview: An icon that opens the Lightning Table for querying. For more information, see Query a Lightning Table in the Cloud Console.
To find a Lightning Table, enter a topic name in the Search topics box.
The Topics page also has a Lightning Tables column, which shows whether each topic has a Lightning Table. You might need to scroll right to see it.
Use the following command to list the topics that have a Lightning Table:
confluent rtce rtce-topic list
Send a GET request to list the topics that have a Lightning Table:
curl -X GET \
"https://api.confluent.cloud/rtce/v1/rtce-topics?environment=<env_id>&spec.kafka_cluster=<lkc_id>" \
-u "<api_key>:<api_secret>"
The following query parameters are required:
environment: The ID of the Confluent Cloud environment.spec.kafka_cluster: The ID of the Kafka cluster.
The following example lists the topics that have a Lightning Table:
curl -X GET \
"https://api.confluent.cloud/rtce/v1/rtce-topics?environment=env-abc123&spec.kafka_cluster=lkc-abc123" \
-u "$KEY:$SECRET"
Query a Lightning Table in the Cloud Console
You can query an active Lightning Table with SQL in the Cloud Console. You can also query it with the API, a Model Context Protocol (MCP) client, the Confluent CLI, or Java Database Connectivity (JDBC). For more information, see Query a Lightning Table in Confluent Cloud.
To query a Lightning Table, complete the following steps.
From the cluster’s navigation menu, select Lightning Tables.
In the Preview column for your Lightning Table, click the Query data icon.
The Preview Lightning Table pane opens.
On the SQL tab, edit the prefilled query to explore your data.
Click Run.
The results appear in the pane.
When you finish querying, close the pane.
Besides the SQL tab, the pane has the following tabs with sample code that you can copy:
API: A
curlcommand that sends a POST request to query the Lightning Table.MCP: An
npxcommand that opens the MCP Inspector, so you can try the MCP server for the Lightning Table.
Results from a Lightning Table query might differ from the results of an Apache Flink® streaming or snapshot query that you run in the SQL workspace. For a comparison of these query types, see How Lightning Table queries work.
For the SQL that a Lightning Table query supports, see SQL Reference for Lightning Table Queries in Confluent Cloud.
If you access the Lightning Table over a private network, you can’t run queries in the Cloud Console. Use the API, MCP, or JDBC with the associated private endpoint instead. For more information, see Private Networking for Lightning Tables in Confluent Cloud.
View Lightning Table status
The Cloud Console shows a Lightning Table’s status in the following places:
The Lightning Table status column on the Lightning Tables page.
The Lightning Tables column on the Topics page.
The status badge on the topic page.
The Status field in the Lightning Table section of the topic’s Settings tab.
A Lightning Table can have the following statuses:
Status |
Description |
|---|---|
Pending |
Confluent Cloud is validating the Lightning Table before setup. |
Provisioning |
Confluent Cloud is setting up resources for the Lightning Table. |
Active |
The Lightning Table is ready to query. |
Unavailable |
Queries are temporarily unavailable while Confluent Cloud restores the Lightning Table. |
Failed |
The Lightning Table can’t recover from an error. Fix the cause. Then disable the Lightning Table and enable it again. |
Deleting |
Confluent Cloud is deleting the Lightning Table. You can’t query it. |
You can query a Lightning Table only when its status is Active.
If a topic doesn’t have a Lightning Table, the Status field on its Settings tab shows Off.
The Lightning Table section of the Settings tab also shows the following details:
Endpoint: The MCP endpoint for the Lightning Table.
Enabled on: The date and time when the Lightning Table was enabled.
Disable a Lightning Table
When you disable a Lightning Table, the topic data becomes unavailable to any applications that query the Lightning Table.
To disable a Lightning Table, complete the following steps.
From the cluster’s navigation menu, select Topics.
Select your topic.
Click the Settings tab.
In the Lightning Table section, click Disable Lightning Table.
In the confirmation dialog, click Disable.
When the Lightning Table is disabled, the Status field shows Off.
Use the following command to disable a Lightning Table for a topic:
confluent rtce rtce-topic delete <topic_name>
The following example disables a Lightning Table for a topic:
confluent rtce rtce-topic delete orders_topic
Send a DELETE request to disable a Lightning Table for a topic:
curl -X DELETE \
"https://api.confluent.cloud/rtce/v1/rtce-topics/<topic_name>?environment=<env_id>&spec.kafka_cluster=<lkc_id>" \
-u "<api_key>:<api_secret>"
The following query parameters are required:
environment: The ID of the Confluent Cloud environment.spec.kafka_cluster: The ID of the Kafka cluster.
The following example disables a Lightning Table for a topic:
curl -X DELETE \
"https://api.confluent.cloud/rtce/v1/rtce-topics/orders_topic?environment=env-abc123&spec.kafka_cluster=lkc-abc123" \
-u "$KEY:$SECRET"
To disable a Lightning Table for a topic, remove the
confluent_rtce_topic resource from your Terraform
configuration. Then run the terraform apply command.
Edit a table description
A table description helps AI agents that query through MCP understand the data in a topic. It also helps them select the right tool for their query. You can add or update a description at any time after you enable a Lightning Table.
Note
You can’t edit a table description in the Cloud Console.
Send a PATCH request to update the description for a topic that has a Lightning Table:
curl -X PATCH \
"https://api.confluent.cloud/rtce/v1/rtce-topics/<topic_name>?environment=<env_id>&spec.cloud=AWS&spec.kafka_cluster=<lkc_id>" \
-H "Content-Type: application/json" \
-u "<api_key>:<api_secret>" \
-d '{
"spec": {
"environment": {"id": "<env_id>"},
"kafka_cluster": {"id": "<lkc_id>"},
"description": "<new_description>"
}
}'
The following query parameters are required:
environment: The ID of the Confluent Cloud environment.spec.cloud: The cloud provider. Set toAWS.spec.kafka_cluster: The ID of the Kafka cluster.
The following example updates the description for a topic that has a Lightning Table:
curl -X PATCH \
"https://api.confluent.cloud/rtce/v1/rtce-topics/orders_topic?environment=env-abc123&spec.cloud=AWS&spec.kafka_cluster=lkc-abc123" \
-H "Content-Type: application/json" \
-u "$KEY:$SECRET" \
-d '{
"spec": {
"environment": {"id": "env-abc123"},
"kafka_cluster": {"id": "lkc-abc123"},
"description": "Customer orders with status and shipping details"
}
}'
Schema changes
A Lightning Table keeps serving queries after a supported schema change, such as adding a nullable column. After an unsupported change, it stops ingesting new records, and queries on it return an error.
For the supported changes and the steps to fix a Lightning Table, see Schema Evolution for Lightning Tables in Confluent Cloud.