Get Started with the Real-Time Context Engine in Confluent Cloud

Enable the Real-Time Context Engine on a Apache Kafka® topic, connect an AI agent, and run your first query.

Prerequisites

You need the following to get started with the Real-Time Context Engine:

A Basic, Standard, Enterprise, or Dedicated cluster on AWS in a supported region.
A Kafka topic with a registered schema (Avro, Protobuf, or JSON Schema).
The following roles, at minimum:
- CloudClusterAdmin to enable the Real-Time Context Engine on a topic.
- DeveloperRead to query data.
- SchemaRegistryRead to list topics and get metadata.

For full details on roles and permissions, see Access Control.

Create an API key

If you plan to use the CLI or MCP to interact with the Real-Time Context Engine, you need a Global API key. If you only use Confluent Cloud Console, you can skip to Enable the RTCE on a topic.

A Global API key provides access across Confluent Cloud resources with a single credential. Each user account or service account can create a maximum of two Global API keys.

Create a Global API key

To create a Global API key, use one of the following methods.

Cloud Console

Sign in to Confluent Cloud.
Open the Administration menu in the upper-right corner.
Select API keys.
Click Add API key.
Select My account, then click Next.
On the Resource scope page, select Global, then click Next.
Optionally, add a name and description for the API key.
Click Create API key.
Download or copy the API key and secret. After you close this dialog, the secret is no longer available.

Confluent CLI

Run the following command to create a Global API key for a service account.

confluent api-key create \
  --resource global \
  --service-account <SERVICE_ACCOUNT_ID> \
  --description "Global API key for the Real-Time Context Engine"

Save the API key and secret that the command returns. The secret is not displayed again.

Confluent Cloud APIs

Send a POST request to create a Global API key.

curl -X POST "https://api.confluent.cloud/iam/v2/api-keys" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "spec": {
      "display_name": "my-global-key",
      "description": "Global API key for the Real-Time Context Engine",
      "owner": {"id": "<SERVICE_ACCOUNT_ID>"},
      "resource": {"id": "global"}
    }
  }'

Save the API key and secret from the response. The secret is not retrievable after the initial response.

Generate the Base64 token

The Real-Time Context Engine authenticates MCP requests with HTTP Basic authentication. Set your API key and secret as environment variables, then Base64-encode them to generate the token.

export KEY=<API_KEY>
export SECRET=<API_SECRET>
export TOKEN=$(echo -n "${KEY}:${SECRET}" | base64)

Use the resulting token in the Authorization header for MCP requests:

Authorization: Basic <TOKEN>

The Real-Time Context Engine also supports OAuth authentication. For details, see Access Control.

Enable the Real-Time Context Engine on a topic

After you enable the Real-Time Context Engine on a topic, Confluent Cloud materializes the topic data so that AI agents can query it.

Cloud Console

To enable the Real-Time Context Engine on a topic, complete the following steps.

Sign in to Confluent Cloud.
From the navigation menu, select Environments and then select your environment.
Click Clusters and then select your cluster.
From the navigation menu, select Topics.
In the topics list, find the Context engine column and click the status cell for the topic you want to enable.
Tip
You might need to scroll the topics table to the right to see the Context engine column.
In the side panel, verify that all prerequisite checks pass. If any check shows a warning, the toggle is disabled.
Toggle the Real-Time Context Engine on.
The status changes to Turning on while the data is materialized. When the process completes, the status changes to On.

Confluent CLI

Use the following command to enable the Real-Time Context Engine on an existing topic:

confluent rtce rtce-topic create \
  --cloud <CLOUD> \
  --region <REGION> \
  --topic-name <TOPIC_NAME> \
  --description "<TOPIC_DESCRIPTION>"

The following command-specific flags are supported:

--cloud: Required. The cloud provider. Set to aws.
--region: Required. The AWS region of the cluster.
--topic-name: Required. The name of the Kafka topic to enable.
--description: Optional. A description of the topic data.

The following example enables the Real-Time Context Engine on a topic:

confluent rtce rtce-topic create \
  --cloud aws \
  --region us-west-2 \
  --topic-name orders_topic \
  --description "Customer orders table"

Confluent Cloud APIs

Send a POST request to enable the Real-Time Context Engine on a topic:

curl -X POST \
  https://api.confluent.cloud/rtce/v1/rtce-topics \
  -H "Content-Type: application/json" \
  -u "<API_KEY>:<API_SECRET>" \
  -d '{
    "spec": {
      "cloud": "<CLOUD>",
      "description": "<TOPIC_DESCRIPTION>",
      "environment": {"id": "<ENV_ID>"},
      "kafka_cluster": {"id": "<LKC_ID>"},
      "region": "<REGION>",
      "topic_name": "<TOPIC_NAME>"
    }
  }'

The following request body parameters are supported:

cloud: Required. The cloud provider. Set to AWS.
region: Required. The AWS region of the cluster.
topic_name: Required. The name of the Kafka topic to enable.
environment.id: Required. The ID of the Confluent Cloud environment.
kafka_cluster.id: Required. The ID of the Kafka cluster.
description: Optional. A description of the topic data.

The following example enables the Real-Time Context Engine on a topic:

curl -X POST \
  https://api.confluent.cloud/rtce/v1/rtce-topics \
  -H "Content-Type: application/json" \
  -u "$KEY:$SECRET" \
  -d '{
    "spec": {
      "cloud": "AWS",
      "description": "Customer orders table",
      "environment": {"id": "env-abc123"},
      "kafka_cluster": {"id": "lkc-abc123"},
      "region": "us-west-2",
      "topic_name": "orders_topic"
    }
  }'

To verify that the topic was enabled, send a GET request:

curl -X GET \
  "https://api.confluent.cloud/rtce/v1/rtce-topics?environment=<ENV_ID>&spec.kafka_cluster=<LKC_ID>" \
  -u "$KEY:$SECRET"

Terraform

Use the confluent_rtce_rtce_topic resource to enable the Real-Time Context Engine on a topic:

resource "confluent_rtce_rtce_topic" "example" {
  cloud = "<CLOUD>"
  description = "<TOPIC_DESCRIPTION>"
  environment {
    id = "<ENV_ID>"
  }
  kafka_cluster {
    id = "<LKC_ID>"
  }
  region = "<REGION>"
  topic_name = "<TOPIC_NAME>"
}

The following arguments are supported:

cloud: Required. The cloud provider. Set to "AWS".
region: Required. The AWS region of the cluster.
topic_name: Required. The name of the Kafka topic to enable.
environment.id: Required. The ID of the Confluent Cloud environment.
kafka_cluster.id: Required. The ID of the Kafka cluster.
description: Optional. A description of the topic data.

The following example enables the Real-Time Context Engine on a topic:

resource "confluent_rtce_rtce_topic" "example" {
  cloud = "AWS"
  description = "Customer orders table"
  environment {
    id = "env-abc123"
  }
  kafka_cluster {
    id = "lkc-abc123"
  }
  region = "us-west-2"
  topic_name = "orders_topic"
}

For complete details on enabling and managing topics, see Manage Topics.

Connect an AI agent

After you enable the Real-Time Context Engine on a topic, you can connect an AI agent to query the data.

Get the MCP endpoint URL

The MCP endpoint URL for your cluster follows this format:

https://mcp.<REGION>.aws.confluent.cloud/mcp/v1/context-engine/organizations/<ORG_ID>/environments/<ENV_ID>/kafka-clusters/<LKC_ID>

You can also find the endpoint URL in Confluent Cloud Console by navigating to your topic and opening the Context engine side panel. Click Copy to copy the URL to your clipboard.

Configure an MCP client

Use the MCP endpoint URL and the Base64 token you generated earlier to configure your MCP client.

The following example configures an MCP client to connect to the Real-Time Context Engine:

{
  "mcpServers": {
    "confluent-rtce": {
      "url": "https://mcp.<REGION>.aws.confluent.cloud/mcp/v1/context-engine/organizations/<ORG_ID>/environments/<ENV_ID>/kafka-clusters/<LKC_ID>",
      "headers": {
        "Authorization": "Basic <TOKEN>"
      }
    }
  }
}

Run your first query

After your MCP client is connected, your AI agent has access to three MCP tools: the query_data tool, the list_topics tool, and the get_metadata tool.

Use the query_data tool to run a simple key lookup:

{
  "tool": "query_data",
  "arguments": {
    "topic_name": "customers",
    "query": "SELECT * WHERE id = 'CUST-123'"
  }
}

If the tool call succeeds, the MCP server returns a response like the following:

{
  "results": [
    {
      "id": "CUST-123",
      "name": "Alice Johnson",
      "status": "active",
      "region": "us-west-2"
    }
  ]
}

For the full set of supported query patterns, including filters, ranges, and predicates with compound logic, see Query Data.

Next steps

Query Data - Learn about supported query patterns and MCP tools.
Manage Topics - Enable, disable, and configure topics.

Note

This website includes content developed at the Apache Software Foundation under the terms of the Apache License v2.