Use Message Browser in Confluent Cloud

Use message browser, a built-in tool in the Confluent Cloud Console, to inspect, produce, and download messages from Apache Kafka® topics in real time. You can browse messages across all partitions, jump to specific offsets or timestamps, produce new messages with or without schemas, and download messages as JSON or CSV. Cloud Console validates messages against their schema before you produce them to the topic.

You pay for data consumed and produced to your cluster by message browser. Message browser uses the Kafka REST API to produce messages, which incurs an additional ingress charge. For details, see Ingress and egress.

Considerations

By default, message browser displays newer data at the top of the messages pane.
Filter across any of the available columns.
You can only filter results that are displayed. To add more results to the current display, increase the maximum results, manipulate the start time, or add partitions.
Message browser displays messages that have associated schemas. Where multiple schema contexts are available, you can select between contexts with a drop-down list. Message browser uses only TopicNameStrategy schemas to deserialize messages.
Non-compacted topics show a tile for Total messages while compacted topics show a tile for Retained bytes.
Production in the last hour tracks all message delivery attempts, regardless of success or failure.

Prerequisite

You must have data being produced to a topic or already present in a topic to use message browser.

Inspect topics

Use message browser to inspect topic details and browse live streaming messages. If you don’t already have data flowing through the topic, use the built-in produce new message to topic feature to send a message to the topic. You can replay old data that was already produced as long as it hasn’t been compacted or deleted yet.

To browse a live stream of messages and inspect a topic:

Sign in to Confluent Cloud.
If you have more than one environment, select an environment.
Select a cluster.
Select Topics in the navigation menu. The Topics page appears.
Select the topic you want to inspect.
Select Messages. The Messages page displays incoming streaming messages for the topic.
Why does message browser not show messages equal to maximum results?
Message browser might not always display results equal to the maximum results selected. This is by design and can happen for various reasons based on your filtering selections and the ordering of data in the topic. For example, your topic might have thousands of total messages, maximum results set to 1,000, but message browser might display fewer than 1,000 results. Change Latest to From beginning to get results from the beginning of the topic up to the 1,000 maximum result limit. You could also select a specific time, an offset, or a timestamp to fetch a different set of results up to the maximum.
To browse the messages pane, use the following features:
- Max results drop-down list changes the maximum displayed results.
- Auto-refresh toggle pauses and resumes message browser updates. The topic continues to receive any messages produced during a pause, but message browser pauses the display.
- Filter by timestamp, offset, key or value filter results by the specified criteria. Enter a timestamp in Unix epoch format. Enter an offset, key, or any string value to filter by that value.
- Select a row to access the contents of the message. A message detail pane opens. In Message details, select Key, Value, or Headers.
- Use From beginning, From offset, From timestamp, or From specific time to refine message searches.
  From beginning, offset, timestamp, or specific time details
  From beginning
  By default, message browser displays the latest messages when you first open it. You can instead view the earliest messages from the topic, up to the selected maximum amount.
  - To view the earliest messages from the topic, select From beginning.
  From offset
  - To view messages starting from an offset, select a partition and then select From offset and enter an offset.
  From timestamp
  When you seek to a specified time, message browser reads messages from the selected time onward until it reaches the maximum number of results.
  - To jump to a message timestamp, select From timestamp and enter the timestamp in milliseconds.
    Tip
    To convert timestamps between human-readable date and time and milliseconds, use a timestamp converter.

Logical type rendering in message browser

By default, message browser displays Apache Avro® decimal logical types as raw bytes. To render decimal fields as decimal values, set the Avro decimal display drop-down list in the Topics > Messages pane to Decimal.

All other Avro logical types (such as date, time-millis, time-micros, timestamp-millis, timestamp-micros, local-timestamp-millis, local-timestamp-micros, and duration) appear as their underlying Avro type. For example, a timestamp-millis field appears as a long (milliseconds since the Unix epoch), and a date field appears as an int (days since the Unix epoch).

When you produce messages from Cloud Console, enter logical type values using each type’s underlying Avro type (for example, enter a timestamp-millis value as a long).

Produce new messages to topics

You can produce messages to topics using Cloud Console. Use Cloud Console for any of the following message-related tasks:

Produce messages to a topic
Validate messages you create for topics that enforce a schema
Determine if a topic enforces a schema
Add headers to messages

Considerations

Cloud Console schema validation does not respect Schema Registry data contract rules.
Cloud Console works with all Schema Registry draft versions of JSON Schema.
JSON schemas of one draft type that reference JSON schemas of a different draft type are not fully supported and might cause issues.
While Protocol Buffers (protobuf) supports nested messages, Cloud Console does not. Cloud Console validates only top-level messages. This means that, while you can define another message type inside a protobuf message, Cloud Console only validates (and allows you to select) the top-level message.
Proto3 version of protobuf revokes support for required fields. This limit isn’t unique to Cloud Console. This note is a reminder that validation ignores optional fields in protobuf messages and only applies to defined messages.
Do not use Cloud Console to generate messages where you use bytes or fixed types as data types.
If Cloud Console cannot produce a message for the schema, it produces an error message.
Cloud Console requires that you wrap unions in Avro schemas to specify the type. A union type needs to specify whether the type of the content is null or string.
Cloud Console uses the AVSC library for handling Avro schemas. This library’s handling might differ slightly from handling by Java and librdkafka clients.

Produce message with schema

To produce new messages with a schema:

From Actions, select Produce new message.
(Optional) Select a different schema from Select schema under Key or Value.
Enter your message or select Generate message to have Cloud Console create a message based on your schema.
(Optional) Select Validate.
(Optional) Select Headers and then select Add header.
1. Enter your header.
2. (Optional) Add more headers as necessary.
Select Produce.

Produce message without schema

To produce new messages without a schema:

From Actions, select Produce new message.
In Select schema under Key or Value, select Produce without schema and then select Continue to dismiss the prompt.
Enter your message.
(Optional) Select Headers and then select Add header.
1. Enter your header.
2. (Optional) Add more headers as necessary.
Select Produce.

Inspect schema enforcement

To determine if the current topic enforces a schema:

From Actions, select Produce new message.
To determine if the topic enforces a schema, review Select schema for Key and Value:
- No schema available - no schema is associated with Key or Value
- Default - <topic-name>-key|value - the default schema associated with Key or Value
- <qualified-subject> - <topic-name>-key|value - additional schemas associated with Key or Value
If multiple schemas are associated with a topic for serialization or deserialization, Cloud Console uses a topic name strategy to organize them.

Download topic messages

Use the bulk download option to download the currently displayed set of results.

Considerations

You can download messages as JSON or CSV.
Filter results to limit what you download.
Use the message shown count indicator to determine how many messages of the current set you would download. For example, if you set the maximum results to 50 (the default) and you filter 25 messages, you see a message that reads 25 of 50 messages shown. This message indicates that you download only 25 messages.

To download messages:

Select the download icon for JSON or CSV. Cloud Console downloads a file that contains the selected messages in the selected format to your downloads directory.

Example filenames:

Download messages as JSON_2023-12-05T00_47_57.084Z.json
Download messages as CSV_2023-12-05T16_16_49.815Z.csv

Example JSON content:

[
   {
      "partition": 1,
      "offset": 1511633,
      "timestamp": 1701793270900,
      "timestampType": "CREATE_TIME",
      "key": 90687471,
      "value": {
            "viewtime": 90687471,
            "userid": "User_1",
            "pageid": "Page_60"
      },
      "headers": [
            {
               "key": "task.generation",
               "value": "25"
            },
            {
               "key": "task.id",
               "value": "0"
            },
            {
               "key": "current.iteration",
               "value": "9068747"
            }
      ],
      "exceededFields": null
   },
   {
      "partition": 1,
      "offset": 1511610,
      "timestamp": 1701793199427,
      "timestampType": "CREATE_TIME",
      "key": 90686071,
      "value": {
            "viewtime": 90686071,
            "userid": "User_8",
            "pageid": "Page_60"
      },
      "headers": [
            {
               "key": "task.generation",
               "value": "25"
            },
            {
               "key": "task.id",
               "value": "0"
            },
            {
               "key": "current.iteration",
               "value": "9068607"
            }
      ],
      "exceededFields": null
   }
]