Use Message Browser in Confluent Cloud¶

In Confluent Cloud Console, message browser enables you to view, produce, and download messages produced to a topic. Message browser enables you to browse message data across all partitions, seek data from specific offsets or timestamps by partition, and produce and download messages. Messages can be produced with or without a schema. Messages with a schema can be validated by Cloud Console before you produce them to the topic.

You pay for data consumed and produced to your cluster by message browser. In terms of cost, message browser is like any other client you might deploy.

Considerations:

By default, message browser displays newer data at the top of the messages pane.
Filter across any of the available columns.
You can only filter results that are displayed. To add more results to the current display, increase the maximum results, manipulate the start time, or add partitions.
Message browser displays messages that have associated schemas. Where multiple schema contexts are available you can select between contexts with a drop down list. Only TopicNameStrategy schemas are used to deserialize messages.
Non-compacted topics show a tile for Total messages while compacted topics show a tile for Retained bytes.

Prerequisites:

You must have data being produced to a topic or already present in a topic to use message browser.

Inspect topics¶

Message browser enables you to inspect topic details and browse live streaming messages. If you don’t already have data flowing through the topic, use the built-in produce new message to topic feature to send a message to the topic. You can replay old data that was already produced as long as it hasn’t been compacted or deleted yet.

To browse a live stream of messages and inspect a topic:

Sign in to Confluent Cloud.
If you have more than one environment, select an environment.
Select a cluster.
Select Topics in the navigation menu. The Topics page appears.
Select the topic you want to inspect.
Select Messages. The Messages page displays incoming streaming messages for the topic.

Why does message browser not show messages equal to maximum results?

Message browser may not always display results equal to the maximum results selected. This is by design and can happen for a variety of reasons based on your filtering selections and the ordering of data in the topic. For example, your topic may have thousands of total messages, maximum results set to 1000, but message browser might display less than 1000 results. Change Latest to From beginning to see results from the beginning of the topic up to the 1000 maximum result limit. You could also select a specific time, an offset, or a timestamp to fetch a different set of results up to the maximum.
To browse the messages pane use the following features:
- Max results drop down changes the maximum displayed results.
- Auto-refresh toggle pauses and resumes message browser updates. The topic continues to receive any messages produced during a pause, but message browser pauses the display.
- Filter by timestamp, offset, key or value filter results by the specified criteria. Enter a timestamp in Unix epoch format. Enter an offset, key, or any string value to filter by that value.
- Select a row to access the contents of the message. A message detail pane opens. In Message details, select Key, Value, or Headers.
- Use From beginning, From offset, From timestamp, or From specific time to refine message searches.
  From beginning, offset, timestamp, or specific time details
  From beginning
  
  By default, the latest messages are displayed upon first entering message browser. You can instead view the earliest messages from the topic, up to the selected maximum amount.
  - To view the earliest messages from the topic, select From beginning.
  From offset
  - To view messages starting from an offset, select a partition and then select From offset and enter an offset.
  From timestamp
  
  When you seek to a specified time, message browser reads messages from the selected time onwards until the maximum limit of results is reached.
  - To jump to a message timestamp, select From timestamp and enter the timestamp in milliseconds.
    
    Tip
    
    To convert timestamps between human-readable date and time and milliseconds, use a timestamp converter .
  From specific time
  1. Select the From specific time menu.
  2. Select the default current date and time box to open a calendar.
  3. Select a calendar date. Use the arrows to go forward or back and select a month and day.
  4. Select the time box to adjust the time in hours, minutes, seconds, and AM or PM.
  5. Select Apply.

Produce new messages to topics¶

You can produce messages to topics using Cloud Console. Use Cloud Console for any of the following message related tasks:

Produce messages to a topic
Validate messages you create for topics that enforce a schema
Determine if a topic enforces a schema
Add headers to messages

Considerations:

Cloud Console schema validation does not respect Schema Registry data contract rules.
Cloud Console supports all the Schema Registry supported draft versions of JSON Schema.
JSON schemas of one draft type that reference JSON schemas of a different draft type are not fully supported and may cause issues.
While Protocol Buffers (protobuf) supports nested messages, Cloud Console does not. Only top-level messages are validated. This means while you can define another message type inside a protobuf message, Cloud Console only validates (and allows you to select) the top-level message.
Proto3 version of protobuf revokes support for required fields. This limit isn’t unique to Cloud Console but is mentioned here as a reminder that validation ignores optional fields in protobuf messages and only applies to defined messages.

To produce new messages with a schema:

From Actions, select Produce new message.
(Optional) Select a different schema from Select schema under Key or Value.
Enter your message.
(Optional) Select Validate.
(Optional) Select Headers and then select Add header.
1. Enter your header.
2. (Optional) Add more headers as necessary.
Select Produce.

Download topic messages¶

Use the bulk download option to download the currently displayed set of results.

Considerations:

You can download messages as JSON or CSV.
Filter results to limit what you download.
Use the message shown count indicator to determine how many messages of the current set you would download. For example, if limit maximum results is set to 50 (the default) and you filter 25 messages, you see a message that reads 25 of 50 messages shown. This message indicates that only you will only download 25 messages.

To download messages:

Select the download icon for JSON or CSV. A file that contains the selected messages in the selected format is downloaded into your downloads directory.

Example filenames:

Download messages as JSON_2023-12-05T00_47_57.084Z.json
Download messages as CSV_2023-12-05T16_16_49.815Z.csv

Example JSON content:

[
   {
      "partition": 1,
      "offset": 1511633,
      "timestamp": 1701793270900,
      "timestampType": "CREATE_TIME",
      "key": 90687471,
      "value": {
            "viewtime": 90687471,
            "userid": "User_1",
            "pageid": "Page_60"
      },
      "headers": [
            {
               "key": "task.generation",
               "value": "25"
            },
            {
               "key": "task.id",
               "value": "0"
            },
            {
               "key": "current.iteration",
               "value": "9068747"
            }
      ],
      "exceededFields": null
   },
   {
      "partition": 1,
      "offset": 1511610,
      "timestamp": 1701793199427,
      "timestampType": "CREATE_TIME",
      "key": 90686071,
      "value": {
            "viewtime": 90686071,
            "userid": "User_8",
            "pageid": "Page_60"
      },
      "headers": [
            {
               "key": "task.generation",
               "value": "25"
            },
            {
               "key": "task.id",
               "value": "0"
            },
            {
               "key": "current.iteration",
               "value": "9068607"
            }
      ],
      "exceededFields": null
   }
]