The Data flow feature provides a visual representation of the data flow paths (edges) between producers, topics, and consumers within a cluster. Drill into producers, topics, and consumers for more detailed information. Inspect the flow throughput metrics between producers, topics, consumer groups and consumers.
To access Data flow, select a cluster from the navigation bar and click Data flow.
- Your VPC must communicate with the Confluent Cloud Data Flow public internet endpoint to use the preview feature.
- If there is not an existing topic in the cluster, the interactive data flow diagram is not displayed. The flows display when data is being produced to and consumed from a topic. For best results, select a cluster that has existing topics that are actively producing and consuming message data.
All producers are listed to the left of topics.
- Click Inspect for a producer to view metrics such as Total production and Bytes produced per topic.
Hover on an edge (path) between a producer and topic, or a topic and consumer, to view throughput.
Topics are centrally located in the Data flow interactive diagram. When you hover on a topic or one its partitions, the paths to its producers and consumers are emphasized prominently so you can easily see the data flows between them.
Click Inspect for a topic to view metrics such as Total throughput, Bytes in and out per partition, and Bytes written per producer. Hover on a chart to view more detail.
Click Show partitions to view the partitions for a topic and the percentage of messages consumed from a producer.
- Click Hide partitions to collapse the view of the partitions within a topic.
Consumer groups and consumers¶
All consumer groups and consumers are listed to the right of topics. For a given consumer group, the total message lag is visible as well as the message lag for each consumer.
- Click Inspect for a consumer group to view metrics for Total consumption.
- Click Show consumers to view the consumers within a consumer group.
- Click Hide consumers to collapse the view of the consumers within a consumer group.