Data Flow

The Data flow feature provides a visual representation of the data flow paths (edges) between producers, topics, and consumers within a cluster. You can inspect the flow throughput metrics between producers, topics, consumer groups and consumers.

Important

  • This feature is available as a preview feature. A preview feature is a component of Confluent Platform that is being introduced to gain early feedback from developers. This feature can be used for evaluation and non-production testing purposes or to provide feedback to Confluent.
  • Your VPC must communicate with the Confluent Cloud Data Flow public internet endpoint to use the preview feature.
Prerequisites
You must have data being produced and consumed from a topic for the data flow interactive diagram to be displayed. For best results, select a cluster that has existing topics that are actively producing and consuming message data. For an example of how to create topics, and produce to and consume from topics on an Apache Kafka® cluster, see the quick start.
  • Select a cluster from the navigation bar and click Data flow. From left to right, producers, topics, and consumers are shown.

    ../../_images/ccloud-data-flow-landing.png

    Data flow landing page

  • Click a producer, topic, or consumer and then Inspect for more information.

    ../../_images/ccloud-data-flow-inspect-producers.png

    Data flow inspect producer

  • Hover over producers, topics, and consumers to view the flows between them.

    ../../_images/ccloud-data-flow-chart-hover.png
  • Click Show partitions to view the partitions for a topic and the percentage of messages consumed from a producer.

    ../../_images/ccloud-data-flow-inspect-topics-parts.png

    Data flow inspect, show, and hide partitions in topics

  • Click Show consumers to view the consumers within a consumer group.

    ../../_images/ccloud-data-flow-inspect-consumers-2.png

    Data flow inspect, show, and hide consumers in a consumer group