Query Tableflow Tables with Flink in Confluent Cloud for Apache Flink®
Confluent Cloud for Apache Flink® supports snapshot queries that read data from a Tableflow-enabled topic at a specific point in time. Snapshot queries work on Tableflow tables in both Iceberg and Delta Lake formats.
Querying a Tableflow-enabled topic is similar to querying a Flink topic. If Tableflow is enabled on a topic, a snapshot query reads the historical data that Tableflow has materialized into tables, unioned with the live streaming data in Kafka. This is true for both Confluent Managed Storage and custom storage.
This guide shows how to run a snapshot query on a Tableflow-enabled topic.
Prerequisites
Access to Confluent Cloud.
The OrganizationAdmin, EnvironmentAdmin, or FlinkAdmin role for creating compute pools, or the FlinkDeveloper role if you already have a compute pool. If you don’t have the appropriate role, contact your OrganizationAdmin or EnvironmentAdmin. For more information, see Grant Role-Based Access in Confluent Cloud for Apache Flink.
A provisioned Flink compute pool.
Step 1: Enable Tableflow on your topic
If you want to try querying a table with mock data, complete the steps in Run a Snapshot Query, then proceed to the next step.
If you want to query a table with mock data or data from your Kafka topic, and you want to use Confluent Managed Storage, complete the following steps, then proceed to Step 2: Run a snapshot query with Tableflow.
In Confluent Cloud Console, navigate to your cluster.
In the navigation menu, click Topics.
In the topics list, find your topic and click it to open the details page.
Click Enable Tableflow.
In the Enable Tableflow dialog, select Iceberg and click Use Confluent storage.
The topic status updates to Tableflow Syncing.
Confluent Managed Storage supports only Iceberg tables. To use Delta Lake tables, use custom storage.
If you want to query a table with data from your Kafka topic, and you want to use custom storage, do one of the following, then proceed to Step 2: Run a snapshot query with Tableflow.
For Iceberg tables, complete steps 1-4 in Tableflow Quick Start Using Your Storage and AWS Glue.
For Delta Lake tables, complete steps 1-3 in Tableflow Quick Start with Delta Lake Tables in Confluent Cloud.
Step 2: Run a snapshot query with Tableflow
Once Tableflow is enabled on your topic, you can run a snapshot query on the table by using the same statements that you use for Flink tables.
In a Flink workspace or the Flink SQL shell, prepend your query with the following SET statement:
SET 'sql.snapshot.mode' = 'now';
Also, in the Flink workspace, you can change the Mode dropdown setting to Snapshot before running your query.
For more information, see Run a Snapshot Query.