Query Tableflow Tables with Flink in Confluent Cloud for Apache Flink®
Confluent Cloud for Apache Flink® supports snapshot queries that read data from a Tableflow-enabled topic at a specific point in time.
Querying a Tableflow-enabled topic is similar to querying a Flink topic.
If Tableflow is enabled on a topic with Confluent Managed Storage, the query reads from both Kafka and Parquet.
If Tableflow is enabled on a topic with custom storage, the query reads from your S3 bucket.
This guide shows how to run a snapshot query on a Tableflow-enabled topic.
Note
Snapshot query is an Open Preview feature in Confluent Cloud for Apache Flink.
A Preview feature is a Confluent Cloud component that is being introduced to gain early feedback from developers. Preview features can be used for evaluation and non-production testing purposes or to provide feedback to Confluent. The warranty, SLA, and Support Services provisions of your agreement with Confluent do not apply to Preview features. Confluent may discontinue providing preview releases of the Preview features at any time in Confluent’s’ sole discretion.
Prerequisites
Access to Confluent Cloud.
The OrganizationAdmin, EnvironmentAdmin, or FlinkAdmin role for creating compute pools, or the FlinkDeveloper role if you already have a compute pool. If you don’t have the appropriate role, contact your OrganizationAdmin or EnvironmentAdmin. For more information, see Grant Role-Based Access in Confluent Cloud for Apache Flink.
A provisioned Flink compute pool.
Step 1: Enable Tableflow on your topic
If you want to try querying a table with mock data, complete the steps in Run a Snapshot Query, then proceed to the next step.
If you want to query a table with mock data or data from your Kafka topic, and you want to use Confluent Managed Storage, complete the following steps, then proceed to Step 2: Run a snapshot query with Tableflow.
In Confluent Cloud Console, navigate to your cluster.
In the navigation menu, click Topics.
In the topics list, find your topic and click it to open the details page.
Click Enable Tableflow.
In the Enable Tableflow dialog, select Iceberg and click Use Confluent storage.
The topic status updates to Tableflow Syncing.
If you want to query a table with data from your Kafka topic, and you want to use custom storage, complete steps 1-4 in Tableflow Quick Start Using Your Storage and AWS Glue and proceed to Step 2: Run a snapshot query with Tableflow.
Step 2: Run a snapshot query with Tableflow
Once Tableflow is enabled on your topic, you can run a snapshot query on the table by using the same statements that you use for Flink tables.
In a Flink workspace or the Flink SQL shell, prepend your query with the following SET statement:
SET 'sql.snapshot.mode' = 'now';
Also, in the Flink workspace, you can change the Mode dropdown setting to Snapshot before running your query.
For more information, see Run a Snapshot Query.