# Snapshot Queries in Confluent Cloud for Apache Flink In Confluent Cloud for Apache Flink®, a *snapshot query* is a query that reads data from a table at a specific point in time. In contrast with a streaming query, which runs continuously and returns results incrementally, a snapshot query runs, returns results, and then exits. Other names for snapshot queries are *point-in-time* or *pull queries*. Currently, a snapshot query always reads the current moment (`AS OF NOW()`). You can’t query an arbitrary point in the past. You can query Kafka topics and Apache Iceberg™ tables by using Confluent [Tableflow](../../topics/tableflow/overview.md#cloud-tableflow). ## Snapshot query uses A snapshot query returns a consistent view of your data at the current point in time, similar to taking a photograph of your data at that moment. This is particularly useful when you need to: * Explore and understand a new dataset before building real-time processing logic. * Generate reports that reflect your data’s state at a specific time. * Analyze historical data for auditing or compliance purposes. * Compare data states across different points in time. * Debug or investigate issues by examining past data states. For example, if you want to know the total number of orders in your system at the current time, you can use a snapshot query. ## Snapshot mode A snapshot query is an ordinary Flink SQL statement that has one additional property, named `sql.snapshot.mode`. To enable snapshot queries, set the `sql.snapshot.mode` property to `now`. You can set this property in the following ways: - **SQL Workspace:** Toggle the **Mode** dropdown to **Snapshot**. - **Flink SQL**: Prepend your query with `SET 'sql.snapshot.mode' = 'now';`. - **Table API:** In the `Cloud.Properties` project file, add `sql.snapshot.mode = now`. - **REST API:** In the statement’s `spec.properties` map, add `"sql.snapshot.mode": "now"`. - **Terraform**: In the statement properties, add `"sql.snapshot.mode" = "now"`. - **Confluent CLI**: In the property map, add `--property "sql.snapshot.mode"="now"`. Snapshot queries use Flink’s batch execution mode, which enables you to run batch processing jobs beside your existing stream processing workloads, within the same Confluent Cloud environment. Also, Confluent Cloud for Apache Flink bounds all sources, which means that Flink processes only a finite set of records up to a specific point in time, rather than continuously processing an infinite stream of incoming data. ## Snapshot write mode By default, snapshot queries guarantee exactly-once delivery when writing results to a sink, in the same way as streaming queries. If your snapshot query doesn’t require this guarantee, you can set the `sql.snapshot.write-mode` property to `fast-write` to improve write performance. The `sql.snapshot.write-mode` property accepts the following values: - `default`: Provides exactly-once delivery guarantees. This is the default value. - `fast-write`: Provides at-least-once delivery guarantees, instead of exactly-once, for improved write performance. #### NOTE The `sql.snapshot.write-mode` property applies only in batch mode, which is how snapshot queries run, so it has no effect on streaming queries. You can set this property in the following ways: - **Flink SQL**: Prepend your query with `SET 'sql.snapshot.write-mode' = 'fast-write';`. - **Table API:** In the `Cloud.Properties` project file, add `sql.snapshot.write-mode = fast-write`. - **REST API:** In the statement’s `spec.properties` map, add `"sql.snapshot.write-mode": "fast-write"`. - **Terraform**: In the statement properties, add `"sql.snapshot.write-mode" = "fast-write"`. - **Confluent CLI**: In the property map, add `--property "sql.snapshot.write-mode"="fast-write"`. For more information about this and other SET options, see [Available SET options](../reference/statements/set.md#flink-sql-set-statement-config-options). ## Compute pool recommendations A snapshot query runs in batch mode and can consume a burst of compute resources before releasing them. This burst can contend with the steadier resource needs of your streaming workloads. Confluent recommends the following: * Run snapshot queries in a compute pool that’s separate from the pool that runs your streaming production workloads. For more information, see [Isolation and Resource Sharing](compute-pools.md#flink-sql-compute-pools). * Run your production queries in streaming mode by default, and reserve snapshot queries for on-demand, point-in-time analysis. ## How snapshot queries work When you execute a snapshot query, Flink performs the following steps: 1. Determines the Kafka offsets corresponding to your current timestamp across all partitions 2. Reads data from the source topics up to these offsets 3. Processes the records to build the state of your tables at this point in time 4. Returns the query results based on this state The query execution is optimized to use Kafka’s time index for efficient offset lookup, to leverage parallel processing across partitions, and to minimize the amount of data that needs to be processed. ## Snapshot queries and Tableflow If [Tableflow](../../topics/tableflow/overview.md#cloud-tableflow) is enabled on a topic, snapshot queries on the topic run in a hybrid mode. - If Tableflow is not enabled on a topic, the query reads from Kafka. - If Tableflow is enabled on a topic, the query reads from both Kafka and Parquet, for Confluent Managed Storage and custom storage (BYOS). #### NOTE Tableflow materializes topic data asynchronously, with a [table freshness](../../topics/tableflow/overview.md#cloud-tableflow-table-freshness) of approximately 5 minutes. Immediately after a compatible schema change, the Tableflow-materialized portion of a snapshot query might briefly reflect an earlier schema version than the one most recently registered in Confluent Cloud Schema Registry. The remaining data that the query reads directly from Kafka always reflects the current schema. For more information, see [Schema Compatibility](../../topics/tableflow/concepts/tableflow-schemas.md#cloud-tableflow-schemas-schema-compatibility). ## Run a snapshot query To run a snapshot query, in a Flink workspace or the Flink SQL shell, prepend your query with the following SET statement: ```sql SET 'sql.snapshot.mode' = 'now'; ``` Also, in a Flink workspace, you can change the **Mode** dropdown setting to **Snapshot**. For more information, see [Run a Snapshot Query](../how-to-guides/run-snapshot-query.md#flink-sql-run-snapshot-query). ## Technical details * **Timestamp resolution**: Flink processes timestamps with millisecond precision. * **State handling**: For tables with state (such as aggregations), Flink reconstructs the state by processing all relevant records up to the specified timestamp. * **Parallelism**: Flink automatically parallelizes queries across available compute resources. * **Resource optimization**: Flink uses Kafka’s time index to quickly locate the relevant offsets, minimizing unnecessary data scanning. ## Relationship to batch mode Snapshot queries are closely related to Flink’s batch processing mode. When you execute a snapshot query: * Flink automatically switches to batch mode processing. * The query processes a finite, bounded dataset up to the current timestamp. * The computation benefits from batch optimizations such as sort-merge joins. * Flink releases resources after the query completes. * Results are deterministic and reproducible. This behavior contrasts with streaming queries, which: * Process continuous, unbounded data streams. * Maintain persistent state and resources. * Produce incremental, real-time results. * Can give different results when rerun due to new data. ## Limitations * **Hints are not supported**: SQL hints are ignored in snapshot mode. For more information, see [SQL Hints](../reference/statements/hints.md#flink-sql-hints). * **Isolation level with Tableflow**: Snapshot queries use `read-committed` as the default isolation level, which aligns with Kafka’s exactly-once semantics (EOS). Flink returns only fully committed transactional messages, so results are consistent and free of partial or aborted writes. When Tableflow is enabled on a topic, snapshot queries read historical data from the Tableflow snapshot and the remaining messages directly from Kafka. Because Tableflow only materializes committed messages, the snapshot portion of the query always reflects only committed data, regardless of the isolation-level setting. `kafka.consumer.isolation-level = read-uncommitted` applies only to the Kafka portion of the query. Results include any uncommitted messages that arrived after the Tableflow snapshot cutoff, but exclude uncommitted messages that predate the snapshot cutoff. If Tableflow is not enabled on a topic, `read-uncommitted` applies to all Kafka reads as expected. * **Unsupported table options**: Flink ignores the following table options in snapshot mode: - [scan.bounded.mode](../reference/statements/create-table.md#flink-sql-create-table-with-scan-bounded-mode) - [scan.bounded.timestamp-millis](../reference/statements/create-table.md#flink-sql-create-table-with-scan-bounded-timestamp-millis) - [scan.startup.mode](../reference/statements/create-table.md#flink-sql-create-table-with-scan-startup-mode) - [scan.startup.timestamp-millis](../reference/statements/create-table.md#flink-sql-create-table-with-scan-startup-timestamp-millis) - [scan.startup.specific-offsets](../reference/statements/create-table.md#flink-sql-create-table-with-scan-startup-specific-offsets) ## Billing Confluent bills snapshot queries in CFUs, the same way it bills streaming queries. For more information, see [Flink Billing](../../billing/billing-dimensions.md#flink-billing). ## Private networking Snapshot queries support the same private networking options as streaming queries. For more information, see [Private Networking for Flink](flink-private-networking.md#flink-sql-private-networking). ## Related content - [Run a Snapshot Query](../how-to-guides/run-snapshot-query.md#flink-sql-run-snapshot-query) - [Query Tableflow Tables with Flink](../../topics/tableflow/how-to-guides/query-engines/query-with-flink.md#cloud-tableflow-query-with-flink) - [Statements](statements.md#flink-sql-statements) - [Compute Pools](compute-pools.md#flink-sql-compute-pools) - [Private Networking for Flink](flink-private-networking.md#flink-sql-private-networking) #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).