# Monitor Tableflow in Confluent Cloud You monitor Tableflow by using the Confluent Cloud Console to track the status and progress of each Tableflow-enabled topic. You can also set up integrations with monitoring services like Prometheus and Datadog, and query metrics through the Confluent Cloud Metrics API. Tableflow runs after you enable it on an Apache Kafka® topic. ## Monitor Tableflow in Cloud Console 1. Log in to the Cloud Console. 2. Navigate to the cluster that has your Tableflow enabled topic. 3. Select the **Topics** tab, and under **Show/hide columns**, ensure Tableflow is selected. The topics that have Tableflow enabled and the syncing status of Tableflow-enabled topics are displayed. Ensure that Tableflow is enabled on at least 1 topic. 4. Select a Tableflow-enabled topic. The Tableflow status under the topic name displays. Also, important Tableflow information is shown as cards in the Tableflow section, under the **Kafka stats** section. 5. To see cluster-level metrics, including storage details, click the **Tableflow** tab. ## Tableflow status Once Tableflow is enabled on a topic, Tableflow emits a general status. - **Config Issue:** There was a problem with the specified configurations. More details about the issue are shown in an additional error field. - **Degraded:** There is an internal issue with Tableflow. More details about the issue are shown in an additional error field. - **Pending:** Tableflow is enabling for the first time and checking configurations. - **Running/Syncing:** Tableflow is successfully syncing incoming Kafka data to the Apache Iceberg™ table. ## Tableflow metrics Tableflow emits the following metrics over the Confluent Cloud [Metrics API](../../../monitoring/metrics-api.md#metrics-api). Metrics marked **(Preview)** are in Preview and can change before General Availability. Bytes Compacted : Metric name: `bytes_compacted` The number of bytes compacted. Bytes Processed : Metric name: `bytes_processed` The number of bytes processed by Tableflow. This value includes the bytes of Kafka data read before materialization and bytes read as part of compaction. Bytes Written : Metric name: `bytes_added` The number of bytes written to the table by Tableflow. Compaction Duration : Metric name: `compaction_duration` The amount of time taken for compaction. Compactions Pending : Metric name: `compactions_pending` The number of pending compactions. Confluent Managed Storage used : Metric name: `storage` The amount of data managed by Confluent-owned buckets for Tableflow. Event Freshness Latency (Preview) : Metric name: `event_freshness_latency_milliseconds` The age of the most recently materialized record for a Tableflow-enabled topic, measured as the time between the source record’s timestamp and when Tableflow writes it to the destination. Reported per destination when a topic syncs to more than one destination (Iceberg or Delta Lake table format, or a catalog such as AWS Glue, Snowflake Open Catalog, or Unity Catalog). Files Compacted : Metric name: `files_compacted` The total number of files that have been used for compaction by Tableflow. Kafka Bytes Read : Metric name: `bytes_read` The number of bytes read by Tableflow. Kafka Rows Read : Metric name: `rows_read` The number of records read for a Tableflow enabled topic. Latest Kafka Topic Offset : Metric name: `kafka_topic_offset` The latest offset of the read record from Kafka for a Tableflow-enabled topic. Latest Table Offset : Metric name: `table_offset` The latest offset of the persisted record for a Tableflow-enabled topic. Number of Topics : Metric name: `num_topics` The total number of Tableflow-enabled topics. Number of Rejected Rows : Metric name: `rows_skipped` The total number of Kafka records that were not able to be processed and added to the table by Tableflow. Processing Freshness Latency (Preview) : Metric name: `processing_freshness_latency_milliseconds` The processing latency for a Tableflow-enabled topic, measured as the time between when Tableflow picks up a record for processing and writes it to the destination. Reported per destination when a topic syncs to more than one destination (Iceberg or Delta Lake table format, or a catalog such as AWS Glue, Snowflake Open Catalog, or Unity Catalog). Rows Written : Metric name: `rows_added` The total number of rows that have been committed to the table by Tableflow. Snapshots Generated : Metric name: `snapshots_generated` The total number of Iceberg snapshots generated to the table by Tableflow. ## Best practices for Tableflow alerting Use the [Metrics API](../../../monitoring/metrics-api.md#metrics-api) to monitor your Tableflow-enabled topics over time. You should monitor and configure alerts for the following conditions: - Per topic - Alert on Tableflow status changes. - Alert on unexpected increases in the number of rows rejected. - Per Cluster - Alert on unexpected changes to storage used. ## Tableflow log categories and events You can access Tableflow logs by exporting them through Observability Integrations, an Early Access feature. For setup steps and current limitations, see [Manage Observability Integrations on Confluent Cloud](../../../monitoring/manage-observability-integrations.md#manage-observability-integrations). Some log events include identifying details, such as topic names, catalog names, or namespaces, that might be sensitive depending on your organization’s naming conventions. Tableflow emits structured logs across the full lifecycle of a Tableflow-enabled topic, so you can troubleshoot issues and monitor Tableflow health. Each log event has a severity level and includes identifying details, such as the topic name, table format, and catalog integration. The following table describes the Tableflow log categories and events. For more information about the degraded, pending, running, and syncing states referenced in the topic health events, see [Tableflow status](#tableflow-status). | Category | Event | Level | |-----------------------|-------------------------------------------------------------------|---------| | Lifecycle | Tableflow enabled on a topic | Info | | Lifecycle | Tableflow disabled on a topic | Info | | Topic health | Topic enters the running or syncing state | Info | | Topic health | Topic enters a failed or degraded state | Error | | Topic health | Topic recovers from a failed or degraded state | Info | | Topic health | Topic is stuck in the pending state | Error | | Topic-to-catalog sync | Table is registered in the external catalog | Info | | Topic-to-catalog sync | Table is updated in the external catalog | Debug | | Topic-to-catalog sync | Catalog sync for a topic fails while the connection stays healthy | Error | To view per-topic catalog sync status and failure details in the Cloud Console, see [Topic catalog sync status](../how-to-guides/catalog-integration/overview.md#cloud-tableflow-topic-catalog-sync-status). ## Tableflow notification types Tableflow integrates with Confluent Cloud notifications, so you can stay informed about the state of your Tableflow-enabled topics without polling the Cloud Console or querying the Metrics API. Tableflow supports the following notification types: | Notification | Severity level | When it fires | |----------------------------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Tableflow started syncing | Info | The topic starts syncing successfully for the first time, or
recovers after a previous failure. | | Tableflow syncing failed | Critical | Materialization stops because of a failure, either during initial
enablement or after a previously healthy topic breaks due to a data
or configuration change. | | Tableflow stuck in pending state | Warn | The topic remains in the `Pending` state for an hour or longer. | | Tableflow topic degraded | Critical | On a topic configured with more than one table format, one format
(Iceberg or Delta Lake) fails while the other continues syncing.
The notification identifies the failed format. | | External catalog sync failed | Critical | A catalog integration (AWS Glue, Snowflake Open Catalog, Apache
Polaris, or Unity Catalog) enters a failed or disconnected state,
for example due to invalid credentials or insufficient permissions. | If you have the ResourceOwner role on a Tableflow-enabled topic, you can subscribe to notifications for it. For steps to configure integrations and manage subscriptions, see [Notifications for Confluent Cloud](../../../monitoring/configure-notifications.md#ccloud-notifications). ## Tableflow error and recovery If Tableflow encounters an issue, its status switches to `Config Issue` or `Degraded`. ### Tableflow enters the `Suspend` state due to a recoverable failure Tableflow enters the `Suspend` state when it encounters an error that has a high chance of requiring user intervention, or when materialization fails and the `Failure` mode is set to `Suspend`. Tableflow remains in the `Suspend` state until manually resumed by sending a PATCH request to the API with the `spec.suspend` property set to `true`. ### Recovery after unrecoverable failure There are situations in which Tableflow is unable to write new data to an existing table, for example, when your table objects are corrupted or missing, or you make a table-breaking change to your schema used on incoming messages. First, back up your data by using a query engine to copy your data to a new table, or by copying the objects directly from your storage to another location. It’s important to copy your data to a location outside the location where Tableflow is managing the table, to ensure it doesn’t get deleted as part of Tableflow cleaning up storage. You can then disable Tableflow on the topic. Tableflow removes the associated table files asynchronously, according to the table’s snapshot retention configuration. If you need to re-enable Tableflow on this topic before cleanup finishes, contact [support](https://support.confluent.io/hc/en-us/requests/new). For more information, see [Disable Tableflow](../concepts/tableflow-storage.md#tableflow-storage-disable). ## Related content - [Configure Tableflow](configure-tableflow.md#tableflow-configure) - [Tableflow Data Plane Auditable Event Methods](../../../monitoring/audit-logging/event-methods/tableflow-data-plane.md#tableflow-data-plane-auditable-events) - [Grant Role-Based Access](tableflow-rbac.md#tableflow-rbac) - [Tableflow Storage](../concepts/tableflow-storage.md#tableflow-storage) - [Notifications for Confluent Cloud](../../../monitoring/configure-notifications.md#ccloud-notifications) - [Topic catalog sync status](../how-to-guides/catalog-integration/overview.md#cloud-tableflow-topic-catalog-sync-status) - [Manage Observability Integrations on Confluent Cloud](../../../monitoring/manage-observability-integrations.md#manage-observability-integrations) #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).