# Tableflow Quick Start with Iceberg Tables Using Your Storage and AWS Glue in Confluent Cloud Confluent Tableflow exposes Apache Kafka® topics as Apache Iceberg™ tables. Iceberg is an open table format for large analytic datasets in object storage. Complete these steps to materialize a Kafka topic as an Iceberg table using your own Amazon S3 storage and the AWS Glue Data Catalog: - [Step 1: Create a topic and publish data](#cloud-tableflow-quick-start-create-topic) - [Step 2: Configure your S3 bucket and provider integration](#cloud-tableflow-quick-start-configure-bucket) - [Step 3: Enable Tableflow on your topic](#cloud-tableflow-quick-start-enable-tableflow) - [Step 4: Configure AWS Glue Data Catalog integration](#cloud-tableflow-quick-start-credentials) - [Step 5: Query Iceberg tables](#cloud-tableflow-quick-start-query-iceberg) - [Step 6: Query data with other analytics engines (optional)](#cloud-tableflow-quick-start-query-analytics-engines) ## Prerequisites - `DeveloperRead` access on all schema subjects. - `CloudClusterAdmin` access on your Kafka cluster. - `Assigner` access on all provider integrations. For more information, see [Grant Role-Based Access for Tableflow in Confluent Cloud](../operate/tableflow-rbac.md#tableflow-rbac). ## Step 1: Create a topic and publish data In this step, you create a `stock-trades` topic in Confluent Cloud Console. 1. Click **Create topic**, provide the topic name, and create it with default settings. You can skip defining a contract. 2. Publish data to the `stock-trades` topic with the Datagen Source connector and the **Stock Trades** dataset. 3. When you configure the Datagen Source connector, click **Additional configuration** and proceed through the provisioning workflow. 4. When you reach the **Configuration** step, in the **Select output record value format** field, select **Avro**. Click **Continue** and keep the default settings. For more information, see [Datagen Source Connector Quick Start](../../../connectors/cc-datagen-source.md#cc-datagen-source-st-qs). ## Step 2: Configure your S3 bucket and provider integration Configure the storage bucket that holds the table data before you materialize your Kafka topic as an Iceberg table. Create a Confluent Cloud [provider integration](../../../connectors/provider-integration.md#cloud-pi-quickstart) to access your Amazon S3 bucket and write materialized data into it. A provider integration is a Confluent Cloud resource that grants access to your AWS account. 1. In the AWS Management Console, [create an S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/GetStartedWithS3.html#creating-bucket) in your preferred AWS account. Name the bucket `tableflow-quickstart-storage`. 2. In your Confluent Cloud environment, navigate to the **Provider integrations** tab to create a provider integration and grant Confluent Cloud access to your S3 bucket. 3. Click **Add integration**. The **Add integration details** page opens. 4. Select **AWS IAM role** and click **Continue**. The **Configure role in AWS** page opens. 5. Select **New role** and click **Continue**. 6. Select **Tableflow S3 Bucket** and copy the AWS Identity and Access Management (IAM) policy template.  7. In the AWS Management Console, navigate to IAM. 8. In the **Access Management** section, click **Policies**, and on the **Policies** page, click **Create Policy**. As a best practice, create a designated IAM policy that grants Confluent Cloud access to your S3 location. 9. Paste the IAM policy template you copied earlier. Update it with the name of your S3 bucket, for example, `tableflow-quickstart-storage`, and create a new AWS IAM policy.  #### NOTE If your S3 bucket uses AWS Key Management Service (KMS) server-side encryption (SSE-KMS or DSSE-KMS), you must add a statement to the KMS key policy that allows Confluent Cloud to use the key. If the KMS key is in a different AWS account from your S3 bucket, you must also grant KMS actions to the IAM role in your permission policy. For more information, see [Amazon S3 storage for Tableflow](../how-to-guides/configure-storage.md#cloud-tableflow-storage-byos-s3) and [Use self-managed encryption keys](../../../security/encrypt/byok/tableflow-byok.md#tableflow-byok-integration). 10. Navigate to AWS **IAM Roles** and click **Create Role**. 11. For the **Trusted entity type**, select **Custom trust policy**. 12. From the Tableflow UI in Cloud Console, copy the `Trust-policy.json` JSON file and paste it into the policy editor in the AWS Management Console. 13. Attach the permission policy you created earlier and save your new IAM role, for example, `tableflow-quickstart-role`. 14. Copy the role Amazon Resource Name (ARN), for example, `arn:aws:iam:::role/tableflow-quickstart-role`. 15. In the Cloud Console, locate the **Map the role in Confluent** section on the **Provider integrations** tab. In the **AWS ARN** section, paste the ARN you copied and click **Continue**.  16. After creating the provider integration, update the trust policy of the AWS IAM role, for example `tableflow-quickstart-role`, by using the policy displayed in Cloud Console.  ## Step 3: Enable Tableflow on your topic You can now enable Tableflow on your Kafka topic to materialize it as an Iceberg table in the storage bucket that you created in [Step 2](#cloud-tableflow-quick-start-configure-bucket). 1. Navigate to your `stock-trades` topic and click **Enable Tableflow**. 2. In the **Enable Tableflow** dialog, click **Configure custom storage**. 3. In the **Choose where to store your Tableflow data** section, click **Store in your bucket**. 4. In the **Provider integration** list, select the provider integration that you created in [Step 2](#cloud-tableflow-quick-start-configure-bucket). Enter the name of the storage bucket that you created, which in this example is `tableflow-quickstart-storage`. 5. Click **Continue** to review the configuration and launch Tableflow. Materializing a newly created topic as an Iceberg table can take a few minutes. Tableflow optimistically tries to publish data every 5 minutes for low-throughput topics whose Kafka segments aren’t full. Tableflow makes this attempt on a best-effort basis without a delivery guarantee. ## Step 4: Configure AWS Glue Data Catalog integration Follow the steps in [Integrate Tableflow with the AWS Glue Catalog](../how-to-guides/catalog-integration/integrate-with-aws-glue-catalog.md#cloud-tableflow-integrate-with-aws-glue-catalog) to configure the AWS Glue Data Catalog as a catalog integration. Confluent Cloud automatically creates the `stock-trades` Iceberg table and a database in the AWS Glue Data Catalog after you configure the catalog integration. Tableflow bases the database name on your Kafka cluster ID. It can take a few minutes for Tableflow to publish Iceberg tables to the AWS Glue Data Catalog. ## Step 5: Query Iceberg tables You can use any AWS Glue and Iceberg-compatible compute engine to query the table. This example uses Amazon Athena SQL. Follow the steps in [Amazon Athena SQL](../how-to-guides/query-engines/query-with-aws.md#cloud-tableflow-query-with-aws-athena) to start writing queries for the `stock-trades` table. ## Step 6: Query data with other analytics engines (optional) Explore other integration options for using Tableflow with analytics engines: - [Query with AWS Analytics Services](../how-to-guides/query-engines/query-with-aws.md#cloud-tableflow-query-with-aws) - [Query with Snowflake](../how-to-guides/query-engines/query-with-snowflake.md#cloud-tableflow-query-with-snowflake) - [Query with Trino](../how-to-guides/query-engines/query-with-trino.md#cloud-tableflow-query-with-trino) ## Related content For more information, see [Stream Processing with Confluent Cloud for Apache Flink](../../../flink/overview.md#ccloud-flink). #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).