# Integrate Tableflow with Snowflake Open Catalog or Apache Polaris in Confluent Cloud Snowflake Open Catalog is a catalog solution designed for Apache Iceberg™ tables, leveraging the open-source Iceberg REST protocol. Tableflow seamlessly connects with Snowflake Open Catalog or other Apache Polaris catalog flavors by using catalog integration. ![Tableflow integration with Apache Polaris catalog](topics/tableflow/images/tableflow-apache-polaris.png) Snowflake Open Catalog integrates with Tableflow at the cluster level, enabling the automatic publication of all Tableflow-enabled topics as tables within the Open Catalog. ## Step 1: Configure your Amazon S3 bucket In this step, you create an Amazon S3 bucket to store your Iceberg tables. If you have an S3 bucket already, you can skip to the next step. 1. In the Amazon S3 console, navigate to **General purpose buckets** and click **Create bucket**. 2. Accept the default values and name your bucket. The bucket name must be unique. In this example, the bucket name is `tableflow-s3-bucket`. 3. Click **Create bucket**. ## Step 2: Create the permission policy In this step, you create a permission policy for the storage provider integration. 1. Log in to Confluent Cloud Console and navigate to the environment that has the Kafka cluster with your topics. 2. In the navigation menu, click **Integrations**. The **Integrations** page opens. 3. Click **Add integration**. The **Add integration details** page opens. 4. Select **AWS IAM role** and click **Continue**. The **Configure role in AWS** page opens. 5. Select **New role** and click **Continue**. The **Create permission policy in AWS** page opens. 6. In the **Select Confluent resource** dropdown, select **Tableflow S3 Bucket**. 7. Copy the provided **Permission-policy.json** and click **Continue**. The **Create role in AWS and map to Confluent** page opens. 8. In a new tab, open your Amazon Web Services account and navigate to [IAM > Policies](https://console.aws.amazon.com/iam/home#/policies), or click the **AWS IAM Console** link in Cloud Console. 9. In the **Policies** page, click **Create policy**. The **Specify permissions** page opens. 10. In the **Policy editor** section, select **JSON**. 11. Paste the copied `Permission-policy.json`. 12. Replace all instances of `` with the name of your bucket, for example, `tableflow-s3-bucket`, and click **Next**. The **Review and create** page opens. 13. In the **Policy details** section, name the policy `tableflow-s3-access-policy` and click **Create policy**. ## Step 3: Create the trust policy In this step, you create a role in AWS that uses the policy you created, plus a `trust-policy.json` to allow Confluent to assume this role. 1. In Confluent Cloud Console, return to the **Create role in AWS and map to Confluent** page. 2. Copy the provided **Trust-policy.json**. 3. In the AWS console, navigate to [IAM > Roles](https://console.aws.amazon.com/iam/home#/roles) and click **Create role**. Tableflow assumes this role. 4. Select **Custom Trust Policy**, paste the `trust-policy.json` into the editor, and click **Next**. The **Add permissions** page opens. 5. In the **Permissions policies** list, find the policy you created in the previous step, for example, `tableflow-s3-access-policy`. Attach it to the role, and click **Next**. The **Name, review, and create** page opens. 6. In the **Role details** section, name the role `s3-tableflow-assume-role` and click **Create role**. 7. When the role is created, click **View role** to see the role details. 8. Copy the **ARN** of the role, which resembles `arn:aws:iam:::role/`. 9. In Cloud Console, return to the **Create role in AWS and map to Confluent** page, and in the **Map the role in Confluent** section, paste the **ARN**. 10. Name the role `s3-provider-integration` and click **Continue**. The **Update trust policy in AWS** page opens. 11. Copy the updated `trust-policy.json`, which contains the Confluent External ID and role. 12. In the AWS Console, return to the AWS IAM role you created, for example, `s3-tableflow-assume-role`. 13. Select the **Trust relationships** tab. 14. Click **Edit trust policy** or **Edit trust relationship**. 15. Replace the entire existing JSON with the updated `trust-policy.json` you copied from Cloud Console. This adds the necessary External ID condition. 16. Click **Update policy** or **Save changes**. 17. In Cloud Console, return to the **Update trust policy in AWS** page and click **Continue**. Your new integration is created and appears in the **Integrations** page. ## Step 4: Enable Tableflow on your Kafka topics With the provider integration configured, you can enable Tableflow for your Kafka topics. 1. In Cloud Console, navigate to your environment, then select the Kafka cluster that has your topics. 2. In the navigation menu, click **Topics**, and in the topic list, find the specific topic you want to enable Tableflow for. 3. In the **Tableflow** column, click **Enable Tableflow**. 4. In the **Enable Tableflow** dialog, click **Configure custom storage**. The **Choose where to store your Tableflow data** page opens. 5. In the **Provider integration** dropdown, select your provider integration, for example, `s3-provider-integration`. 6. In the **AWS S3 Bucket name** field, enter the name of your bucket, for example, `tableflow-s3-bucket`, and click **Continue**. 7. Review the details and click **Launch**. The details page for your topic displays **Tableflow sync pending**. After a few seconds the status updates to **Syncing**, which means your topic is connected to the table in your S3 bucket and Tableflow is syncing the data. ## Configure Snowflake Open Catalog integration Follow these steps to enable Snowflake Open Catalog integration at the cluster level. Start by creating an external catalog in Snowflake Open Catalog. #### IMPORTANT Topics must be materialized in order for catalog synchronization to complete. Enable Tableflow on a topic before enabling your external catalog provider. Catalog sync remains in the `pending` state until at least one topic is enabled with Tableflow. 1. Create an IAM policy that grants access to your bucket, for example, `my_snow_table_bucket`, where you plan to materialize Iceberg tables by following the steps in [Create an IAM policy that grants access to your S3 location](https://other-docs.snowflake.com/en/opencatalog/create-catalog#step-1-create-an-iam-policy-that-grants-access-to-your-s3-location). 2. Create an IAM role in AWS by following the steps in [Create an IAM role to grant privileges on your S3 bucket](https://other-docs.snowflake.com/en/opencatalog/create-catalog#step-2-create-an-iam-role-to-grant-privileges-on-your-s3-bucket). Attach the IAM policy you created in the previous step. 3. In Snowflake Open Catalog, create an external catalog, for example, `my_tableflow_ext_catalog`, using a storage bucket, for example, `my_table_bucket`, by following the steps in [Create a catalog in Open Catalog](https://other-docs.snowflake.com/en/opencatalog/create-catalog#step-3-create-a-catalog-in-open-catalog). ![Screenshot showing catalog creation in Snowflake for Tableflow integration](topics/tableflow/images/tableflow-snowflake-create-catalog.png) 4. Retrieve the Snowflake Open catalog `arn` and external ID by following the steps in [Retrieve the AWS IAM user for your Open Catalog account](https://other-docs.snowflake.com/en/opencatalog/create-catalog#step-4-retrieve-the-aws-iam-user-for-your-open-catalog-account). 5. Grant the IAM user access to your bucket following the steps in [Grant the IAM user permissions to access bucket objects](https://other-docs.snowflake.com/en/opencatalog/create-catalog#step-5-grant-the-iam-user-permissions-to-access-bucket-objects). 6. Configure a new service connection for Tableflow and obtain the `client ID` and `client secret` by following the steps in [Configure a service connection](https://other-docs.snowflake.com/en/opencatalog/configure-service-connection#configure-a-service-connection). ![Screenshot showing service connection configuration in Snowflake Open Catalog for Tableflow integration](topics/tableflow/images/tableflow-snowflake-open-catalog-service-connection.png) 7. Set up catalog privileges for the connection by following the steps in [Set up catalog privileges](https://other-docs.snowflake.com/en/opencatalog/tutorials/open-catalog-gs#set-up-catalog-privileges). 8. In Confluent Cloud Console, create a catalog integration in Tableflow by providing the following inputs. - Snowflake Open Catalog URL: `https://.snowflakecomputing.com` - Warehouse name: `` - Client ID: `` - Client secret: `` - Allowed scope: `` 9. After connecting Tableflow with the Snowflake Polaris catalog, navigate to the external catalog that you created and verify that the Iceberg tables are published to the catalog. ## Related content - [Integrate Tableflow with the AWS Glue Catalog](integrate-with-aws-glue-catalog.md#cloud-tableflow-integrate-with-aws-glue-catalog) - [Integrate Tableflow with Unity Catalog](integrate-with-unity-catalog.md#cloud-tableflow-integrate-with-unity-catalog) - [Integrate with Cloud Service Providers in Confluent Cloud](../../../../integrations/provider-integrations/overview.md#provider-integrations-overview) - [Stream Processing with Confluent Cloud for Apache Flink](../../../../flink/overview.md#ccloud-flink) #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).