Integrate with Cloud Service Providers in Confluent Cloud

Provider integrations in Confluent Cloud enable you to securely connect Confluent Cloud resources with your cloud service provider (CSP) environments by leveraging the CSP’s identity and access management (IAM) roles. When you configure a provider integration, you specify an IAM role in your CSP account (such as AWS IAM). Confluent Cloud is then authorized to assume this role using secure, temporary credentials, allowing it to access only the specific resources (like Amazon S3 buckets or Amazon DynamoDB tables) permitted by the role’s policies. This approach eliminates the need for long-term access keys and ensures that Confluent Cloud’s access is tightly scoped and controlled by your CSP’s IAM policies. Access to Confluent Cloud resources themselves continues to be managed by Confluent Cloud access controls, such as access control lists (ACLs) or role-based access control (RBAC).

Note

Provider integrations in Confluent Cloud support AWS IAM and Google Cloud IAM.

Overview of provider integrations

A provider integration is a Confluent Cloud resource that establishes and manages secure connections between Confluent Cloud and third-party cloud services. Each integration is scoped to a specific Confluent Cloud environment, which means you can create separate integrations for development, staging, and production environments within your Confluent Cloud organization.

With provider integrations, you can:

  • Create secure connections between Confluent Cloud and your CSP using IAM-based authentication.
  • Use a single integration across multiple connectors and services within the same environment.
  • Maintain centralized, auditable access control with granular permissions.
  • Automate integration management using REST APIs.
  • Meet compliance requirements with built-in security best practices.

Provider integrations are supported for:

  • AWS IAM roles (create)
  • Google Cloud service accounts (create)
  • Microsoft Entra ID (create)

Prerequisites

Before you can create a provider integration, you need:

In your Confluent Cloud environment

  • OrganizationAdmin or EnvironmentAdmin role to create provider integrations.
  • Access to the target Confluent Cloud environment where you want to create the integration.

In your AWS account

  • Permissions to create and manage IAM roles and policies.
  • Access to the AWS resources you want Confluent Cloud services to use.

In your Azure account

  • Permissions to create and manage service principals and IAM policies.
  • Access to the Azure resources you want Confluent Cloud services to use.

In your Google Cloud account

  • Permissions to create and manage service accounts and IAM policies.
  • Access to the Google Cloud resources you want Confluent Cloud services to use.

Planning requirements

  • Clear understanding of which cloud resources your connectors and Tableflow need to access.
  • Appropriate cloud permissions for your specific use case, for example, Amazon S3, Amazon DynamoDB, Google Cloud Storage, or BigQuery.

For AWS integrations, the IAM role you create must have:

  • Appropriate permissions to access the required AWS resources.
  • A trust relationship that allows Confluent Cloud to assume the role.
  • The correct external ID for security provided during integration setup.

For Google Cloud integrations, the service account you create must have:

  • Appropriate permissions to access the required Google Cloud resources.
  • An impersonation policy that allows Confluent Cloud to impersonate the service account.
  • Appropriate IAM role bindings on the Google service account for the specific Google Cloud services you plan to access.

Use cases

Provider integrations enable secure, credential-free connectivity for the following use cases:

Data integration scenarios

  • Amazon S3 data pipelines - Connect Amazon S3 Sink and Source connectors to your Amazon S3 buckets.
  • Amazon DynamoDB data pipelines - Access Amazon DynamoDB for data synchronization.
  • Amazon Kinesis data pipelines - Stream data to AWS analytics services like Kinesis or EMR.
  • |gc| Storage data pipelines - Connect Google Cloud Storage Sink and Source connectors to your Cloud Storage buckets.
  • BigQuery data pipelines - Stream data to BigQuery for analytics and data warehousing.
  • Cloud SQL data pipelines - Access Cloud SQL databases with IAM authentication for secure data integration.

Security and compliance

  • Eliminate long-term credentials - Replace access keys and service account keys with temporary, scoped IAM roles and service account impersonation.
  • Centralized access control - Manage permissions through your existing cloud IAM policies (AWS IAM or Google Cloud IAM).
  • Audit and monitoring - Track all access through cloud audit logs (AWS CloudTrail or Google Cloud Audit Logs) and Confluent Cloud audit logs.

Operational benefits

  • Multi-environment support - Use separate integrations for development, staging, and production.
  • Resource sharing - One integration supports multiple connectors accessing the same cloud resources.
  • Infrastructure as code - Manage integrations programmatically using the Confluent Cloud APIs and Confluent CLI.

Next steps

Tip

Video walkthrough: This video shows how to create an AWS provider integration in Confluent Cloud that uses an AWS IAM role to let Confluent Cloud securely access your AWS resources, without having to manage static keys or passwords!

To create and manage a provider integration between Confluent Cloud resources and your CSP resources, see:

Related content