Manage Provider Integration for Fully-Managed Connectors in Confluent Cloud

Fully-managed connectors in Confluent Cloud can use Confluent provider integration to authenticate with your cloud service provider (CSP) resources using Identity and Access Management (IAM) roles. With this feature, connectors leverage temporary security credentials from an IAM role, allowing secure, role-based access to CSP resources for data ingestion and transfer. This eliminates the need for long-term access keys and ensures connector access is tightly scoped and governed by your CSP’s IAM policies.

When configuring a connector, you specify an IAM role in your CSP account (for example, an AWS IAM role). Confluent provider integration then assumes this role using secure, temporary credentials, enabling the connector to access only the specific resources (such as Amazon S3 buckets or DynamoDB tables) permitted by the role’s policies. Access to Confluent Cloud resources (like connectors and topics) remains managed by Confluent Cloud access controls, such as access control lists (ACLs) or role-based access control (RBAC).

This guide describes how to configure a provider integration for fully-managed source and sink connectors. For more information about provider integration in Confluent Cloud, see Integrate with Cloud Service Providers in Confluent Cloud. The guide assumes that you already have a working Confluent Cloud environment and a CSP account.

Limitations

By default, the resource quota for number of integrations per Organization is set to 100. Contact Confluent Support if you want to increase the quota limit.

Manage an AWS Provider Integration

Create an AWS Provider Integration

You must configure a provider integration in Confluent Cloud before you modify an existing connector or create a new one with provider integration enabled.

To configure an AWS Provider Integration, you begin by selecting a role type (new or existing) and creating an IAM permission policy in AWS. Next, you create a new role in AWS and map it within Confluent Cloud. Finally, you update the trust policy in AWS with the Confluent IAM role configurations to complete the setup.

For detailed setup, see Steps to create an AWS provider integration.

Create a connector with an AWS IAM role

Now that you’ve successfully configured a Confluent IAM role to assume the role in your AWS account, you can now create a connector, for example, a S3 sink connector or a DynamoDB source connector, with the IAM role.

Make a note of the following:

  • You must have a ProviderIntegrationResourceOwner or ProviderIntegrationAssigner role to create a connector using the provider integration resource.
  • Use Read a provider integration API to fetch provider.integration.id in your Confluent environment.

Authenticate using AWS IAM role

At the Add Source/Sink connector screen, complete the following in the Authentication step:

  1. Under AWS credentials, select IAM Roles in the Authentication method field.
  2. Select your provider integration name in the Provider integration name field.

Validate AWS IAM role authorization

After the connector is running, verify the messages and the resources for successful IAM role authorization through Confluent Provider Integration.

  1. Sign in to your Confluent Cloud account.
  2. In the left navigation menu, click Connectors.
  3. Click the connector you just created above.
  4. View the messages produced to a topic.

Delete resources

When you are finished with the setup, delete the resources you created to avoid unexpected charges to your account.

  1. Sign in to your Confluent Cloud account.
  2. Click the connector you created and choose the Settings tab.
  3. Click Delete connector, enter the connector name, and click Confirm.

Related content