Create a secret manager integration in Confluent Cloud

This topic provides an overview of secret manager integration with Confluent Cloud to retrieve authentication-related secrets for fully-managed connectors.

Overview

Fully-managed connectors in Confluent Cloud can integrate with secret managers from cloud service providers (CSPs) such as Azure Key Vault or AWS Secrets Manager to retrieve sensitive information such as database credentials, access keys, and similar authentication-related secrets. This feature ensures that your secrets never persist within Confluent’s boundary, and Confluent Cloud always fetches them at runtime as needed.

Secret manager integration uses Confluent Provider Integration to authenticate with your CSP and securely access secrets stored in services like Azure Key Vault or AWS Secrets Manager. For more information about provider integration in Confluent Cloud, see Manage Provider Integration for Fully-Managed Connectors in Confluent Cloud.

Confluent Cloud supports:

Key terms

  • Secret manager: A secret manager is a centralized system or service designed to securely store, manage, and retrieve sensitive data, often called secrets.

  • Secret manager integration: A Confluent Cloud feature that lets it access secrets stored in a secret manager like AWS Secrets Manager or Azure Key Vault.

  • Provider integration: A Confluent Cloud feature that builds a trusted handshake between a customer’s AWS identity or Azure principal, and their Confluent identity.

    • With Azure, it is the service principal through workload identity federation that lets Confluent Cloud call Azure APIs.

    • With AWS, it is an IAM role that Confluent Cloud assumes to call AWS APIs.

  • Secret identifier: The string that you enter into a connector configuration field so that Confluent Cloud can resolve the actual credential value at runtime from the secret manager.

Supported connectors

The following connectors support secret manager integration:

Limitations

  • Cross-cloud limitations: The secret manager must be in the same cloud provider as your Confluent Cloud cluster. For example, using Azure Key Vault with a Connect cluster on AWS is not supported.

  • Supported cluster type: Currently, you can integrate a secret manager within Dedicated clusters only.

    Limitations on integrating a CSP with Confluent Cloud are applicable to secret manager integration as well. For more information, see Provider integration limitations.

Secret rotation and lifecycle

Secret manager integration retrieves secrets when:

  • A connector starts or restarts.

  • A connector enters into a failed state and it tries to recover.

    While it tries recovery, Confluent Cloud invalidates cached credentials and the connector tries to fetch latest secrets from the secret manager.

To apply rotated credentials from your secret manager immediately:

  1. Rotate the secret in your secret manager using your CSP’s secret management tools.

  2. Restart the connector in Confluent Cloud to fetch the updated secret using one of these methods:

    • On the Confluent Cloud Console, navigate to the connector settings and click Restart.

    • Use the Restart a Connector Confluent REST API.

Related content