Create a secret manager integration in Confluent Cloud

Secret manager integration lets fully-managed connectors in Confluent Cloud retrieve authentication credentials from AWS Secrets Manager or Azure Key Vault at runtime, so sensitive values such as database passwords and API keys never persist in Confluent Cloud.

Overview

Fully-managed connectors in Confluent Cloud can integrate with secret managers from cloud service providers (CSPs) such as Azure Key Vault or AWS Secrets Manager to retrieve sensitive information such as database credentials, access keys, and similar authentication-related secrets. This feature ensures that your secrets never persist within the Confluent boundary, and Confluent Cloud always fetches them at runtime as needed.

Secret manager integration uses Confluent Provider Integration to authenticate with your CSP and securely access secrets stored in services like Azure Key Vault or AWS Secrets Manager. For more information about provider integration in Confluent Cloud, see Manage Provider Integration for Fully-Managed Connectors in Confluent Cloud.

Confluent Cloud supports:

Key terms

  • Secret manager: A secret manager is a centralized system or service designed to securely store, manage, and retrieve sensitive data, often called secrets.

  • Secret manager integration: A Confluent Cloud feature that lets Confluent Cloud access secrets stored in a secret manager like AWS Secrets Manager or Azure Key Vault.

  • Provider integration: A Confluent Cloud feature that builds a trusted handshake between your AWS identity or Azure principal and your Confluent identity.

    • With Azure, it is the service principal through workload identity federation that lets Confluent Cloud call Azure APIs.

    • With AWS, it is an IAM role that Confluent Cloud assumes to call AWS APIs.

  • Secret identifier: The string that you enter into a connector configuration field so that Confluent Cloud can resolve the actual credential value at runtime from the secret manager.

Supported connectors

The following connectors support secret manager integration:

Limitations

  • Cross-cloud limitations: The secret manager must be in the same cloud provider as your Confluent Cloud cluster. For example, using Azure Key Vault with a Connect cluster on AWS is not supported.

  • Supported cluster type: Currently, you can integrate a secret manager only with Dedicated clusters.

    Limitations on integrating a CSP with Confluent Cloud are applicable to secret manager integration as well. For more information, see Provider integration limitations.

Secret rotation and lifecycle

Secret manager integration retrieves secrets when:

  • A connector starts or restarts.

  • A connector enters into a failed state and it tries to recover.

    While it tries recovery, Confluent Cloud invalidates cached credentials and the connector tries to fetch the latest secrets from the secret manager.

To apply rotated credentials from your secret manager immediately:

  1. Rotate the secret in your secret manager using your CSP’s secret management tools.

  2. Restart the connector in Confluent Cloud to fetch the updated secret using one of these methods:

    • On the Confluent Cloud Console, navigate to the connector settings and click Restart.

    • Use the Restart a Connector Confluent REST API.

Related content