# Use Self-Managed Encryption Keys in Confluent Cloud on Azure Required RBAC role: [OrganizationAdmin](../../access-control/rbac/predefined-rbac-roles.md#organizationadmin-role) or [EnvironmentAdmin](../../access-control/rbac/predefined-rbac-roles.md#environmentadmin-role). Protect the data at rest stored in your Enterprise or Dedicated Kafka clusters in Confluent Cloud on Azure using [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview) to create and manage encryption keys. ## Requirements Self-managed encryption keys are supported only on Kafka clusters created using the self-managed encryption mode. To use self-managed encryption keys on Azure for supported Kafka cluster types, follow these requirements: ### Key creation and management Required RBAC role: [OrganizationAdmin](../../access-control/rbac/predefined-rbac-roles.md#organizationadmin-role) or [EnvironmentAdmin](../../access-control/rbac/predefined-rbac-roles.md#environmentadmin-role). - Create an Enterprise or Dedicated Kafka cluster on Azure using the “Self-managed” encryption mode. After provisioning your cluster, you cannot switch modes between Automatic (default) and Self-managed. - Use [Azure Key Vault](https://azure.microsoft.com/services/key-vault) to generate, use, rotate, and destroy your encryption keys. - Enable [Azure RBAC](https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide) and [purge protection](https://learn.microsoft.com/en-us/azure/key-vault/general/soft-delete-overview#purge-protection). - For Dedicated Kafka clusters, if you have network restrictions, enable **Allow trusted Microsoft services to bypass this firewall?** - For Enterprise Kafka clusters, you must configure the **Firewall and virtual networks** settings (under **Networking**) in your Azure Key Vault to **Allow public access from all networks** to enable Confluent Cloud access. - Keys created and managed using external key managers (EKM) are not supported. - Key rotation: - Automatic key rotation is available using Azure Key Vault, but manual key rotation is not supported. - WARNING: It is not recommended that you delete your old keys. This is a permanent operation that cannot be undone and may result in data loss and that leads to cluster failure. The processing time for a deleted key is managed internally by Azure. - If you delete a cluster, you can delete the self-managed encryption key immediately. However, you can’t reuse a deleted key to create another cluster for five days. As a security best practice, don’t reuse encryption keys for production clusters. ### FIPS 140-2 certification - Software-protected keys (FIPS 140-2 Level 1): Available for Azure Key Vault (Premium SKU and Standard SKU). See [RSA (software-protected) keys](https://learn.microsoft.com/en-us/azure/key-vault/keys/about-keys#software-protected-keys). - HSM-protected keys (FIPS 140-2 Level 2): Available for Azure Key Vault (Premium SKU) when using [RSA-HSM (HSM-protected) keys](https://learn.microsoft.com/en-us/azure/key-vault/keys/about-keys#hsm-protected-keys). ## Create a self-managed encryption key A self-managed encryption key can be created in two ways: 1. **From the global Encryption Keys page** 2. **During cluster creation** ### Method 1: From the global Encryption Keys page To create a self-managed encryption key from the global Encryption Keys page: 1. In the Confluent Cloud Console, click the hamburger menu in the upper right corner. 2. Select **Encryption keys** from the menu. 3. Click **Add new key**. 4. **Step 1: Choose a cloud provider** - Select **Microsoft Azure**. 5. **Step 2: Enter key details** - Provide the following information: - **Key Alias** (optional): A human-friendly name to identify the key - **Azure Key Vault Resource ID**: Found in your Azure Portal under Key Vaults → select your vault → Overview → JSON View - **Azure Key Vault Key identifier without version**: The key ID from your Azure Key Vault (format: `https://.vault.azure.net/keys/`) - **Azure Key Vault Tenant ID**: Your Microsoft Entra ID tenant ID 6. Click **Register key**. The key is created and will appear in the encryption keys table. 7. **Step 3: Configure permissions and policy** - Follow the Azure-specific instructions to configure RBAC permissions. 8. Click **Finish**. The key enters an initializing state which runs asynchronously and may take up to 5 minutes. The key will show a status of “Initializing” until validation completes. Once validated, the key can be used when creating clusters. ### Method 2: Create a Kafka cluster with self-managed encryption To create an encrypted Confluent Cloud Kafka cluster on Azure that uses a self-managed encryption key: 1. Navigate to the **Clusters** page for your environment and click **Create cluster** if you are creating the first cluster in your environment, or click **Add cluster** if other clusters exist. 2. For **Select cluster type** under **Create cluster**, select a supported Kafka cluster type (Enterprise or Dedicated), and click **Begin Configuration**. 3. For **Regions/zones** under **Create cluster**, select Azure as the cloud service provider, select the **Region** and **Availability**, and then click **Continue**. 4. For **Networking** under **Create cluster**, select the [networking](../../../networking/overview.md#cloud-networking) type and click **Continue**. 5. For **Security** under **Create cluster**, select **Self-managed** to manage your own encryption key using Azure Key Vault. You can either: - **Select an existing key**: Choose from the dropdown list of previously created and validated encryption keys from the global Encryption Keys page. - **Add a new key**: Create a new encryption key during cluster creation. #### NOTE Key validation during cluster creation is asynchronous and may take a few minutes. **If adding a new key during cluster creation:** In a separate browser window, go to [Key Vaults](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.KeyVault%2Fvaults) on your Azure Portal account, select the Azure Key vault key to use, and then enter the following information in the Confluent Cloud Console: | Entry | Description and Location | |----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **Azure Key Vault Resource ID** | The resource ID of the Azure Key Vault. **Location**: Go to
[Key Vaults](https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.KeyVault%2Fvaults) →
select your vault → **Overview** → **JSON View** (to the right of **Essentials**) →
copy the value for **Resource ID**. | | **Azure Key Vault Key identifier without version** | The key ID of the Azure Key Vault. **Format**:
`https://.vault.azure.net/keys/`
(without the version suffix). **Location**: In your Key Vault → **Keys** →
select your key → copy the **Key Identifier** and remove the version part. | | **Azure Key Vault Tenant ID** | The tenant ID of the Microsoft Entra ID associated with your subscription.
**Location**: See [Find tenant ID through the Azure portal](https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant#find-tenant-id-through-the-azure-portal). | #### IMPORTANT - The encryption key and your cluster must be in the same region. - When you create keys in Azure Key Vaults, you must: - Use an RSA (software-protected) key or RSA-HSM (HSM-protected) key. See [Requirements](#byok-requirements-azure) for details. - Enable purge protection (enforces a mandatory retention period for deleted vaults and vault objects). - Enable Azure RBAC for access to the key. Confluent creates a customer key-specific Active Directory (AD) on our site. Use the CLI snippet provided to create matching role assignments in Azure for: - [Key Vault Crypto Service Encryption User](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-crypto-service-encryption-user) - [Key Vault Reader](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#key-vault-reader) - For Dedicated clusters, if you have network restrictions, enable **Allow trusted Microsoft services to bypass this firewall?** - For Enterprise clusters, you must configure the **Firewall and virtual networks** settings (under **Networking**) in your Azure Key Vault to **Allow public access from all networks** to enable Confluent Cloud access. **Configure Azure permissions**: While signed in to the correct cluster, copy the CLI snippet, then run the command in your terminal. After running the command, return to Confluent Cloud Console and click **Continue**. The Confluent Cloud cluster is created using your encryption key and is ready to use after provisioning. #### NOTE A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you get an error message. Close the modal; any invalid fields are highlighted in the original form. Reenter a valid value in the highlighted field. If the key is not valid or not authorized for Confluent, you can revisit the policy and authorization instructions from the global Encryption Keys page. Navigate to the Encryption Keys page, find your key, and click **View key details** to access the permissions and policy configuration instructions. ## Update an Azure Key Vault access policy Follow these steps to safely update your Azure Key Vault access policy without disrupting your Confluent Cloud cluster operations: Prerequisites : * Administrative access to your Azure account and Key Vault service. * Current working backup of your key policy. * Planned maintenance window for policy updates. * Understanding of the required Confluent permissions. Procedure : 1. Create a backup of your current working access policy. You can retrieve the current policy using the Azure CLI: ```shell az keyvault show \ --name \ --query properties.accessPolicies \ > key-vault-access-policy-backup-$(date +%Y-%m-%d).json ``` 2. During your planned maintenance window, apply the policy update: ```shell az keyvault set-policy \ --name \ --object-id \ --key-permissions ```
For more information, see the [az keyvault set-policy](https://docs.microsoft.com/en-us/cli/azure/keyvault?view=azure-cli-latest#az-keyvault-set-policy) documentation. 3. Verify your cluster is still operational: * Check cluster status in the Confluent Cloud Console. * Verify producers and consumers are still functioning. * Monitor for any error messages or alerts. * Check Azure Monitor logs for any access denied errors related to your Key Vault: ```shell az monitor log-analytics query \ --workspace \ --analytics-query "AzureDiagnostics | where ResourceProvider == 'MICROSOFT.KEYVAULT' and OperationName == 'KeyGet' and ResultType == 'Forbidden'" \ --out table ``` 4. Monitor your cluster for at least 30 minutes after the policy update: * Watch cluster metrics and health indicators. * Check for any encryption-related errors. * Verify that automatic operations (like scaling) continue to work.
If you encounter problems after updating the access policy: * Immediately restore the backup policy. * Monitor cluster recovery for up to 30 minutes. * Contact Confluent Support if the cluster doesn’t recover after restoring the original policy.