Use Self-Managed Encryption Keys in Confluent Cloud on Google Cloud

Confluent Cloud clusters on Google Cloud are encrypted to protect your data at rest. As an option, when you create a Dedicated Kafka cluster or Enterprise Kafka cluster on Google Cloud, you can use self-managed encryption keys to protect your data, allowing only the appropriate entity or user to decrypt it. Self-managed keys provide you greater privacy and data security, and allow you to maintain control over your encryption keys.

When you use self-managed encryption keys to encrypt Kafka clusters, Confluent does not control access to your encryption keys. You are responsible for managing your keys and using Google Cloud Key Management Service (KMS) to generate, use, rotate, and destroy your encryption keys.

Requirements

Self-managed encryption keys are supported only on Kafka clusters created with the Self-managed encryption mode. To use self-managed encryption keys on Google Cloud for supported Kafka cluster types, follow these requirements:

Key creation and management

Required RBAC role: OrganizationAdmin or EnvironmentAdmin.

• Create a Dedicated or Enterprise Kafka cluster on Google Cloud using the “Self-managed” encryption mode. After provisioning your Dedicated or Enterprise cluster, you cannot switch modes between Automatic (default) and Self-managed.
• Review the general requirements that apply across all cloud service providers.
• Manage encryption keys on Google Cloud using Cloud Key Management
  • Cloud Key Management Service (Cloud KMS) for generating, using, rotating, and destroying Cloud KMS encryption keys.
  • Cloud External Key Manager (Cloud EKM) for communicating with an external key manager used to manage encryption keys.
• Key rotation:
  • Automatic key rotation is available using the Google Cloud KMS console, but manual key rotation is not supported.
  • For external key manager (EKM) keys, follow the key rotation process and policies of your external key manager and Google Cloud EKM.
  • WARNING: Deleting old keys or key versions is a permanent operation that cannot be undone and results in data loss.
• Customer-managed encryption keys (CMEKs) are supported.
• Only “software-protected” keys are supported. Importing key material is not supported.
• If you delete a cluster, the encryption key is released after five days and is available for reuse during cluster creation. As a security best practice, encryption keys should not be reused for production clusters.

FIPS 140-2 certification

• Software-protected keys (FIPS 140-2 Level 1): Google Cloud KMS encryption keys use the BoringCryptoModule (BCM).
• Hardware Security Module (HSM) keys (FIPS 140-2 Level 3): Google Cloud CloudHSM is validated to FIPS 140-2 Level 3.
• For details, see FIPS 140-2 Validated.

Warning

If you accidentally delete the master key, you are no longer able to access your encrypted data. Neither Confluent nor Google Cloud can regain access to your data.

Warning

Avoid updating key policies unless absolutely necessary. Key policy misconfigurations can cause immediate cluster unavailability and service disruption. For details on how to manage your key policy, see Manage Key Policies on Confluent Cloud.

Create a self-managed encryption key

A self-managed encryption key can be created in two ways:

1. From the global Encryption Keys page (recommended)
2. During cluster creation

Method 1: From the global Encryption Keys page

To create a self-managed encryption key from the global Encryption Keys page:

1. In the Confluent Cloud Console, click the hamburger menu in the upper right corner.
2. Select Encryption keys from the menu.
3. Click Add new key.
4. Step 1: Choose a cloud provider - Select Google Cloud.
5. Step 2: Enter key details - Provide the following information:
   • Key Alias (optional): A meaningful name to identify the key
   • Google Cloud resource name: The full resource name of your cryptographic key from the Google Cloud KMS console
6. Click Register key. The key is created and will appear in the encryption keys table.
7. Step 3: Configure permissions and policy - Follow the Google Cloud-specific instructions to configure the required permissions.
8. Click Finish. The key enters an initializing state which runs asynchronously and may take up to 5 minutes.

The key will show a status of “Initializing” until validation completes. Once validated, the key can be used when creating clusters.

Method 2: Create a Kafka cluster with self-managed encryption

Confluent Cloud ConsoleConfluent CLI

To create an encrypted Kafka cluster on Confluent Cloud on Google Cloud that uses a self-managed encryption key:

1. Navigate to the Clusters page for your environment and click Create cluster if you are creating the first cluster in your environment, or click Add cluster if other clusters exist.

2. For Select cluster type under Create cluster, select a supported Kafka cluster type (Dedicated or Enterprise) and click Begin Configuration.

3. For Regions/zones under Create cluster, select Google Cloud as the cloud service provider, select the Region and Availability, and then click Continue.

4. For Networking under Create cluster, select the networking type and click Continue.

5. For Security under Create cluster, select Self-managed to manage your own encryption key using Google Cloud Key Management Service.

   You can either:

   • Select an existing key: Choose from the dropdown list of previously created and validated encryption keys from the global Encryption Keys page.
   • Add a new key: Create a new encryption key during cluster creation by providing the Google Cloud resource name.

   Note

   • Only symmetric keys are supported.
   • Importing key material is not supported.
   • Key validation during cluster creation is asynchronous and may take a few minutes.
   • The key must be in the same region as your cluster.

   Step 1: Go to the Google Cloud KMS console, copy the resource name of the cryptographic key, return to Confluent Cloud, and paste it in the Google Cloud resource name field.

   Step 2: In the Google Cloud KMS console, set the required permissions for your key as described in Manage Key Policies on Confluent Cloud.

   • For more information, see:
     • REST Resource: group
     • Google Cloud Key Management Service documentation

   After completing the two steps above in Google Cloud KMS, return to Confluent Cloud and click Continue.

6. For 5. Review and launch under Create cluster. enter the Cluster name and click Launch Cluster.

Note

A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal dialog; any invalid fields will be highlighted in the original form. Reenter a valid value in the highlighted field.

If the key is not valid or not authorized for Confluent, you can revisit the policy and authorization instructions from the global Encryption Keys page. Navigate to the Encryption Keys page, find your key, and click View key details to access the permissions and policy configuration instructions.