Use Self-managed Encryption Keys on Google Cloud on Confluent Cloud

Confluent Cloud clusters on Google Cloud are encrypted to protect your data at rest. As an option, when you create a Dedicated Kafka cluster on Google Cloud, you can use self-managed encryption keys to protect your data, allowing only the appropriate entity or user to decrypt it. Self-managed keys provide you greater privacy and data security, and allow you to maintain control over your encryption keys.

When you use self-managed encryption keys to encrypt Dedicated Kafka clusters, Confluent does not have access to your encryption keys. You are responsible for managing your keys and use Google Cloud Key Management Service (KMS) to generate, use, rotate, and destroy your encryption keys.

Requirements

Self-managed encryption keys are supported only on Dedicated Kafka clusters created using the Self-managed encryption mode. To use self-managed encryption keys on Google Cloud for Dedicated Kafka clusters, follow these requirements:

Key creation and management

Required RBAC role: OrganizationAdmin or EnvironmentAdmin.

  • Create a Dedicated Kafka cluster on Google Cloud using the “Self-managed” encryption mode. After provisioning your Dedicated cluster, you cannot switch modes between Automatic (default) and Self-managed.
  • Review the general requirements that apply across all cloud service providers.
  • Use Google Cloud Key Management Service (KMS) to generate, use, rotate, and destroy your encryption keys.
  • Automatic key rotation is available using the Google Cloud KMS console. Manual key rotation is not supported.
  • Customer-managed encryption keys (CMEKs) are supported.
  • Only “software-protected” keys are supported. Importing key material is not supported.
  • If you delete a cluster, the encryption key is released after five days and is available for reuse during cluster creation. As a security best practice, encryption keys should not be reused for production clusters.

FIPS 140-2 certification

  • Software-protected keys (FIPS 140-2 Level 1): Google Cloud KMS encryption keys use the BoringCryptoModule (BCM).
  • Hardware Security Module (HSM) keys (FIPS 140-2 Level 3): Google Cloud CloudHSM is validated to FIPS 140-2 Level 3.
  • For details, see FIPS 140-2 Validated.

Warning

If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor Google Cloud can regain access to your data.

Create a Dedicated cluster with self-managed encryption

To create an encrypted Confluent Cloud Dedicated cluster on Google Cloud that uses a self-managed encryption key:

  1. Navigate to the Clusters page for your environment and click Create cluster if you are creating the first cluster in your environment, or click Add cluster if other clusters exist.

  2. For Select cluster type under Create cluster, select Dedicated and click Begin Configuration.

  3. For Regions/zones under Create cluster, select Google Cloud as the cloud service provider, select the Region and Availability, and then click Continue.

  4. For Networking under Create cluster, select the networking type and click Continue.

  5. For Security under Create cluster, select Self-managed to manage your own encryption key using Google Cloud Key Management Service. Additional steps appear.

    Note

    • Only symmetric keys are supported.
    • Importing key material is not supported.
    • The key must be for the zone selected in 2. Regions/zone under Create cluster.

    Step 1: Go to the Google Cloud KMS console, copy the resource name of the cryptographic key, return to Confluent Cloud, and paste it in the Google Cloud resource name field.

    Step 2: In the Google Cloud KMS console, complete the following tasks:

    • Create a custom role granted the following required permissions:
      • cloudkms.cryptoKeyVersions.useToDecrypt
      • cloudkms.cryptoKeyVersions.useToEncrypt
      • cloudkms.cryptoKeys.get
    • Copy the Google Group ID from the Confluent Cloud Console, return to the Google Cloud KMS console, select the key of the Google Cloud resource name (entered in Step 1) and in the Permissions tab, click ADD MEMBER and paste the Google Group ID you copied as a new member, assign the custom role, and click SAVE.
    • For more information, see:

    After completing the two steps above in Google Cloud KMS, return to Confluent Cloud and click Continue.

  6. For 5. Review and launch under Create cluster. enter the Cluster name and click Launch Cluster.

Note

A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal dialog; any invalid fields will be highlighted in the original form. Reenter a valid value in the highlighted field.