Use Self-Managed Encryption Keys in Confluent Cloud on AWS

Protect the data at rest stored in your Confluent Cloud Enterprise or Dedicated Kafka clusters on AWS using AWS Key Management Service to create and manage encryption keys,

Requirements

Self-managed encryption keys can be used with supported Kafka clusters on Confluent Cloud created using the Self-managed encryption mode. To use self-managed encryption keys on AWS for supported Kafka cluster types, follow these requirements:

Key creation and management

Required RBAC role: OrganizationAdmin or EnvironmentAdmin.

• Create a Dedicated or Enterprise Kafka cluster on AWS using the “Self-managed” encryption mode. After provisioning your Dedicated or Enterprise cluster, you cannot switch modes between Automatic (default) and Self-managed.
• Use AWS Key Management Service (KMS) to generate, use, rotate, and destroy customer master keys (CMKs).
• Only symmetric, software-protected keys are supported.
• Importing key material is not supported.
• AWS KMS External Key Store (XKS) is supported.
  • Your External Key Manager (EKM) configuration (including any proxies), AWS External Key Store (XKS), AWS Key Management Service (KMS), and Kafka cluster must all be located in the same AWS region. Cross-region configurations are not supported.
  • Each XKS instance must be dedicated exclusively to Confluent. Using XKS keys for other services impacts Confluent’s ability to guarantee SLA performance due to quota conflicts and performance factors outside of Confluent’s control.
  • Each XKS instance supports a maximum of four Confluent encryption keys. If you need more than four keys, create additional XKS instances.
• Key rotation:
  • Automatic key rotation is available using the AWS KMS console, but manual key rotation is not supported.
  • WARNING: Deleting old keys is a permanent operation that cannot be undone and results in data loss.
• Use a unique encryption key for each active cluster.
• If you delete a cluster, the encryption key is released after five days and is available for reuse during cluster creation. As a security best practice, encryption keys should not be reused for production clusters.

FIPS 140-2 certification

• Self-managed keys created after March 2023 are FIPS 140-2 Level 3 certified except for AWS regions only offering FIPS 140-2 Level 2 certification. For details, see AWS KMS FAQs.
• You can also use AWS KMS customer managed keys (CMKs) backed by AWS-managed hardware security modules (HSMs) for FIPS 140-2 Level 3 certification.
• For more information, see FIPS 140-2.

Create a Kafka cluster with self-managed encryption

Warning

If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor AWS can regain access to your data.

Confluent Cloud ConsoleConfluent CLI

To create an encrypted Confluent Cloud cluster on AWS that uses a self-managed encryption key:

1. Navigate to the Clusters page for your environment and click Create cluster if you are creating the first cluster in your environment, or click Add cluster if other clusters exist.

2. For Select cluster type under Create cluster, select a supported Kafka cluster type, and click Begin Configuration.

3. For Regions/zones under Create cluster, select AWS as the cloud service provider, select the Region and Availability, and then click Continue.

4. For Networking under Create cluster, select the networking type and click Continue.

5. For Security under Create cluster, select Self managed to manage your own encryption key using AWS Key Management Service. Additional steps appear.

   Important

   • Only symmetric keys are supported.
   • Importing key material is not supported.
   • The key must be for the same region as the cluster.

6. Select Create new if you need a new key, or select Use existing if you already have a key.

   • If you select Create new, go to the AWS KMS (requires sign-in) console and create a new key. Copy the key ARN and then paste it into the Amazon Resource Name (key ID) field.

   • If you select Use existing, select your key ARN from the drop-down list, then ensure that the policy of the key in your AWS KMS includes the block of code provided in the Confluent Cloud Console.

     The key must be for the region selected earlier. The key ARN is a unique, fully-qualified identifier of a customer-managed key (CMK) and cannot be changed. Each Confluent Cloud cluster requires a unique key.

   Relevant AWS documentation:

   • Finding the key ID and key ARN
   • Creating keys

7. From the Confluent Cloud Console, copy the provided permission statements exactly as-is (do not edit or modify) into the existing Key policy of the ARN in your AWS KMS. The code block gives your AWS key policy authorization to access your Confluent Cloud cluster and authorizes Confluent access to your KMS. For details on how to create and manage your key policy, see Manage Key Policies on Confluent Cloud.

   Important

   Make sure to append the code block (that includes two new permission statements) immediately after the existing permission statement and remember to add the expected comma separator between the existing statement and the new statements.

   After you complete the cluster creation process, this cluster-key pairing is locked. You cannot change it for the lifetime of the cluster. You can still modify permissions related to the key, and also disable or delete it, as long as your AWS permissions allow for it.

   The updated key policy should look similar to the following example:

   {
     "Sid": "Allow Confluent account(s) (152535741197) to use the key",
     "Effect": "Allow",
     "Principal": {
       "AWS": [
         "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
       ]
     },
     "Action": [
       "kms:Encrypt",
       "kms:Decrypt",
       "kms:ReEncrypt*",
       "kms:GenerateDataKey*",
       "kms:DescribeKey"
     ],
     "Resource": "*"
   },
   {
     "Sid": "Allow Confluent account(s) (152535741197) to attach persistent resources",
     "Effect": "Allow",
     "Principal": {
       "AWS": [
         "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
       ]
     },
     "Action": [
       "kms:CreateGrant",
       "kms:ListGrants",
       "kms:RevokeGrant"
     ],
     "Resource": "*"
   }
   

8. Click Continue.

9. On the Review and launch page, enter a meaningful name in Cluster name and click Launch Cluster.

   A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal; any invalid fields will be highlighted in the original form. Reenter a valid value in the highlighted field.