Use Self-managed Encryption Keys on AWS

Protect the data at rest stored in your Confluent Cloud Dedicated Kafka clusters on AWS using AWS Key Management Service to create and manage encryption keys,

Requirements

• You can only create Dedicated clusters that use self-managed encryption keys during cluster creation. You cannot switch between Automatic (default) and Self-managed encryption modes after the cluster has been provisioned.
• Use AWS Key Management Service (KMS) to generate, use, rotate, and destroy customer master keys (CMKs).
• Only symmetric keys are supported.
• Only “software-protected” keys are supported. Importing key material is not supported.
• FIPS 140-2 validation
  • When you create a self-managed Kafka cluster on AWS, the default encryption keys are FIPS 140-2 Level 3 certified for keys created after March 2023.
  • FIP 140-2 Level 3 certification (NIST) support is available for AWS KMS CMKs (customer managed keys) backed by AWS-managed HSMs (hardware security modules).
  • For some regions, only FIPS 140-2 Level 2 certification is available. For details, see AWS KMS FAQs.
  • For more information, see FIPS 140-2.
• Automatic key rotation can be enabled using AWS KMS. Manual key rotation is not supported.
• Each encryption key is associated with one active Kafka cluster. You cannot use the same encryption key for multiple clusters.
• When a cluster is deleted, the encryption key is released after five days and is available for reuse during cluster creation. As a security best practice, encryption keys should not be reused for production clusters.

Create a Dedicated Kafka cluster with self-managed encryption

Warning

If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor AWS can regain access to your data.

Confluent Cloud ConsoleConfluent CLI

To create an encrypted Confluent Cloud Dedicated cluster on AWS that uses a self-managed encryption key:

1. Navigate to the Clusters page for your environment and click Create cluster if you are creating the first cluster in your environment, or click Add cluster if other clusters exist.

2. For Select cluster type under Create cluster, select Dedicated and click Begin Configuration.

3. For Regions/zones under Create cluster, select AWS as the cloud service provider, select the Region and Availability, and then click Continue.

4. For Networking under Create cluster, select the networking type and click Continue.

5. For Security under Create cluster, select Self managed to manage your own encryption key using AWS Key Management Service. Additional steps appear.

   Important

   • Only symmetric keys are supported.
   • Importing key material is not supported.
   • The key must be for the same region as the cluster.

6. Select Create new if you need a new key, or select Use existing if you already have a key.

   • If you select Create new, go to the AWS KMS (requires sign-in) console and create a new key. Copy the key ARN and then paste it into the Amazon Resource Name (key ID) field.

   • If you select Use existing, select your key ARN from the drop-down list, then ensure that the policy of the key in your AWS KMS includes the block of code provided in the Confluent Cloud Console.

     The key must be for the region selected earlier. The key ARN is a unique, fully-qualified identifier of a customer-managed key (CMK) and cannot be changed. Each Confluent Cloud cluster requires a unique key.

   Relevant AWS documentation:

   • Finding the key ID and key ARN
   • Creating keys

7. From the Confluent Cloud Console, copy the provided permission statements exactly as-is (do not edit or modify) into the existing Key policy of the ARN in your AWS KMS. The code block gives your AWS key policy authorization to access your Confluent Cloud cluster and authorizes Confluent access to your KMS. For details, see Key policies.

   Important

   Make sure to append the code block (that includes two new permission statements) immediately after the existing permission statement and remember to add the expected comma separator between the existing statement and the new statements.

   After you complete the cluster creation process, this cluster-key pairing is locked. You cannot change it for the lifetime of the cluster. You can still modify permissions related to the key, and also disable or delete it, as long as your AWS permissions allow for it.

   The updated key policy should look similar to the following example:

   {
     "Sid": "Allow Confluent account(s) (152535741197) to use the key",
     "Effect": "Allow",
     "Principal": {
       "AWS": [
         "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
       ]
     },
     "Action": [
       "kms:Encrypt",
       "kms:Decrypt",
       "kms:ReEncrypt*",
       "kms:GenerateDataKey*",
       "kms:DescribeKey"
     ],
     "Resource": "*"
   },
   {
     "Sid": "Allow Confluent account(s) (152535741197) to attach persistent resources",
     "Effect": "Allow",
     "Principal": {
       "AWS": [
         "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
       ]
     },
     "Action": [
       "kms:CreateGrant",
       "kms:ListGrants",
       "kms:RevokeGrant"
     ],
     "Resource": "*"
   }
   

8. Click Continue.

9. On the Review and launch page, enter a meaningful name in Cluster name and click Launch Cluster.

   A successful validation results in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal; any invalid fields will be highlighted in the original form. Reenter a valid value in the highlighted field.

Create a AWS key policy

KMS keys belong to the AWS account in which they are created.

• The IAM user who creates a KMS key is not automatically considered to be the key owner and does not automatically have permission to use or manage the KMS key they created. To get permission, you must specify a key policy that includes permission statements that control access. For details, see Permissions for creating KMS keys.

By default, the key policy grants every principal in an AWS account access to the AWS KMS customer-managed key (CMK). To restrict access to your encryption key, you can create your own AWS KMS key policy that specifies a permission set.

• The permission sets must be added to the key policy when you create the KMS key to ensure that you can control access to the KMS key.
• Aliases can be used to control access to KMS keys, following the best practice of granting least privileged access. For details, see Using aliases.

For details, see Key policies in AWS KMS.

Example of an AWS KMS key policy

Your AWS KMS key policy consists of two distinct permission statements that grant access for compute and storage requirements.

The first permission statement grants Confluent AWS accounts the permissions required to describe the KMS encryption key, list metadata information about the key, perform encryption, decryption, re-encryption, and data-key generation. In the example below, the first statement specifies the required Confluent AWS accounts that are granted these permissions using the two roles for blob (tiered) and block (disk) storage. As the current Confluent Cloud infrastructure continues to grow and scale, an increasing number of accounts is required to maintain scalability.

The second permission statement grants Confluent AWS accounts the permissions required to manage (create, list, and revoke) grants for data storage. These grants are internally used to provide access for performing cryptographic operations to the specified grantee principal. In this case, the permission statement creates a grant for the role that manages block storage, which in turn creates the grant for the AWS internal roles (for example aws:ec2-infrastructure) to persistently attach AWS EBS volumes (block storage) to the AWS EC2 instance where the Kafka broker (pod) runs.

{
  "Sid": "Allow Confluent account(s) (152535741197) to use the key",
  "Effect": "Allow",
  "Principal": {
    "AWS": [
      "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
    ]
  },
  "Action": [
    "kms:Encrypt",
    "kms:Decrypt",
    "kms:ReEncrypt*",
    "kms:GenerateDataKey*",
    "kms:DescribeKey"
  ],
  "Resource": "*"
},
{
  "Sid": "Allow Confluent account(s) (152535741197) to attach persistent resources",
  "Effect": "Allow",
  "Principal": {
    "AWS": [
      "arn:aws:iam::152535741197:role/cc-kafka-d45e6381-b878-11ed-bdff-028e28f108bd"
    ]
  },
  "Action": [
    "kms:CreateGrant",
    "kms:ListGrants",
    "kms:RevokeGrant"
  ],
  "Resource": "*"
}

For "Resource": "*" above, the wildcard character (*) indicates that the KMS key policy applies to the customer-managed key (CMK).