Creating Encrypted Confluent Cloud Clusters Using Your Own Key

You can encrypt your at-rest cluster data to ensure only the appropriate entity or user can decrypt it. This provides a greater degree of privacy and data integrity, which is frequently required by government, health, finance, and many other industries.

Confluent Cloud data resides in clusters that you can deploy across multiple components, and each must support privacy and data confidentiality. When creating a Confluent Cloud cluster using the dedicated cluster type, you have the option to use your own Amazon Web Services (AWS)-generated encryption key to encrypt your cluster data or Amazon EBS volumes and data stored in S3. This is also known as BYOK (bring-your-own-key) encryption.

Important

You can only create encrypted clusters with your own key when using Amazon Web Services as the provider and the Dedicated cluster type. Only symmetric keys are supported. For details, refer to the AWS KMS documentation.

Create an Encrypted Confluent Cloud Cluster Using Your Own Key

To create an encrypted Confluent Cloud cluster using your own key:

Warning

If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor AWS will be able to regain access to your data.

  1. Navigate to the clusters page for your environment and click Create cluster if this is the first cluster in your environment, or click + Add cluster if there are already other clusters.

  2. Select the Dedicated cluster type and click Begin Configuration.

  3. Select Amazon Web Services as the provider, and then specify the Region and Availability. Click Continue.

  4. Specify the networking type and click Continue.

    Note

    If you are using VPC Peering, see Networking in Confluent Cloud.

  5. Select how you want to manage the encryption key:

    • If you select Automatic (default), then the default encryption key is created, managed, and used on your behalf by AWS. After making this selection and completing the cluster creation process, this encryption option is locked and cannot be changed for the lifetime of the cluster. Automatic key rotation (using AWS KMS) is supported by Confluent. Proceed to the next step.

    • If you select Self-managed, then you create and manage your own keys in your AWS account. This option may be preferable for users who want to use their own key to encrypt data at rest, or who need the option to disable Confluent’s access to data at any time.

      Step 1: Enter the Amazon resource name ID (ARN) for your encryption key. To locate your ARN, log in to the AWS KMS Console and create or locate the ARN. (“key” and “ARN” are one and the same for the purposes of this document).

      Step 2: You must authorize your AWS key policy to include authorization access for Confluent. Copy and append the permissions provided by Confluent Cloud into your AWS KMS key policy, then click Continue. This authorizes Confluent access to your KMS. For details see the AWS KMS documentation.

      Note

      After you complete the cluster creation process, this cluster-key pairing is locked. You cannot change it for the lifetime of the cluster. You can still modify permissions related to the key, and also disable or delete it, as long as your AWS permissions allow for it.

  6. Enter the cluster name. The Configuration & cost tab summarizes your cluster configuration details. The Usage limits tab summarizes your cluster limits, and the Uptime SLA tab displays the uptime policy for your service-level license agreement. Click Launch Cluster, which will attempt to validate your cluster configuration.

    A successful validation will result in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal; any invalid fields will be highlighted in the original form. Re-enter a valid value in the highlighted field.

    Important

    After successfully provisioning your cluster, you cannot change your BYOK configuration. The only option is to delete the cluster and create a new one with the desired configuration changes.

Using the Confluent Cloud CLI to Create Encrypted Clusters Using Your Own Key

As mentioned above, you must use AWS as the provider, and specify the dedicated cluster type.

The following Confluent Cloud CLI example shows how to create an encrypted Confluent Cloud cluster using your customer-managed key. Any content in brackets (<>) must be customized for your environment.

ccloud kafka cluster create sales092020 --cloud "aws" --region "<KMS-region>" --type "dedicated" --cku <CKU-value> --encryption-key "<AWS-ARN-ID>"

Copy and append these permissions to the existing "Statements" array field in the key policy of your ARN to authorize access for Confluent:

{
    "Sid" : "Allow Confluent account (123456789101) to use the key",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : ["arn:aws:iam::123456789101:root"]
    },
    "Action" : [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ],
    "Resource" : "*"
  }, {
    "Sid" : "Allow Confluent account (123456789101) to attach persistent resources",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : ["arn:aws:iam::123456789101:root"]
    },
    "Action" : [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ],
    "Resource" : "*"
}

Please confirm you've authorized the key for this account: 123456789101 (y/n):y

Any time you specify the --encryption-key option you will be prompted to update your AWS KMS policy. For details, see ccloud kafka cluster create, and your AWS KMS documentation.

Viewing Dedicated Cluster Security Settings

You can view the Security settings for all dedicated clusters provisioned on AWS. In other words, if you used Automatic, Self-managed, or have an existing dedicated cluster on AWS that you created prior to using BYOK, you can view the cluster security settings. The data in the cluster security settings is informational only, and serves to identify the keys in use.

To view your dedicated AWS cluster security settings:

  1. Select your Confluent Cloud cluster.
  2. Click the Cluster settings tab and then Security.

Note that anyone authorized to view your dedicated AWS clusters can view this data.