Creating Encrypted Confluent Cloud Clusters Using Your Own Key¶
This feature is available as a preview feature. This Confluent Cloud preview feature is being introduced to gain early feedback from developers. You can use this feature for evaluation and non-production testing purposes or to provide feedback to Confluent.
You can encrypt your at-rest cluster data to ensure only the appropriate entity or user can decrypt it. This provides a greater degree of privacy and data integrity, which is frequently required by government, health, finance, and many other industries.
Confluent Cloud data resides in clusters that you can deploy across multiple components, and each must support privacy and data confidentiality. When creating a Confluent Cloud cluster using the dedicated cluster type, you have the option to use an Amazon Web Services (AWS)-generated encryption key to encrypt your cluster data or Amazon EBS volumes and data stored in S3. This is also known as bring your own key (BYOK) encryption.
You can only create encrypted clusters with your own key when using Amazon Web Services as the provider and the Dedicated cluster type.
Create an Encrypted Confluent Cloud Cluster Using Your Own Key¶
To create an encrypted Confluent Cloud cluster using your own key:
If you accidentally delete the master key, you will no longer be able to access your encrypted data. Neither Confluent nor AWS will be able to regain access to your data.
Navigate to the clusters page for your environment and click Create cluster if this is the first cluster in your environment, or click + Add cluster if there are already other clusters.
Specify a cluster name and select Amazon Web Services as the provider. Specify region, and select the Dedicated cluster type. Dedicated clusters are designed for critical production workloads with high traffic or private networking requirements.
Specify availability, cluster size, and networking.
- The cluster size is determined by the number of Confluent Unit for Kafka (CKU). CKUs determine the capacity and limits of your cluster. For details on CKUs, see Dedicated cluster CKUs and limits. You can add CKUs to your cluster after provisioning to increase its capacity.
- If you are using VPC Peering, see Networking in Confluent Cloud.
Select how you want to manage the encryption key:
If you select Automatic (default), then the default encryption key is created, managed, and used on your behalf by AWS. After making this selection and completing the cluster creation process, this encryption option is locked and cannot be changed for the lifetime of the cluster. Proceed to the next step.
If you select Self Managed key, then you create and manage your own keys in your AWS account. This option may be preferable for users who want to use their own key to encrypt data at rest, or who need the option to disable Confluent’s access to data at any time. Click Add an Amazon resource key ID to continue.
Enter the Amazon resource name ID (ARN) for your encryption key. To locate your ARN, log in to the AWS KMS Console and create or locate the ARN (“key” and “ARN” are one and the same for the purposes of this document). After you specify the ARN and complete the cluster creation process, this cluster-key pairing is locked. You cannot change it for the lifetime of the cluster. You can still modify permissions related to the key, and also disable or delete it, as long as your AWS permissions allow for it.
If at any point you click Cancel, close the window, or click outside the window, the value entered into the ARN field is cleared out. To lock in the ARN that you have here, you must successfully validate the key and authorize Confluent permissions.
Click Continue to authorization.
You must authorize your AWS key policy to include authorization access for Confluent.
Copy and append the permissions provided by Confluent Cloud into your AWS key policy. This authorizes Confluent access to your KMS. For details see the AWS KMS documentation. After copying the permissions into and saving your AWS key policy, click I have authorized Confluent in my KMS.
Note that clicking Link to AWS KMS in the AWS Authorization window takes you to directly to the AWS KMS key context where you can edit your key policy.
Click Continue. Confirm your cluster subscription details, payment information, and click Launch, which will attempt to validate your cluster configuration.
A successful validation will result in the provisioning of your cluster. If the cluster configuration is invalid because the encryption key is not valid or not authorized for Confluent, then you will get an error message indicating so. Close the modal; any invalid fields will be highlighted in the original form. Re-enter a valid value in the highlighted field.
After successfully provisioning your cluster, you cannot change your BYOK configuration. The only option is to delete the cluster and create a new one with the desired configuration changes.
Using the Confluent Cloud CLI to Create Encrypted Clusters Using Your Own Key¶
The following Confluent Cloud CLI example shows how to create an encrypted Confluent Cloud cluster using your customer-managed key:
ccloud kafka cluster create sales092020 --cloud "aws" --type "dedicated" --encryption-key "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"
Any time you specify the
--encryption-key option your will be prompted to update
your KMS policy. For details, see ccloud kafka cluster create.
Viewing Dedicated Cluster Security Settings¶
You can view the Security settings for all dedicated clusters provisioned on AWS. In other words, if you used Automatic, Self Managed, or have an existing dedicated cluster on AWS that you created prior to using BYOK, you can view the cluster security settings. The data in the cluster security settings is informational only, and serves to identify the keys in use.
To view your dedicated AWS cluster security settings:
- Select your Confluent Cloud cluster.
- Click the Cluster settings tab and then Security.
Note that anyone authorized to view your dedicated AWS clusters can view this data.