Manage Encryption Keys for CSFLE in Confluent Cloud¶
Key encryption keys (KEKs) and data encryption keys (DEKs) are used together for client-side field level encryption (CSFLE) in Confluent Cloud. DEKs are used to encrypt and decrypt the sensitive data in data fields in your Confluent Cloud resources, while KEKs are used to securely manage the DEKs. The KEKs are managed and stored in an external key management service (KMS), such as AWS KMS, Azure Key Vault, or Google Cloud KMS.
Key encryption keys (KEKs)¶
Key encryption keys (KEKs) are master keys used to securely manage data encryption keys (DEKs), decrypting them only when needed by the client. Only users with access to the KEK can decrypt the DEK and access your sensitive data.
The KEK is managed and stored at the Key Management Service (KMS) provider, such as AWS KMS, Azure Key Vault, or Google Cloud KMS. However, the DEK Registry in Confluent Cloud needs to be aware of the KEK in order to use it for encrypting and decrypting DEKs. This means that while the KEK itself is not stored in the DEK Registry, metadata about the KEK (such as its name and key identifier) is registered in the DEK Registry.
KEK parameters¶
A key encryption key (KEK) registered to the DEK Registry has the following parameters:
| Parameter | Description |
|---|---|
name |
A meaningful name for the KEK. The name is used when referring to the KEK elsewhere, such as in RBAC. |
kmsType |
The type of KMS, typically one of “aws-kms”, “azure-kms”, and “gcp-kms”. |
kmsKeyId |
The key identifier for the KEK. When using AWS KMS, the For AWS multi-region keys (MRKs), if the ARN is copied from one region to another, the ARN is dynamically translated to be an MRK for the new region. The region specified in the ARN is dynamically replaced with the actual region in which the AWS MRK is used. This facilitates use cases like Schema Linking across different regions. Example of an AWS multi-region key (MRK): If the initial region is arn:aws:kms:us-east-1:XXXXXX:key/mrk-XXXXXX
When the AWS MRK is used in the arn:aws:kms:us-west-2:XXXXXX:key/mrk-XXXXXX
The only difference in the ARN is the region, where To disable this dynamic behavior, set the property Ensure the correct policy is added for each MRK in every region. Without the correct policy, the Confluent Cloud DEK Registry cannot access the appropriate key in each region. For details, see Multi-Region keys in AWS KMS. |
kmsProps |
Additional key-value properties used to access the KMS. |
doc |
(Optional) A meaningful description for the KEK. |
shared |
(Optional) Whether the DEK Registry has shared access to the KMS. |
ts |
(Optional) The timestamp indicating when the KEK was registered or updated. |
Create a KEK¶
Warning
Ensure that the KEK is created in a secure environment and that only authorized users have access to it. Improper handling of KEKs can lead to unauthorized access to DEKs and sensitive data.
To create a KEK, use the key management service (KMS) you are using for CSFLE.
For details on how to create a KEK, see the documentation for your key management service (KMS).
Register a KEK¶
After you create a KEK, you can register the KEK before use by using the register-deks
command to register both the DEKs and KEKs. For details, see the Register a DEK
section and review the example that specifies the KEK parameters.
If you do not want to register the KEK before use, you can register it on demand by the client, assuming that the client has the appropriate permissions with the DEK Registry.
Delete a KEK¶
Warning
Deleting a KEK without properly handling associated DEKs can result in data loss. Ensure that all DEKs are properly managed before deleting the KEK.
To delete a KEK, follow these steps:
- Identify the KEK. Determine the KEK you want to delete by its name or key identifier.
- Delete Associated DEKs. Ensure that all Data Encryption Keys (DEKs) associated
with the KEK are deleted. This might involve:
- Decrypting any data that was encrypted with the DEKs.
- Re-encrypting the data with DEKs associated with a different KEK.
- Delete the KEK. Use the KMS Interface to access the key management service (KMS) management interface or API to delete the KEK.
For details on how to delete a KEK, see the documentation for your key management service (KMS).
Update a KEK¶
You can update a KEK by publishing a new version of the schema (defining how data
is encrypted and decrypted) with the same value for encrypt.kek.name.
Warning
Updating a KEK might require decrypting and re-encrypting DEKs. Ensure that you have appropriate permissions and that the process is performed in a secure environment to prevent unauthorized access to sensitive keys.
To update a KEK, follow these steps:
- Identify the KEK: Determine the KEK you want to update by its name or key identifier.
- Update KEK Parameters: Modify the parameters of the KEK as needed. This might involve changing properties such as the KEK name, KMS type, or KMS key identifier.
- Update the KEK: Use the KMS Interface to access the key management service (KMS) management interface or API to apply the updates to the KEK.
For details on how to update a KEK, see the documentation for your key management service (KMS).
Rotate a KEK¶
You can manually rotate the KEK on your key management service (KMS). When you rotate a KEK, you need to re-encrypt the existing DEKs with the new KEK.
Warning
Rotating a KEK might require decrypting and re-encrypting DEKs. Ensure that you have appropriate permissions and that the process is performed in a secure environment to prevent unauthorized access to sensitive keys.
To manually rotate the DEK, publish a new version of the schema with a different
value for encrypt.kek.name.
To rotate a KEK, you need to:
- Decrypt the DEKs that are encrypted with the existing KEK.
- Encrypt the DEKs with the new KEK.
- Update the DEK Registry with the DEKs encrypted with the new KEK.
This process ensures that the DEKs remain secure and can still be used to encrypt and decrypt your data.
For details on how to rotate a KEK, see the documentation for your key management service (KMS).
For best practices on rotating KEKs, see NIST guidelines for key rotation.
Data encryption keys (DEKs)¶
Data encryption keys (DEKs) are symmetric keys that are used to encrypt and decrypt the sensitive data in your Confluent Cloud resources. Each DEK is encrypted with a key encryption key (KEK) that is only accessible to authorized users. The encrypted DEK and encrypted data are stored together. Only users with access to the KEK can decrypt the DEK and access the sensitive data.
DEK parameters¶
During encryption, the client asks the DEK Registry for an existing DEK for a specified KEK name and subject (when using automtic DEK rotation, version is required). If a DEK does not exist, then depending on whether the DEK Registry has access to the KMS, either the DEK Registry or the client will generate and encrypt the DEK, and then register it with the DEK Registry. If the DEK Registry generates the DEK, the decrypted DEK is sent to the client.
A DEK registered to the DEK Registry has the following parameters:
| Parameter | Description |
|---|---|
kekName |
The name of the KEK used to encrypt this DEK. |
subject |
The subject for the DEK. |
| version | The version of the DEK. |
algorithm |
The encryption algorithm being used. Valid values include: AES128_GCM,
AES256_GCM (default), or AES256_SIV. |
encryptedKeyMaterial |
The encrypted key material for the DEK. |
ts |
The timestamp indicating when the DEK was registered. |
Create a DEK¶
Warning
Ensure that the DEK is created in a secure environment and that only authorized users have access to it. Improper handling of DEKs can lead to unauthorized access to sensitive data.
To create a DEK, you need to specify the KEK name and subject. The subject is a unique identifier for the DEK, such as a namespace or collection.
For details on how to create a DEK, see the documentation for your key management service (KMS).
Register a DEK¶
To register a DEK, use the register-deks command, replacing <url> with the
URL of the Schema Registry, <subject> with the subject of the schema, and <version>
with the version of the schema.
./bin/register-deks [-hV] [-X=<prop=val>]... <url> <subject> [<version>]
The register-deks command has the following parameters:
| Parameter | Description |
|---|---|
url |
The URL of the Schema Registry. |
subject |
The subject of the schema. |
version |
(Optional) The version of the schema. Default is latest. |
-X, --property=<prop=val> |
(Optional) Set the configuration properties to pass to the serdes. |
Example¶
To register a DEK for the schema in subject mysubject for version 1,
the following command registers the DEK and its associated KEK:
./bin/register-deks http://localhost:8081 mysubject 1
This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields.
To register a DEK for the schema in subject mysubject for version 1 using
a KEK named mykek with key identifier arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678,
use the following command:
./bin/register-deks \
-X encrypt.kek.name=mykek \
-X encrypt.kek.key.id=arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678 \
http://localhost:8081 mysubject 1
This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields.
Delete a DEK¶
Warning
Deleting a DEK without properly handling associated data can result in data loss. Ensure that all data associated with the DEK is properly managed before deleting the DEK.
To delete a DEK, follow these steps:
- Identify the DEK you want to delete by its name or key identifier.
- Ensure that there are no active dependencies on the DEK. This might involve:
- Decrypting any data that was encrypted with the DEK.
- Re-encrypting the data with a different DEK.
- Delete the DEK using the DEK management interface or API for the key management service (KMS) that you are using.
For detailed instructions on deleting DEKs, see the documentation for your key management service (KMS).
Update a DEK¶
Warning
Updating a DEK might require decrypting and re-encrypting data. Ensure that you have appropriate permissions and that the process is performed in a secure environment to prevent unauthorized access to sensitive keys.
You can update a DEK by publishing a new version of the schema (defining how data
is encrypted and decrypted) with the same value for encrypt.kms.key.id and
encrypt.kms.type, but use a different value for encrypt.kek.name.
For details on how to update a DEK, see the documentation for your key management service (KMS).
Rotate a DEK¶
Warning
Rotating a DEK might require decrypting and re-encrypting data. Ensure that you have appropriate permissions and that the process is performed in a secure environment to prevent unauthorized access to sensitive keys.
You can manually rotate a DEK by publishing a new version of the schema (defining
how data is encrypted and decrypted) with the same value for encrypt.kms.key.id
and encrypt.kms.type, but use a different value for encrypt.kek.name.
For details on how to rotate a DEK, see the documentation for your key management service (KMS).
For best practices on rotating DEKs, see NIST guidelines for key rotation.
NIST guidelines for key rotation¶
Periodic rotation of the encryption keys is recommended, even in the absence of compromise. Following the guidelines of NIST publication 800-38D, “Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC”, AES-GCM key rotation should occur before approximately 2^32 encryptions have been performed by a key version. For example, if the estimated encryption rate of a key is 40 million operations per day, then rotating a key every three months is sufficient.