# Manage Encryption Keys for CSFLE in Confluent Cloud Client-side field level encryption (CSFLE) uses two types of keys: * Key Encryption Keys (KEK), which are master keys stored in your KMS * Data Encryption Keys (DEK), which encrypt your fields KEKs and DEKs are used together for encryption in Confluent Cloud. Manage and store your KEKs in an external key management service (KMS), such as AWS KMS, Azure Key Vault, or Google Cloud KMS. DEKs are used to encrypt and decrypt the sensitive data in data fields in your Confluent Cloud resources, while KEKs are used to securely manage the DEKs. Confluent Cloud’s DEK Registry automatically generates and manages DEKs for encrypting data. These keys are stored in a hierarchy. The KEK is a master key stored in your KMS. You control the KEK; Confluent Cloud never stores it. The DEK Registry generates your DEK, encrypted by KEK, and used to encrypt your actual data fields. ## Key encryption keys Key encryption keys (KEKs) are master keys used to securely manage data encryption keys (DEKs), decrypting them only when needed by the client. Only users with access to the KEK can decrypt the DEK and access your sensitive data. The KEK is managed and stored at the KMS provider, such as AWS KMS, Azure Key Vault, or Google Cloud KMS. However, the DEK Registry in Confluent Cloud needs to be aware of the KEK in order to use it for encrypting and decrypting DEKs. This means that while the KEK itself is not stored in the DEK Registry, metadata about the KEK (such as its name and key identifier) is registered in the DEK Registry. ### KEK parameters A key encryption key (KEK) registered to the DEK Registry has the following parameters: | Parameter | Description | |-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `name` | A meaningful name for the KEK. The name is used when referring to the
KEK elsewhere, such as in RBAC. | | `kmsType` | The type of KMS, typically one of “aws-kms”, “azure-kms”, and “gcp-kms”. | | `kmsKeyId` | The key identifier for the KEK. When using AWS KMS, the `kmsKeyId`
is the ARN.

For high availability, you can register multiple KEKs that reference the
same encryption key material. See [KEK parameters](#kek-parameters-csfle) for details.

For AWS multi-region keys (MRKs), if the ARN is copied from one region to
another, the ARN is dynamically translated to be an MRK for the new region.
The region specified in the ARN is dynamically replaced with the actual
region in which the AWS MRK is used. This facilitates use cases like
Schema Linking across different regions.

**Example of an AWS multi-region key (MRK)**:

If the initial region is `us-east-1` and the following AWS MRK is used:

```text
arn:aws:kms:us-east-1:XXXXXX:key/mrk-XXXXXX
```

When the AWS MRK is used in the `us-west-2` region, the ARN is dynamically
translated to:

```text
arn:aws:kms:us-west-2:XXXXXX:key/mrk-XXXXXX
```

The only difference in the ARN is the region, where `us-east-1` is replaced
with `us-west-2`.

To disable this dynamic behavior, set the property `aws.disable.dynamic.mrk=true`
in the `kmsProps` of the KEK in the DEK Registry. Use this when strict
regional isolation is required or when you do not want aliases translated
across regions.

Ensure the correct policy is added for each MRK in every region. Without
the correct policy, the Confluent Cloud DEK Registry cannot access the appropriate
key in each region.

**Schema Linking across regions**

Typically, when using an MRK with [Schema Linking](../../../sr/schema-linking.md#schema-linking),
the MRK in the source region must have a replica in the destination region.
If the source MRK is referenced by an alias, then another alias must be created
in the destination region that refers to the destination MRK. This destination
alias must exactly match the source alias, with only the region portion changed
(for example, `alias/mrk-example-use-west-2` in the source region becomes
`alias/mrk-example-use-east-1` in the destination region).

This matching alias naming pattern allows applications to reference the region-specific
alias rather than the MRK ARN directly, enabling seamless region switching. Applications
can use the same alias reference code regardless of region, which supports disaster
recovery scenarios while avoiding cross-region API calls that would add latency and
potential points of failure.

For details, see [Multi-Region keys in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html). | | `kmsProps` | Additional key-value properties used to access the KMS.

A KEK can be configured with alternate KMS key IDs. These KMS key IDs are
used if the primary KMS key ID is not available as specified in `kmsKeyId`.
Each alternate KMS key ID must have the identical key material as the primary
KMS key ID. To specify the alternate KMS key IDs, add a property to `kmsProps`
with key `encrypt.alternate.kms.key.ids` and with the value being a
comma-separated list of key identifiers for the alternate keys. When using
AWS KMS, each key identifier is an ARN. | | `doc` | (Optional) A meaningful description for the KEK. | | `shared` | (Optional) Whether the DEK Registry has shared access to the KMS. | | `ts` | (Optional) The timestamp indicating when the KEK was registered or updated. | ### Create a KEK #### WARNING Ensure that the KEK is created in a secure environment and that only authorized users have access to it. Improper handling of KEKs can lead to unauthorized access to DEKs and sensitive data. To create a KEK, use the KMS you are using for CSFLE. For details on how to create a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Register a KEK After you create a KEK, you can register the KEK before use by using the `register-deks` command to register both the DEKs and KEKs. For details, see the [Register a DEK](#register-dek-csfle) section and review the example that specifies the KEK parameters. If you do not want to register the KEK before use, you can register it on demand by the client, assuming that the client has the appropriate permissions with the DEK Registry. ### Delete a KEK #### WARNING Deleting a KEK without properly handling associated DEKs can result in data loss. Ensure that all DEKs are properly managed before deleting the KEK. To delete a KEK, follow these steps: 1. Identify the KEK. Determine the KEK you want to delete by its name or key identifier. 2. Delete Associated DEKs. Ensure that all Data Encryption Keys (DEKs) associated with the KEK are deleted. This might involve: - Decrypting any data that was encrypted with the DEKs. - Re-encrypting the data with DEKs associated with a different KEK. 3. Delete the KEK. Use the KMS Interface to access the KMS management interface or API to delete the KEK. For details on how to delete a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Update a KEK You can update a KEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kek.name`. To update a KEK, follow these steps: 1. **Identify the KEK**: Determine the KEK you want to update by its name or key identifier. 2. **Update KEK Parameters**: Modify the parameters of the KEK as needed. This might involve changing properties such as the KEK name, KMS type, or KMS key identifier. 3. **Update the KEK**: Use the KMS Interface to access the key management service (KMS) management interface or API to apply the updates to the KEK. For details on how to update a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Rotate a KEK You can manually rotate the KEK on your KMS. To manually rotate the DEK, publish a new version of the schema with a different value for `encrypt.kek.name`. For details on how to rotate a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). For best practices on rotating KEKs, see [NIST guidelines for key rotation](#nists-guidelines-key-rotation-csfle). After rotating a KEK, rewrap the existing DEKs with the new KEK. For details, see [Rewrap DEKs](#rewrap-deks-csfle). ## Data encryption keys Data encryption keys (DEKs) are symmetric keys that are used to encrypt and decrypt the sensitive data in your Confluent Cloud resources. Each DEK is encrypted with a key encryption key (KEK) that is only accessible to authorized users. The encrypted DEK and encrypted data are stored together. Only users with access to the KEK can decrypt the DEK and access the sensitive data. ### DEK parameters During encryption, the client asks the DEK Registry for an existing DEK for a specified KEK name and subject (when using automatic DEK rotation, version is required). If a DEK does not exist, then depending on whether the DEK Registry has access to the KMS, either the DEK Registry or the client will generate and encrypt the DEK, and then register it with the DEK Registry. If the DEK Registry generates the DEK, the decrypted DEK is sent to the client. A DEK registered to the DEK Registry has the following parameters: | Parameter | Description | |------------------------|-----------------------------------------------------------------------------------------------------------------------| | `kekName` | The name of the KEK used to encrypt this DEK. | | `subject` | The subject for the DEK. | | version | The version of the DEK. | | `algorithm` | The encryption algorithm being used. Valid values include: `AES128_GCM`,
`AES256_GCM` (default), or `AES256_SIV`. | | `encryptedKeyMaterial` | The encrypted key material for the DEK. | | `ts` | The timestamp indicating when the DEK was registered. | ### Create a DEK #### WARNING Ensure that the DEK is created in a secure environment and that only authorized users have access to it. Improper handling of DEKs can lead to unauthorized access to sensitive data. To create a DEK, you need to specify the KEK name and subject. The subject is a unique identifier for the DEK, such as a namespace or collection. For details on how to create a DEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Register a DEK To register a DEK, use the `register-deks` command, replacing `` with the URL of the Schema Registry, `` with the subject of the schema, and `` with the version of the schema. ```shell ./bin/register-deks [-hV] [-X=]... [] ``` The `register-deks` command has the following parameters: | Parameter | Description | |-----------------------------|--------------------------------------------------------------------| | `url` | The URL of the Schema Registry. | | `subject` | The subject of the schema. | | `version` | (Optional) The version of the schema. Default is latest. | | `-X, --property=` | (Optional) Set the configuration properties to pass to the serdes. | #### Example To register a DEK for the schema in subject `mysubject` for version `1`, the following command registers the DEK and its associated KEK: ```shell ./bin/register-deks http://localhost:8081 mysubject 1 ``` This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields. To register a DEK for the schema in subject `mysubject` for version `1` using a KEK named `mykek` with key identifier `arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678`, use the following command: ```shell ./bin/register-deks \ -X encrypt.kek.name=mykek \ -X encrypt.kms.key.id=arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678 \ http://localhost:8081 mysubject 1 ``` This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields. ### Delete a DEK #### WARNING Deleting a DEK without properly handling associated data can result in data loss. Ensure that all data associated with the DEK is properly managed before deleting the DEK. To delete a DEK, follow these steps: 1. Identify the DEK you want to delete by its DEK ID or by its KEK name, subject, and version. 2. Ensure that there are no active dependencies on the DEK. This might involve: - Decrypting any data that was encrypted with the DEK. - Re-encrypting the data with a different DEK. 3. Delete the DEK from the DEK Registry using the Schema Registry Data Encryption Keys API. For API details on deleting DEKs, see [Data Encryption Keys (v1)](https://docs.confluent.io/cloud/current/ccloud/data-encryption-keys-v-1/) in the Confluent Cloud API reference. ### Update a DEK You can update a DEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kms.key.id` and `encrypt.kms.type`, but use a different value for `encrypt.kek.name`. For details on how to update a DEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Rotate a DEK As a best practice method for rotating DEKs, use `encrypt.dek.expiry.days` to automatically rotate the DEK. The DEK will automatically rotate based on this parameter. For example, if `encrypt.dek.expiry.days` is set to 30, the DEK will be rotated every 30 days. Using this method, encrypted DEKs are persisted in the DEK Registry in Schema Registry itself, rather than in producers or consumers. The version information or the DEK is part of the metadata kept with the message. Consumers know the version of the DEK in effect for any given message, and make calls to Schema Registry to get the proper encrypted DEK. Schema Registry then makes the call to KMS to decrypt the DEK with the customer-owned KEK, which is managed in their KMS. Alternatively, you can manually rotate a DEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kms.key.id` and `encrypt.kms.type`, but use a different value for `encrypt.kek.name`. This method is generally not recommended, as it requires manual intervention, and more complexity to manage the relationships between the DEK and the KEK. To learn more, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). For best practices on rotating DEKs, see [NIST guidelines for key rotation](#nists-guidelines-key-rotation-csfle). ### Rewrap DEKs After rotating a KEK, rewrap the existing DEKs with the new KEK using the `rewrap-deks` command: ```shell ./bin/rewrap-deks [-hV] [-X=]... [] [] ``` The `rewrap-deks` command takes the following parameters: | Parameter | Description | |-----------------------------|---------------------------------------------------------------------------------------------| | `url` | The URL of the Schema Registry. | | `kekName` | The name of the KEK. | | `subject` | (Optional) The subject of the schema to rewrap. Default is all subjects. | | `includeDeleted` | (Optional) `true` to include soft-deleted DEKs, `false` to exclude them. Default is `true`. | | `-X, --property=` | (Optional) Set configuration properties. | #### Example To rewrap the DEKs for the KEK `mykek` and subject `mysubject`, run: ```shell ./bin/rewrap-deks http://localhost:8081 mykek mysubject ``` #### Rewrap using the Data Encryption Keys API The Data Encryption Keys API provides an alternative to the `rewrap-deks` command. When a DEK already exists for a subject, set the `rewrap` query parameter to `true` on the create request to rewrap the existing encrypted key material with the current KEK. For example, to rewrap the DEK for a subject: ```shell curl -u $API_KEY:$API_SECRET \ --request POST \ --header 'Content-Type: application/json' \ --data '{"subject": "{subject}"}' \ --url "$SCHEMA_REGISTRY_URL/dek-registry/v1/keks/{kekName}/deks/{subject}?rewrap=true" ``` For details, see [Data Encryption Keys (v1)](https://docs.confluent.io/cloud/current/ccloud/data-encryption-keys-v-1/) in the Confluent Cloud API reference. ## DEK durability and backup Data encryption keys (DEKs) are critical for accessing your encrypted data. Understanding how DEKs are stored and protected helps ensure business continuity and data recovery. ### DEK storage and durability DEKs are stored in the DEK Registry, which uses a Kafka topic named `_dek_registry_keys` with the following durability characteristics: - **Replication Factor**: RF=3 ensures DEKs are replicated across multiple brokers. - **Multi-AZ Protection**: DEKs are distributed across multiple availability zones. - **Synchronous Replication**: DEKs are synchronously replicated to provide strong durability guarantees. - **Minimum In-Sync Replicas**: The system requires multiple replicas to be in-sync before considering writes committed. This storage mechanism provides high availability and protection against data loss due to individual broker or availability zone failures. #### WARNING **KEK Safety is Critical**: If your Key Encryption Key (KEK) is lost or compromised, all DEKs encrypted with that KEK become inaccessible. Always ensure your KEK is properly backed up and secured in your KMS. ### Backup strategies You can implement additional backup strategies for DEKs beyond the built-in durability provided by the DEK Registry: #### Back up using the Data Encryption Keys API You can use the Data Encryption Keys API to export encrypted DEKs for backup purposes. Specify the KEK name (`{kekName}`) and subject (`{subject}`). Optionally specify the version (`{version}`) and algorithm (`{algorithm}`). - The `algorithm` parameter defaults to `AES256_GCM`. - The `version` parameter defaults to `1` if not specified. ```shell # List all DEK subjects (requires appropriate permissions) curl -u $API_KEY:$API_SECRET \ --request GET \ --url "$SCHEMA_REGISTRY_URL/dek-registry/v1/keks/{kekName}/deks" # Get specific DEK (returns version 1 by default) curl -u $API_KEY:$API_SECRET \ --request GET \ --url "$SCHEMA_REGISTRY_URL/dek-registry/v1/keks/{kekName}/deks/{subject}?algorithm={algorithm}" # Get specific DEK version curl -u $API_KEY:$API_SECRET \ --request GET \ --url "$SCHEMA_REGISTRY_URL/dek-registry/v1/keks/{kekName}/deks/{subject}?version={version}&algorithm={algorithm}" ``` #### IMPORTANT The exported DEKs remain encrypted with their associated KEK. Ensure you have secure access to the KEK in your key management service to decrypt the backed-up DEKs when needed for recovery. For details, see [Data Encryption Keys (v1)](https://docs.confluent.io/cloud/current/ccloud/data-encryption-keys-v-1/) in the Confluent Cloud API reference. #### Back up using Schema Linking Use [Schema Linking](../../../sr/schema-linking.md#schema-linking) to automatically replicate schemas to a secondary Schema Registry instance. For detailed information about Schema Linking, see [Schema Linking](../../../sr/schema-linking.md#schema-linking). - **Schema Replication**: Schema Linking replicates schemas, DEKs, and KEKs across Schema Registry instances. - **Cross-Region Protection**: Configure Schema Linking across different regions for geographic redundancy. - **Disaster Recovery**: Combined with [Cluster Linking](../../../multi-cloud/cluster-linking/dr-failover.md#cloud-cluster-linking-dr), this provides comprehensive backup and recovery capabilities for schemas. When the source subjects use CSFLE, the exporter principal needs KEK-scoped RBAC on the destination Schema Registry cluster in addition to subject-level roles. For the required roles, the diagnostic commands, and the fix for `Register on Kek` / `40301` errors, see [RBAC for Schema Linking exporters with CSFLE](../../../sr/schema-linking.md#csfle-schema-linking-rbac) and [Troubleshooting CSFLE-related exporter failures](../../../sr/schema-linking.md#csfle-schema-linking-troubleshooting) in the Schema Linking docs. #### Best practices for DEK backup - **Regular Backups**: Implement automated backup procedures for DEKs using the Data Encryption Keys API. Consider backing up DEKs immediately after creation and on a regular schedule (for example, daily or weekly) depending on your data protection requirements. - **Geographic Distribution**: Use the Data Encryption Keys API to backup DEKs across different regions. - **KEK Management**: Ensure your KEK is properly backed up and secured in your KMS. - **Access Control**: Limit access to backup DEK data to authorized personnel only. - **Secure Storage**: Store backed-up DEK data in encrypted form with appropriate access controls. - **Testing**: Regularly test your backup and recovery procedures to ensure they work correctly. - **Monitoring**: Set up monitoring and alerting for backup operations to detect failures promptly. - **Documentation**: Maintain clear documentation of your backup procedures and recovery processes. #### Recovery procedures To restore DEKs from backup: 1. **Verify KEK Access**: Ensure you have access to the original KEK in your KMS that was used to encrypt the backed-up DEKs. 2. **Restore DEKs via API**: Use the Data Encryption Keys API to restore DEKs to a new or recovered environment: ```shell # Create/restore a DEK (replace placeholders with actual values) curl -u $API_KEY:$API_SECRET \ --request POST \ --url "$SCHEMA_REGISTRY_URL/dek-registry/v1/keks/{kekName}/deks" \ --header 'Content-Type: application/vnd.dek-registry.v1+json' \ --data '{ "subject": "{subject}", "encryptedKeyMaterial": "{backed-up-encrypted-key-material}", "keyMaterialType": "AES256_GCM" }' ``` 3. **Validate Recovery**: Test that restored DEKs can successfully decrypt your data. 4. **Update Applications**: Ensure applications can access the restored DEKs in the new environment. #### WARNING **Recovery testing**: Regularly test your DEK recovery procedures to ensure they work as expected. A backup that cannot be restored is of no value. ## NIST guidelines for key rotation You are responsible for your key rotation schedule based on your organization’s compliance and security requirements. Periodic rotation of the encryption keys is recommended, even in the absence of compromise. Following the guidelines of [NIST publication 800-38D, “Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC”](https://csrc.nist.gov/pubs/sp/800/38/d/final), AES-GCM key rotation should occur before approximately 2^32 encryptions have been performed by a key version. For example, if the estimated encryption rate of a key is 40 million operations per day, then rotating a key every three months is sufficient. ## Related content **AWS KMS** > - [AWS Management Console for AWS KMS](https://console.aws.amazon.com/kms) > - [AWS CLI Command Reference for aws kms](https://docs.aws.amazon.com/cli/latest/reference/kms/index.html) > - [AWS Key Management Service API Reference](https://docs.aws.amazon.com/kms/latest/APIReference/Welcome.html) **Azure Key Vault** > - [Azure Portal](https://portal.azure.com) > - [Azure CLI Command Reference for az keyvault](https://docs.microsoft.com/en-us/cli/azure/keyvault) > - [Azure Key Vault REST API reference](https://docs.microsoft.com/en-us/rest/api/keyvault/) **Google Cloud KMS** > - [Google Cloud Console](https://console.cloud.google.com/security/kms) > - [gcloud CLI Command Reference for ‘gcloud kms’](https://cloud.google.com/sdk/gcloud/reference/kms) > - [Cloud Key Management Service (KMS) API](https://cloud.google.com/kms/docs/reference/rest) **HashiCorp Vault**: > - [Vault UI](https://www.vaultproject.io/docs/ui/index.html) > - [Vault commands (CLI)](https://www.vaultproject.io/docs/commands) > - [Vault HTTP API](https://www.vaultproject.io/api/)