# Manage Encryption Keys for CSFLE on Confluent Platform Key encryption keys (KEKs) and data encryption keys (DEKs) are used together for client-side field level encryption (CSFLE) on Confluent Platform. DEKs are used to encrypt and decrypt the sensitive data in data fields in your Confluent Platform resources, while KEKs are used to securely manage the DEKs. The KEKs are managed and stored in an external key management service (KMS), such as AWS KMS, Azure Key Vault, or Google Cloud KMS. #### NOTE RBAC bindings for CSFLE are evaluated on KEK resources using the KEK name (`Kek:`). You do not bind roles directly on DEKs. DEK operations are authorized by the KEK binding and request scope (for example, subject/version). ## Key encryption keys (KEKs) Key encryption keys (KEKs) are master keys used to securely manage data encryption keys (DEKs), decrypting them only when needed by the client. Only users with access to the KEK can decrypt the DEK and access your sensitive data. The KEK is managed and stored at the Key Management Service (KMS) provider, such as AWS KMS, Azure Key Vault, or Google Cloud KMS. However, the DEK Registry needs to be aware of the KEK in order to use it for encrypting and decrypting DEKs. This means that while the KEK itself is not stored in the DEK Registry, metadata about the KEK (such as its name and key identifier) is registered in the DEK Registry. #### IMPORTANT The ability to decrypt a DEK is NOT controlled by Confluent Platform RBAC roles. DEK encryption and decryption permissions are enforced by your cloud provider KMS (for example, AWS IAM on the KMS key). Ensure the DEK Registry principal (or client) has the necessary KMS permissions. ### KEK parameters A key encryption key (KEK) registered to the DEK Registry has the following parameters: | Parameter | Description | |-------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `name` | A meaningful name for the KEK. The name is used when referring to the
KEK elsewhere, such as in RBAC. | | `kmsType` | The type of KMS, typically one of “aws-kms”, “azure-kms”, and “gcp-kms”. | | `kmsKeyId` | The key identifier for the KEK. When using AWS KMS, the `kmsKeyId`
is the ARN.

For AWS multi-region keys (MRKs), if the ARN is copied from one region to
another, the ARN is dynamically translated to be an MRK for the new region.
The region specified in the ARN is dynamically replaced with the actual
region in which the AWS MRK is used. This facilitates use cases like
Schema Linking across different regions.

**Example of an AWS multi-region key (MRK)**:

If the initial region is `us-east-1` and the following AWS MRK is used:

```text
arn:aws:kms:us-east-1:XXXXXX:key/mrk-XXXXXX
```

When the AWS MRK is used in the `us-west-2` region, the ARN is dynamically
translated to:

```text
arn:aws:kms:us-west-2:XXXXXX:key/mrk-XXXXXX
```

The only difference in the ARN is the region, where `us-east-1` is replaced
with `us-west-2`.

To disable this dynamic behavior, set the property `aws.disable.dynamic.mrk=true`
in the `kmsProps` of the KEK in the DEK Registry.

Ensure the correct policy is added for each MRK in every region. Without
the correct policy, the DEK Registry cannot access the appropriate
key in each region.

For details, see [Multi-Region keys in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html). | | `kmsProps` | Additional key-value properties used to access the KMS. | | `doc` | (Optional) A meaningful description for the KEK. | | `ts` | (Optional) The timestamp indicating when the KEK was registered or updated. | ### Create a KEK #### WARNING Ensure that the KEK is created in a secure environment and that only authorized users have access to it. Improper handling of KEKs can lead to unauthorized access to DEKs and sensitive data. To create a KEK, use the key management service (KMS) you are using for CSFLE. For details on how to create a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Register a KEK After you create a KEK, you can register the KEK before use by using the `register-deks` command to register both the DEKs and KEKs. For details, see the [Register a DEK](#register-dek-csfle) section and review the example that specifies the KEK parameters. If you do not want to register the KEK before use, you can register it on demand by the client, assuming that the client has the appropriate permissions with the DEK Registry. ### Delete a KEK #### WARNING Deleting a KEK without properly handling associated DEKs can result in data loss. Ensure that all DEKs are properly managed before deleting the KEK. To delete a KEK, follow these steps: 1. Identify the KEK. Determine the KEK you want to delete by its name or key identifier. 2. Delete Associated DEKs. Ensure that all Data Encryption Keys (DEKs) associated with the KEK are deleted. This might involve: - Decrypting any data that was encrypted with the DEKs. - Re-encrypting the data with DEKs associated with a different KEK. 3. Delete the KEK. Use the KMS Interface to access the key management service (KMS) management interface or API to delete the KEK. For details on how to delete a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Update a KEK You can update a KEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kek.name`. To update a KEK, follow these steps: 1. **Identify the KEK**: Determine the KEK you want to update by its name or key identifier. 2. **Update KEK Parameters**: Modify the parameters of the KEK as needed. This might involve changing properties such as the KEK name, KMS type, or KMS key identifier. 3. **Update the KEK**: Use the KMS Interface to access the key management service (KMS) management interface or API to apply the updates to the KEK. For details on how to update a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). ### Rotate a KEK You can manually rotate the KEK on your key management service (KMS). When you rotate a KEK, you need to re-encrypt the existing DEKs with the new KEK. To manually rotate the DEK, publish a new version of the schema with a different value for `encrypt.kek.name`. To rotate a KEK, you need to: 1. Decrypt the DEKs that are encrypted with the existing KEK. 2. Encrypt the DEKs with the new KEK. 3. Update the DEK Registry with the DEKs encrypted with the new KEK. This process ensures that the DEKs remain secure and can still be used to encrypt and decrypt your data. For details on how to rotate a KEK, see the [documentation for your key management service (KMS)](#kms-related-content-csfle). For best practices on rotating KEKs, see [NIST guidelines for key rotation](#nists-guidelines-key-rotation-csfle). After rotating a KEK, rewrap the existing DEKs with the new KEK. For details, see [Rewrap DEKs](#rewrap-deks-csfle). ### Bind a role to KEK resource Set role bindings for a KEK resource when you need to control which users, groups, or service principals have the authority to perform actions related to that particular KEK. You can set these bindings using the following Confluent Platform CLI command: ```bash confluent iam rbac role-binding create \ --schema-registry my-schema-registry-id \ --principal User:system-user \ --role DeveloperRead \ --resource Kek:mykek_new ``` In this example, the command creates a new role binding associated with the principal `User:system-user` and grants the `DeveloperRead` role on the `Kek:mykek_new` within the `my-schema-registry-id` Schema Registry instance. This command allows the system-user to perform actions like reading the KEK’s details, which might be necessary for decrypting data encrypted using DEKs protected by this KEK. The exact permissions granted by the `DeveloperRead` role for a KEK resource defines these permissions within the Confluent Platform’s RBAC configuration. ## Data encryption keys (DEKs) Data encryption keys (DEKs) are symmetric keys that are used to encrypt and decrypt the sensitive data in your Confluent Platform resources. Each DEK is encrypted with a key encryption key (KEK) that is only accessible to authorized users. The encrypted DEK and encrypted data are stored together. Only users with access to the KEK can decrypt the DEK and access the sensitive data. DEKs are managed in the DEK Registry in Schema Registry, not in your key management service (KMS). Your KMS holds only the KEK that encrypts each DEK. As a result, you create, update, rotate, and delete DEKs through Schema Registry, not through your KMS. ### DEK parameters During encryption, the client asks the DEK Registry for an existing DEK for a specified KEK name and subject (when using automatic DEK rotation, version is required). If a DEK does not exist, the client generates and encrypts the DEK using the KEK in your KMS, and then registers the encrypted DEK with the DEK Registry. A DEK registered to the DEK Registry has the following parameters: | Parameter | Description | |------------------------|-----------------------------------------------------------------------------------------------------------------------| | `kekName` | The name of the KEK used to encrypt this DEK. | | `subject` | The subject for the DEK. | | version | The version of the DEK. | | `algorithm` | The encryption algorithm being used. Valid values include: `AES128_GCM`,
`AES256_GCM` (default), or `AES256_SIV`. | | `encryptedKeyMaterial` | The encrypted key material for the DEK. | | `ts` | The timestamp indicating when the DEK was registered. | ### Create a DEK #### WARNING Ensure that the DEK is created in a secure environment and that only authorized users have access to it. Improper handling of DEKs can lead to unauthorized access to sensitive data. To create a DEK, you need to specify the KEK name and subject. The subject is a unique identifier for the DEK, such as a namespace or collection. For the steps to create and register a DEK, see the following section. ### Register a DEK To register a DEK, use the `register-deks` command, replacing `` with the URL of the Schema Registry, `` with the subject of the schema, and `` with the version of the schema. ```shell ./bin/register-deks [-hV] [-X=]... [] ``` The `register-deks` command has the following parameters: | Parameter | Description | |-----------------------------|--------------------------------------------------------------------| | `url` | The URL of the Schema Registry. | | `subject` | The subject of the schema. | | `version` | (Optional) The version of the schema. Default is latest. | | `-X, --property=` | (Optional) Set the configuration properties to pass to the serdes. | #### Example To register a DEK for the schema in subject `mysubject` for version `1`, the following command registers the DEK and its associated KEK: ```shell ./bin/register-deks http://localhost:8081 mysubject 1 ``` This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields. To register a DEK for the schema in subject `mysubject` for version `1` using a KEK named `mykek` with key identifier `arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678`, use the following command: ```shell ./bin/register-deks \ -X encrypt.kek.name=mykek \ -X encrypt.kek.key.id=arn:aws:kms:us-east-1:123456789012:key/abcd-1234-efgh-5678 \ http://localhost:8081 mysubject 1 ``` This command registers the DEK and its associated KEK with the DEK Registry, making them available for use in encrypting and decrypting data fields. ### Delete a DEK #### WARNING Deleting a DEK without properly handling associated data can result in data loss. Ensure that all data associated with the DEK is properly managed before deleting the DEK. To delete a DEK, follow these steps: 1. Identify the DEK you want to delete by its name or key identifier. 2. Ensure that there are no active dependencies on the DEK. This might involve: - Decrypting any data that was encrypted with the DEK. - Re-encrypting the data with a different DEK. 3. Delete the DEK using the DEK Registry interface or API in Schema Registry. ### Update a DEK You can update a DEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kms.key.id` and `encrypt.kms.type`, but use a different value for `encrypt.kek.name`. For details on the parameters used to update a DEK, see [Configure client-side field level encryption](client-side.md#configure-csfle). ### Rotate a DEK As a best practice method for rotating DEKs, use `encrypt.dek.expiry.days` to automatically rotate the DEK. The DEK will automatically rotate based on this parameter. For example, if `encrypt.dek.expiry.days` is set to 30, the DEK will be rotated every 30 days. Using this method, encrypted DEKs are persisted in the DEK Registry in Schema Registry itself, rather than in producers or consumers. The version information of the DEK is part of the metadata kept with the message. Consumers know the version of the DEK in effect for any given message, and make calls to Schema Registry to get the proper encrypted DEK. The client then makes the call to the KMS to decrypt the DEK with the customer-owned KEK, which is managed in their KMS. Alternatively, you can manually rotate a DEK by publishing a new version of the schema (defining how data is encrypted and decrypted) with the same value for `encrypt.kms.key.id` and `encrypt.kms.type`, but use a different value for `encrypt.kek.name`. This method is generally not recommended, as it requires manual intervention, and more complexity to manage the relationships between the DEK and the KEK. To learn more about the `encrypt.dek.expiry.days` and `encrypt.kek.name` parameters used to rotate DEKs, see [Configure client-side field level encryption](client-side.md#configure-csfle). For best practices on rotating DEKs, see [NIST guidelines for key rotation](#nists-guidelines-key-rotation-csfle). ### Rewrap DEKs After rotating a KEK, rewrap the existing DEKs with the new KEK using the `rewrap-deks` command: ```shell ./bin/rewrap-deks [-hV] [-X=]... [] [] ``` The `rewrap-deks` command takes the following parameters: | Parameter | Description | |-----------------------------|-------------------------------------------------------------------------------------------------| | `url` | The URL of the Schema Registry. | | `kekName` | The name of the KEK. | | `subject` | (Optional) The subject of the schema to rewrap. Default is all subjects. | | `includeDeleted` | (Optional) `true` to include soft-deleted DEKs, `false` to exclude
them. Default is `true`. | | `-X, --property=` | (Optional) Set configuration properties. | #### Example To rewrap the DEKs for the KEK `mykek` and subject `mysubject`, run: ```shell ./bin/rewrap-deks http://localhost:8081 mykek mysubject ``` ## NIST guidelines for key rotation Periodic rotation of the encryption keys is recommended, even in the absence of compromise. Following the guidelines of [NIST publication 800-38D, “Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC”](https://csrc.nist.gov/pubs/sp/800/38/d/final), AES-GCM key rotation should occur before approximately 2^32 encryptions have been performed by a key version. For example, if the estimated encryption rate of a key is 40 million operations per day, then rotating a key every three months is sufficient. ## Key management service (KMS) - related content **AWS KMS** : - [AWS Management Console for AWS KMS](https://console.aws.amazon.com/kms) - [AWS CLI Command Reference for aws kms](https://docs.aws.amazon.com/cli/latest/reference/kms/index.html) - [AWS Key Management Service API Reference](https://docs.aws.amazon.com/kms/latest/APIReference/Welcome.html) **Azure Key Vault** : - [Azure Portal](https://portal.azure.com) - [Azure CLI Command Reference for az keyvault](https://docs.microsoft.com/en-us/cli/azure/keyvault) - [Azure Key Vault REST API reference](https://docs.microsoft.com/en-us/rest/api/keyvault/) **Google Cloud KMS** : - [Google Cloud Console](https://console.cloud.google.com/security/kms) - [gcloud CLI Command Reference for ‘gcloud kms’](https://cloud.google.com/sdk/gcloud/reference/kms) - [Cloud Key Management Service (KMS) API](https://cloud.google.com/kms/docs/reference/rest) **HashiCorp Vault**: : - [Vault UI](https://www.vaultproject.io/docs/ui/index.html) - [Vault commands (CLI)](https://www.vaultproject.io/docs/commands) - [Vault HTTP API](https://www.vaultproject.io/api/)