Manage Client-Side Field Level Encryption using Confluent Cloud Console¶
You can use the Confluent Cloud Console to manage client-side field level encryption (CSFLE) for protecting your organization’s sensitive data in Confluent Cloud on AWS, Azure, and Google Cloud.
For requirements and supported clients, see Requirements.
Steps to manage CSFLE¶
To manage CSFLE, here are the high-level steps:
- Define the schema for the data that you want to encrypt.
- Add tags to the fields in the schema that you want to encrypt.
- Create encryption keys.
- Add encryption rules.
- Grant RBAC permissions.
If you do not grant Confluent access to your Key Encryption Key (KEK), you need to set up the proper configurations for producers and consumers to access the KEKs.
View encryption rules¶
To view encryption rules:
Go to the Confluent Cloud Console and select your environment.
In the left navigation menu, click Stream Governance > Schema Registry > Rules.
The Rules page displays.
On the Rules page, you see a listing of any existing encryption rules. For each of the existing rules, the following information is displayed:
- Schema subject name
- Rule name
- Rule category
- Tags
Add an encryption rule¶
Follow the steps below to add encryption rules, which define the fields that you want to encrypt and the encryption key that you want to use to encrypt the fields. You also specify tags to identify the fields that you want to encrypt.
- To minimize risk, you can use different encryption keys for different fields. For example, you can use different encryption keys for a credit card number and the CVC code. If someone compromises one of the keys, the secured key continues to protect some of your data.
- Encryption is only supported for fields of type
string
orbytes
.
Open the Confluent Cloud Console and select your environment.
In the left navigation menu, click Stream Governance > Schema Registry > Rules.
The Rules page displays.
If there are no existing rules, click Evolve and then click Add rules. If there are existing rules, click Data contracts, click Evolve, and then click Rules.
The Domain rules page displays.
Click Add rule. The Add rule side panel displays.
Create encryption rules for the fields in the schema subject that you want to encrypt.
Open the Category dropdown list and select Data encryption rule.
Enter a Rule name. Enter a meaningful name that is unique within the schema subject.
Enter a Description.
In the Encrypt fields with section, select the Tags that the rules apply to and select the Encryption key that you want to use.
In the Apply action on section, select the options that you want to apply to the fields that you want to encrypt. You can apply the following actions:
- On failure (WRITE): Default is ERROR.
- On success (WRITE): Default is NONE.
- On failure (READ): Default is ERROR.
- On success (READ): Default is NONE.
For each action, you can select the following options:
- None
- DLQ
- Error
For example, if you select On failure (WRITE) and DLQ, and the write operation fails, the record is sent to the dead letter queue.
If you select On success (WRITE) and Error, and the write operation succeeds, an error is returned.
In the Parameters section, you can optionally add parameters to the rule.
The parameters are key-value pairs that you can use to configure the rule.
Click Add. The new rule defines the fields that you want to encrypt and the encryption key that you want to use to encrypt the fields. You can use the same encryption key for multiple fields.
Click Add to save the rule, or click Add another rule to add more encryption rules.
The Encryption rules page displays with the new encryption rules.
Add an encryption key¶
Before you can add an encryption rule, you must create an encryption key. You can create an encryption key using the Confluent Cloud Console.
Open the Confluent Cloud Console and select your environment.
In the left navigation menu, click Stream Governance > Schema Registry > Encryption keys.
The Encryption keys page displays.
Click Add encryption key.
The Add encryption key page displays.
In the Name field, enter a meaningful name.
In the Key management system provider field, select the key management service that you want to use to manage the encryption key. Supported key management options include AWS, Azure, and Google Cloud.
Follow the steps below to create an encryption key for each key management service if you are allowing Confluent to access your key management service. If you are not allowing Confluent to access your key management service, you can skip the steps below.
- Enter the following information.
Field | Description |
---|---|
Name | Enter a meaningful name. |
Key management system provider | Select AWS. |
Amazon resource name (key ID) | Go to your AWS Management Console, get the resource name, and enter it. |
Description | Enter a useful description. |
Enable Share encryption key access with Confluent Cloud.
This enables Confluent Cloud to use the Key Encryption Key (KEK) to temporarily decrypt fields required for operations within Confluent Cloud resources, such as Schema Registry, ksqlDB, and Flink. To filter data within your records, encrypted content needs to be temporarily decrypted within Confluent Cloud to perform the operation. Confluent Cloud does not store decrypted data, and Confluent Cloud users cannot access it.
Ensure that the AWS key policy of the AWS KMS key entered above contains the code block (displaying in the Confluent Cloud Console) to authorize access to the key by Confluent to perform stream processing.
Click Add. The Encryption rules page displays with the new encryption key.
Enter the following information.
Field Description Name Enter a meaningful name. Key management system provider Select Azure. Azure Key Vault resource name (key ID) Go to your Azure Portal Key Vaults page, copy the resource name, and paste it. Azure tenant ID Enter your Azure tenant ID. Azure client ID Enter your Azure client ID. The clientId
is created below in the step for creating a user-assigned managed identity.Description Enter a useful description. Enable Share encryption key access with Confluent Cloud. This enables Confluent Cloud to use the Key Encryption Key (KEK) to temporarily decrypt fields required for operations within Confluent Cloud resources, such as Schema Registry, ksqlDB, and Flink. To filter data within your records, encrypted content needs to be temporarily decrypted within Confluent Cloud to perform the operation. Confluent Cloud does not store decrypted data, and Confluent Cloud users cannot access it.
Create a Resource Group or use an existing one. Here’s an example of creating a Resource Group:
az group create --name rg-confluent-keyvault-access --location <region>
For more information, see az group create.
Create a user-assigned managed identity or use an existing one. Make note of the
clientId
. Here’s an example of creating an identity and outputting itsclientId
:az identity create --name confluent-keyvault-access --resource-group rg-confluent-keyvault-access --query clientId -o tsv
For more information, see az identity create.
Create a custom role with the following permissions:
Microsoft.KeyVault/vaults/keys/read Microsoft.KeyVault/vaults/keys/encrypt/action Microsoft.KeyVault/vaults/keys/decrypt/action
Here’s an example of creating a custom role assignable at the subscription level, called “Confluent Schema Registry Key Vault Access”:
az role definition create \ --role-definition '{ "Name": "Confluent Schema Registry Key Vault Access", "Description": "Confluent Schema Registry Key Vault Access", "Actions": [ "Microsoft.KeyVault/vaults/keys/read", "Microsoft.KeyVault/vaults/keys/encrypt/action", "Microsoft.KeyVault/vaults/keys/decrypt/action" ], "AssignableScopes": [ "/subscriptions/<subscription-id>" ] }'
For more information, see Azure custom roles.
Get the
principalId
of the user-assigned managed identity and grant it access to the key vault using the “Confluent Schema Registry Key Vault Access” role created above. Here’s an example:principalId=$(az identity show -n confluent-keyvault-access -g rg-confluent-keyvault-access --query principalId -o tsv) az role assignment create \ --role "Confluent Schema Registry Key Vault Access" \ --scope "$(az keyvault show --name "<your keyvault name>" --query id --output tsv)" \ --assignee-object-id "$principalId"
Create a federated identity credential between the user-assigned managed identity and the PSRC’s service account. This can be done running the Azure CLI command displaying in Confluent Cloud Console. Here’s an example for
psrc-abc123
:az identity federated-credential create \ --name fc-confluent-keyvault-access \ --identity-name confluent-keyvault-access \ --resource-group rg-confluent-keyvault-access \ --issuer https://centralus.oic.prod-aks.azure.com/00000000-0000-0000-0000-000000000000/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/ \ --subject system:serviceaccount:psrc-abc123:default
For more information, see Federated Identity Credential.
Click Add. The Encryption rules page displays with the new encryption key.
Enter the following information.
Field Description Name Enter a meaningful name. Key management system provider Select GCP. Google Cloud resource name (key ID) Go to your Google Cloud Key Management page, copy the resource name, and paste it. Description Enter a useful description. Enable Share encryption key access with Confluent Cloud. This enables Confluent Cloud to use the Key Encryption Key (KEK) to temporarily decrypt fields required for operations within Confluent Cloud resources, such as Schema Registry, ksqlDB, and Flink. To filter data within your records, encrypted content needs to be temporarily decrypted within Confluent Cloud to perform the operation. Confluent Cloud does not store decrypted data, and Confluent Cloud users cannot access it.
In the Google Cloud Console, create a custom role and add the following required permissions to the role:
cloudkms.cryptoKeyVersions.useToDecrypt cloudkms.cryptoKeyVersions.useToEncrypt
Go to the Google Cloud KMS console and give access to the Confluent Service Account with the custom role selected. Here is an example of the Confluent Service Account:
psrc-abc123@cc-gc.iam.gserviceaccount.com
Click Add. The Encryption rules page displays with the new encryption key.