Manage Client-Side Encryption for Fully-Managed Connectors in Confluent Cloud

You can enable client-side field level encryption (CSFLE) or client-side payload encryption (CSPE) to make your data more secure on fully-managed connectors. CSFLE or CSPE encrypts sensitive information, like personally identifiable information (PII) or organizational identifiable information (OII), when data enters or leaves the Confluent Cloud. This means only authorized users can access the sensitive information. CSFLE or CSPE uses encryption/decryption key through access to an external Key Management Service (KMS).

When data comes in from the external system, source connectors using CSFLE or CSPE encrypt the data before writing it to a Kafka topic. Then, sink connectors that have the correct encryption keys can decrypt the data to read the protected information or send to other target system. This way, both when data is being produced and consumed, CSFLE or CSPE protects sensitive data, allowing only authorized people access.

This guide describes how developers can use CSFLE or CSPE in fully-managed source and sink connectors.

Limitations

The connector does not support auto schema registration. You must manually register the schema.
The connector does not support auto schema evolution.
The connector only supports encryption for schema-based input data formats such as Avro, JSON_SR, and Protobuf.
The connector only supports encryption for fields of type string or bytes for CSFLE.
The CSFLE does not work for string and bytes fields within nested JSON_SR formats when the value.converter.decimal.format is set to BASE64. To workaround this limitation, set value.converter.decimal.format to NUMERIC.
For source connector, CSFLE or CSPE fails to encrypt data if Confluent Cloud does not have access to Key Encryption Keys (KEKs) in your external KMS. For more information, see Creating KEK in an external KMS.
For sink connector, CSFLE or CSPE may continue to share encrypted data to the target system if you choose to ignore (NONE) connector behavior on data decryption failures, specifically related to accessing the KEKs. By default, the connector behavior on data decryption failure is set to ERROR and the connector throws a DataException error and writes the encrypted data to the dead letter queue (DLQ).
Important
When using CSFLE or CSPE with connectors that route failed messages to a Dead Letter Queue (DLQ), be aware that data sent to the DLQ is written in plaintext (unencrypted). This poses a significant security risk as sensitive data that should be encrypted may be exposed in the DLQ.
Do not use DLQ with CSFLE or CSPE in the current version. If you need error handling for CSFLE- or CSPE-enabled data, use alternative approaches such as:
- Setting the connector behavior to ERROR to throw exceptions instead of routing to DLQ
- Implementing custom error handling in your applications
- Using NONE to pass encrypted data through without decryption
CSFLE- or CSPE-enabled connectors typically require about 5 minutes to apply the latest rules. For example, if you add a new PII tag to a new field, the active connector will start using this rule 5 minutes later.

Supported connectors

The following connectors support CSFLE and CSPE:

Source connectors

The following fully-managed source connectors support CSFLE and CSPE:

Sink connectors

The following fully-managed sink connectors support CSFLE and CSPE:

Manage client-side encryption

The client-side encryption is available in Confluent Cloud to help you protect sensitive data in your Confluent Cloud account and perform stream processing on encrypted data. You must configure CSFLE or CSPE in Confluent Cloud before you modify an existing connector or create a new one with CSFLE or CSPE enabled.

At a high level, you can manage client-side encryption for connectors using the following 2-step process:

Configure CSFLE in Confluent Cloud or Configure CSPE in Confluent Cloud
Enable CSFLE or CSPE in connectors

Configure CSFLE in Confluent Cloud

Complete the following configuration steps:

Define the schema for the topic and add tags to the fields in the schema that you want to encrypt.
Create encryption keys for each KMS and allow Confluent Cloud to access your KMS.
Add encryption rules that specify the encryption key you want to use to encrypt the tags.
Modify an existing service account or create a new one with DeveloperWrite RBAC grant to schema and encryption keys in the schema registry. This ensures that the connector can read and create schemas and data encryption keys (DEKs) it may require for upstream and downstream operations.

For more information about CSFLE configuration in Confluent Cloud, see Manage Client-Side Field Level Encryption using Confluent Cloud Console.

Configure CSPE in Confluent Cloud

Complete the following configuration steps:

Define the schema for the topic that you want to encrypt.
Create encryption keys and allow Confluent Cloud to access your KMS.
Add payload encryption encoding rules that specify the encryption key you want to use to encrypt the tags.
Modify an existing service account or create a new one with DeveloperWrite RBAC grant to schema and encryption keys in the schema registry. This ensures that the connector can read and create schemas and data encryption keys (DEKs) it may require for upstream and downstream operations.

For more information about CSPE configuration in Confluent Cloud, see Protect Sensitive Data Using Client-Side Payload Encryption on Confluent Cloud.

Enable CSFLE or CSPE in connectors

Use Confluent Cloud console to modify connectors or create new ones with CSFLE or CSPE enabled, protecting your most sensitive data across producers and consumers, thereby securing all data in motion, including the most sensitive information.

Source connector considerations:

Confluent recommends creating new source connectors for CSFLE or CSPE with separate topics for encryption and non-encryption scenarios, rather than modifying existing connectors for CSFLE or CSPE.

However, if you choose to enable CSFLE or CSPE on an existing source connector, you should follow the below step:

Pause the connector and ensure no other source connector writes to the same topic, to prevent schema conflicts and overrides.
Register the new schema, tags, encryption keys, and rules, ensuring the latest schema version adds these elements. For more information, see Configure CSFLE in Confluent Cloud or Configure CSPE in Confluent Cloud.
Update the paused connector’s configuration to enable CSFLE or CSPE.
Resume the connector.

To activate CSFLE or CSPE for supported connectors, see the quick start section of the supported connectors.

Source connector

At the Add Source Connector screen, complete the following in the configuration step:

In Data encryption section, click Enable Client-Side Field Level Encryption to encrypt the data when writing to a topic.
Specify a service account to access the Schema Registry and associated encryption rules or keys with that schema.

For more information about complete connector configuration, including CSFLE or CSPE setup, see the configuration step, for example, when creating a S3 Source connector.

Sink connector

At the Add Sink Connector screen, complete the following in the configuration step:

In Data decryption section, click Enable Client-Side Field Level Encryption to decrypt the encrypted data before transferring into a target system.
Specify a service account to access the Schema Registry and associated encryption rules or keys with that schema.
Select the connector behavior on data decryption failure. Allowed values are ERROR and NONE.
- If you select NONE, the connector writes the encrypted data in the target system without decryption.
- If you select ERROR, the connector throws an error and writes the encrypted data in the DLQ system.

For more information about complete connector configuration, including CSFLE or CSPE setup, see the configuration step, for example, when creating a S3 Sink connector.