Protect Sensitive Data Using Client-Side Field Level Encryption on Confluent Platform

Important

CSFLE for Confluent Platform is available as an early access feature as a component of Confluent Platform introduced to gain feedback. This feature should be used only for evaluation and non-production testing purposes or to provide feedback to Confluent, particularly as it becomes more widely available in follow-on preview editions.

Early Access Program features are intended for evaluation use in development and testing environments only, and not for production use. Early Access Program features are provided: (a) without support; (b) “AS IS”; and (c) without indemnification, warranty, or condition of any kind. No service level commitment will apply to Early Access Program features. Early Access Program features are considered to be a Proof of Concept as defined in the Confluent Terms of Service. Confluent may discontinue providing preview releases of the Early Access Program features at any time in Confluent’s sole discretion.

Confluent Enterprise provides a suite of security features to tackle issues related to authorization, data confidentiality, and encryption. These include role-based access control, encryption at rest with the option for using your own encryption keys, and encryption in transit using TLS. Using client-side field level encryption (CSFLE) enhances the protection of sensitive data for both producers and consumers, ensuring comprehensive security for information in motion, particularly highly sensitive data.

CSFLE enables encryption at the individual field level, allowing the secure management of sensitive information, like personally identifiable information (PII), at both ends – producer and consumer. This method allows for the encrypted data to be accessible only to authorized parties, ensuring that sensitive information remains confidential and secure.

This form of encryption adds a layer of security by protecting selective data elements, making it inaccessible to unauthorized users who, even upon accessing the data, would only encounter encrypted, indecipherable values without the proper decryption keys.

Field-level encryption is carried out on the client side, meaning sensitive data is encrypted before it’s stored or sent and decrypted only by those with authorization. Benefits of using CSFLE include:

  • A targeted and flexible approach to securing sensitive information, enabling selective encryption of specific data fields.
  • Maintains stringent control over who can access sensitive data, preventing access even by administrators unless authorized, thereby bolstering data privacy.
  • Reduces infrastructure and operational costs by negating the need for multiple data copies for access management.
  • Helps in complying with regulations tailored to protect sensitive information like PII, ensuring data remains secure throughout its lifecycle, even when shared across systems or with third parties.

For requirements and supported clients, see Requirements.

Use CSFLE for Confluent Enterprise

Client-side field level encryption (CSFLE) is available in Confluent Enterprise to help you protect sensitive data in your Confluent Enterprise and perform stream processing on encrypted data. You can use CSFLE with Confluent Enterprise without sharing access to your Key Encryption Keys (KEKs). Here are some key points about using CSFLE:

  • You must use a key management service (KMS) to manage access to your Key Encryption Keys (KEKs).
  • Extensive security checks and balances provided by Confluent protect your sensitive data.
  • No user or application in Confluent Enterprise can access your encrypted fields in plaintext.
  • Stream processing in Confluent Enterprise using Flink is not possible because the data is encrypted and cannot be decrypted to perform operations.
  • Your organization manages running producers and consumers with the proper configurations to access the KEKs and encrypt or decrypt data.
  • You must use a key management service (KMS) to manage access to your Key Encryption Keys (KEKs).
  • Extensive security checks and balances provided by Confluent protect your sensitive data.
  • You own and manage your Key Encryption Keys (KEKs) and are responsible for overseeing the entire lifecycle of the KEKs.
  • Sharing access to KEKs is not supported.
  • Confluent never directly accesses your Key Encryption Keys (KEKs). Each KEK remains securely stored in your key management service (KMS) that is owned and managed by you. Confluent interacts with two APIs that use a KEK identifier and a payload to either encrypt or decrypt the payload with the specified KEK. Confluent can only see the KEK identifier and the payloads (encrypted or decrypted DEKs).
  • Use the logging and auditing capabilities provided by your KMS to monitor and trace all access to KEKs to address any compliance or regulatory requirements.

The steps are summarized in the diagram below.

Steps for client-side field level encryption and access control

FIPS 140-2 compliance

FIPS 140-2 serves as a standard for validating cryptographic modules, and is often used to ensure that cryptographic modules are secure and compliant with regulatory requirements. CSFLE uses Google Tink format for DEKs to ensure interoperability with Java and Google Cloud clients. You can use Google Tink for common cryptographic operations to meet FIPS 140-2 requirements. For more information, see Use Tink to meet FIPS 140-2 security requirements.

Performance considerations

CSFLE can impact performance. The extent of this impact depends primarily on the number of encrypted fields and the nature of your workload. In light usage scenarios, the throughput and latency overhead might be minimal. However, heavy encryption and decryption across multiple fields in every read and write operation can significantly degrade performance. Note the following:

  • CSFLE can reduce the effectiveness of data compression. Encryption increases the randomness of data, making it harder for compression algorithms to find patterns.
  • CSFLE can reduce throughput (fewer messages per second). All encryption and decryption occur on the client, so there is no direct performance burden placed on the brokers.