Configure Client-Side Field Level Encryption on Confluent Platform

Client-side field level encryption (CSFLE) supports the following configuration parameters for the different rule executors. A rule executor is a component that performs field-level encryption transformations and is responsible for the following:

  • Executing encryption and decryption operations based on defined rules.
  • Determining which fields should be encrypted or decrypted.
  • Applying the appropriate encryption keys and algorithms according to your configuration.

The rule executor ensures that your encryption policies are consistently applied across your application without requiring manual encryption or decryption calls for each operation.

Configure CSFLE

You can configure CSFLE using either configuration parameters or environment variables. Both methods are supported across all rule executors, and you can choose the approach that best fits your security and deployment requirements.

Configuration Methods

Use configuration parameters

Specify parameters directly in your configuration file or application settings:

# AWS Example
rule.executors._default_.param.access.key.id=AKIAIOSFODNN7EXAMPLE
rule.executors._default_.param.secret.access.key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Azure Example
rule.executors._default_.param.tenant.id=your-tenant-id
rule.executors._default_.param.client.id=your-client-id
rule.executors._default_.param.client.secret=your-client-secret

Use environment variables

Set environment variables before running your application:

# AWS Example
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Azure Example
export AZURE_TENANT_ID=your-tenant-id
export AZURE_CLIENT_ID=your-client-id
export AZURE_CLIENT_SECRET=your-client-secret

# Google Cloud Example
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json

# HashiCorp Vault Example
export VAULT_TOKEN=your-token-value
export VAULT_NAMESPACE=your-namespace

Best Practices

Environment variables are generally preferred for sensitive credentials to:

  • Keep secrets out of configuration files
  • Reduce the risk of accidentally committing credentials to version control
  • Allow for easier credential rotation in containerized environments

For development and testing:

  • Use the Local CSFLE rule executor with a simple secret.
  • Generate a test secret: openssl rand -base64 16.
  • Set it using: export LOCAL_SECRET=generated-secret-value.

For production environments:

  • Use cloud provider default credential chains when possible.
  • Implement proper secret management and rotation procedures.
  • Consider using managed identity services where available.

Common Parameters

Use the following configuration parameters with all CSFLE rule executors.

Parameter Description
rule.executors._default_.param.preserve.source.fields For performance reasons, the fields of a message are updated during field-level transforms. For field-level encryption, this results in the field values being replaced with the encrypted field values. If the original field values should be retained in the message, then set this property to true.

AWS CSFLE rule executor

You can pass the following configuration parameters or specify the values using environment variables, such as the following:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_PROFILE
  • AWS_ROLE_ARN
  • AWS_ROLE_SESSION_NAME
  • AWS_ROLE_EXTERNAL_ID

If no configuration parameters are passed, the client uses the default credentials provider chain.

Parameter Description
rule.executors._default_.param.access.key.id The AWS access key identifier.
rule.executors._default_.param.secret.access.key The AWS secret access key.
rule.executors._default_.param.profile The AWS profile to use.
rule.executors._default_.param.role.arn The AWS role ARN to use.
rule.executors._default_.param.role.session.name The AWS role session name to use.
rule.executors._default_.param.role.external.id The AWS role external ID to use.

Azure CSFLE rule executor

You can pass the following configuration parameters or specify the values using environment variables, such as:

  • AZURE_TENANT_ID
  • AZURE_CLIENT_ID
  • AZURE_CLIENT_SECRET

If no configuration parameters are passed, the client uses the default credentials chain.

Parameter Description
rule.executors._default_.param.tenant.id The Azure tenant identifier.
rule.executors._default_.param.client.id The Azure client identifier.
rule.executors._default_.param.client.secret The Azure client secret.

Google Cloud CSFLE rule executor

You can pass the following configuration parameters or specify the values using the environment variable, such as GOOGLE_APPLICATION_CREDENTIALS.

If no configuration parameters are passed, the client uses the application default credentials.

Parameter Description
rule.executors._default_.param.account.type The Google Cloud account type.
rule.executors._default_.param.client.id The Google Cloud client identifier.
rule.executors._default_.param.client.email The Google Cloud client email address.
rule.executors._default_.param.private.key.id The Google Cloud private key identifier.
rule.executors._default_.param.private.key The Google Cloud private key.

HashiCorp Vault CSFLE rule executor

You can pass the following configuration parameters or specify the values using environment variables named:

  • VAULT_TOKEN
  • VAULT_NAMESPACE
Parameter Description
rule.executors._default_.param.token.id The token identifier for HashiCorp Vault.
rule.executors._default_.param.namespace (Optional) The namespace for HashiCorp Vault Enterprise.

Local CSFLE rule executor

For testing only, you can pass the following configuration parameter or specify the value using an environment variable named LOCAL_SECRET.

Parameter Description
rule.executors._default_.param.secret A randomly generated secret, such as one obtained by running openssl rand -base64 16quick.

Related content