Protect Data at Rest Using Self-managed Encryption Keys on Confluent Cloud

Basic encryption of data at rest is provided to all Confluent Cloud clusters by default. This encryption is transparent to the user and is managed by Confluent. As a stronger security option, Confluent Cloud provides optional support for self-managed encryption keys when you create Dedicated Kafka clusters in Confluent Cloud. Also known as bring-your-own-key (BYOK) encryption, this option might be preferable for organizations that want to use their own encryption key to encrypt data at rest or require the option to disable access by Confluent. This provides a greater degree of privacy and data integrity, which is often required for compliance by government, health, finance, and many other industries.

Requirements

To use self-managed encryption keys for Confluent Cloud clusters, you must meet the following requirements:

  • Use Dedicated Kafka clusters on Confluent Cloud. To learn more about Dedicated Kafka clusters, see Dedicated clusters.

  • A key management service (KMS) that supports the Key Management Interoperability Protocol (KMIP) version 1.1 or later.

    Confluent Cloud supports the following key management services:

    • Amazon Web Services Key Management Service (AWS KMS)
    • Azure Key Vault
    • Google Cloud Key Management Service (Cloud KMS)

  • Importing key material is not supported.

  • Each Kafka cluster must use a unique encryption key. As a security best practice, you cannot use the same encryption key for multiple clusters.

  • Access to your cluster data is revokable by disabling access to the encryption key, but the cluster is inaccessible only after you delete the cluster.

  • After a cluster is deleted, the associated encryption key is released after five days and can be reused to create a new encrypted cluster. As a security best practice, encryption keys should not be reused for production clusters.

  • The Kafka cluster and KMS must be colocated in the same region.

  • Confluent Cloud supports using self-managed encryption keys on the following cloud service providers:

  • Currently, BYOK applies only to Kafka, and not to any consumer, including Confluent Cloud for Apache Flink®.

List the service quota for self-managed encryption keys

The service quota for self-managed encryption keys (BYOK keys) is applied at the organization scope rather than the Kafka cluster scope because a self-managed encryption key must be created before you create an encrypted Kafka cluster that uses the key. When you use the Confluent CLI and the Scopes (service-quota/v1) API, the service quota for self-managed encryption keys is displayed as Max BYOK keys per organization and the quota code is byok.max_keys.per_org.

The default service quota for self-managed encryption keys is listed in organization service quotas.

To use the Confluent CLI to view the currently applied service quota for self-managed encryption keys in your organization, run the following command:

confluent service-quota list organization

To use the Scopes (service-quota/v1) API to display the currently applied service quota for self-managed encryption keys in your organization, see Organization quotas.

View the Dedicated cluster security settings

After you create a Dedicated cluster that uses a self-managed encryption key (and the cluster is provisioned), you can view the security settings. You can use these cluster security settings to verify the correct encryption key is used.

To view your Dedicated cluster security settings:

  1. Select your Confluent Cloud cluster.
  2. Click Cluster settings and then Security.

Note that anyone authorized to view your Dedicated clusters can view this data.