Best Practices for Using Self-Managed Encryption Keys

Do not reuse encryption keys

By using a unique encryption key for each active Kafka cluster, you minimize the security risks if an encryption key is compromised.

Confluent enforces using unique encryption keys for each active cluster. This requirement, based on security best practices, helps minimize the risk associated with a compromised key. A unique key for each cluster provides the following benefits:

  • Isolation of potential data breaches to one cluster.
  • Granular control of access to the cluster, to help ensure that only authorized users and applications access the cluster and data.
  • Easier to track and audit key usage for the cluster.

Rotate encryption keys on a regular basis

Rotate the encryption keys for each cluster on a regular basis. This practice helps limit the amount of data that can be decrypted if a key is compromised. By frequently rotating keys, you reduce the amount of data that can be decrypted with a compromised key. Many regulatory compliance standards require key rotation on a regular basis.

Minimize key policy modifications

Avoid updating key policies unless absolutely necessary. Key policies should follow a “set it and forget it” approach - configure them correctly during initial setup and leave them unchanged whenever possible. Key policy modifications carry significant risks:

  • Immediate cluster unavailability if permissions are accidentally removed
  • Service disruption that may require Confluent Support intervention
  • Complex recovery procedures that can take 30+ minutes to resolve

Treat key policies as static configurations and avoid changes after initial setup. Modifications should only be made if strictly required by your organization’s compliance or for disaster recovery.

Follow safe key policy update procedures

If key policy updates are absolutely necessary, follow these practices to prevent accidental misconfigurations that could disrupt cluster operations:

  • Always test policy changes in a non-production environment first before applying them to production clusters.
  • Create a backup of your current key policy before making any changes. Store the backup in a secure location for quick recovery if needed.
  • Validate policy syntax and permissions using your cloud provider’s policy validation tools before applying changes.
  • Apply policy changes during planned maintenance windows to minimize impact if issues occur.
  • Monitor cluster health immediately after policy updates to ensure continued access to encrypted data.
  • Never remove Confluent’s required permissions without first deleting the cluster, as this causes immediate service disruption.

Validate key policies before deployment

Before applying key policy changes, use this validation checklist:

AWS KMS Policies

  • Verify that both required permission statements are present:

    • Statement for key usage (kms:Encrypt, kms:Decrypt, kms:ReEncrypt*, kms:GenerateDataKey*, kms:DescribeKey)
    • Statement for grant management (kms:CreateGrant, kms:ListGrants, kms:RevokeGrant)

  • Confirm the Confluent AWS account ARNs are correct and complete.

  • Ensure the "Resource": "*" field is properly set.

  • Validate your policy with AWS IAM Access Analyzer before applying:

    aws accessanalyzer validate-policy \
--policy-document file://policy.json \
--policy-type RESOURCE_POLICY

    Review findings and resolve any ERROR severity issues. For details, see Validate policies with IAM Access Analyzer and the validate-policy CLI reference.

Azure Key Vault Policies

  • Verify that Azure RBAC is enabled for the key vault.
  • Confirm the Confluent Azure service principal has the required roles:
  • Key Vault Crypto Service Encryption User
  • Key Vault Reader
  • Ensure purge protection is enabled.
  • For Enterprise clusters, verify firewall settings allow public access.
  • For Dedicated clusters, verify trusted Microsoft services are allowed.

Google Cloud KMS Policies

  • Verify the custom role includes all required permissions:

    • cloudkms.cryptoKeyVersions.useToDecrypt
    • cloudkms.cryptoKeyVersions.useToEncrypt
    • cloudkms.cryptoKeys.get

  • Confirm the Google Group ID is correctly assigned to the key

  • Test permissions using the Google Cloud CLI command:

    gcloud kms keys get-iam-policy <key-name> \
--location=<location> \
--keyring=<keyring>

Establish key policy recovery procedures

Prepare for potential key policy issues by establishing recovery procedures:

  • Document your current working key policies and store them in a secure, accessible location.
  • Maintain contact information for team members who have administrative access to your cloud key management services.
  • Create step-by-step recovery procedures specific to your cloud provider for quickly restoring working key policies.
  • Test your recovery procedures regularly in non-production environments.
  • Monitor cluster health dashboards and set up alerts for encryption-related issues that could indicate key policy problems.