Terraform Security Best Practices for Confluent Cloud

When using Terraform to manage your Confluent Cloud infrastructure, follow these security best practices to protect sensitive information like API keys, credentials, and secrets.

This topic provides guidance on:

Securing Terraform state files
Managing secrets with external tools
Using encrypted backends
Configuring access controls
Integrating with secrets management systems

General security best practices

Follow these security practices when working with Terraform configurations

Don’t hardcode sensitive values

Never hardcode sensitive values such as API keys, database passwords, or other secrets into your Terraform configuration files. Hardcoded secrets can be accidentally committed to version control systems, exposing them to unauthorized access.

Bad example

provider "confluent" {
  cloud_api_key    = "ABC123XYZ"           # Don't do this!
  cloud_api_secret = "secret_password123"  # Don't do this!
}

Good example

provider "confluent" {
  cloud_api_key    = var.confluent_cloud_api_key
  cloud_api_secret = var.confluent_cloud_api_secret
}

Use environment variables

Store sensitive values as environment variables on the host machine and reference them in your Terraform configuration.

export CONFLUENT_CLOUD_API_KEY="<your-api-key>"
export CONFLUENT_CLOUD_API_SECRET="<your-api-secret>"

The Confluent Terraform provider automatically reads these environment variables when the cloud_api_key and cloud_api_secret fields are not explicitly set in the provider configuration.

Example Terraform configuration:

# Provider automatically reads CONFLUENT_CLOUD_API_KEY and
# CONFLUENT_CLOUD_API_SECRET from the environment
provider "confluent" {
}

resource "confluent_environment" "staging" {
  display_name = "Staging"
}

Alternatively, you can explicitly reference variables that are set with environment variables using the TF_VAR_ prefix:

export TF_VAR_confluent_cloud_api_key="<your-api-key>"
export TF_VAR_confluent_cloud_api_secret="<your-api-secret>"

variable "confluent_cloud_api_key" {
  type      = string
  sensitive = true
}

variable "confluent_cloud_api_secret" {
  type      = string
  sensitive = true
}

provider "confluent" {
  cloud_api_key    = var.confluent_cloud_api_key
  cloud_api_secret = var.confluent_cloud_api_secret
}

Use secrets managers

Consider using a secrets manager like HashiCorp Vault, AWS Secrets Manager, or Google Cloud Secret Manager to securely store and retrieve sensitive values. For integration examples, see Secrets management integration.

Use encrypted state files

Terraform state files can contain sensitive information. Ensure that your state is protected at rest by using encrypted backends such as Terraform Cloud or Terraform Enterprise, or cloud storage services with encryption enabled, for example, AWS S3 with SSE-S3 or SSE-KMS, Google Cloud Storage with encryption, or Azure Blob Storage with encryption. For local state files, rely on operating system or disk-level encryption, for example, full-disk encryption, to protect the files on disk. For more information, see Encrypted Terraform state files.

Limit access to Terraform state

Only allow authorized users to access the Terraform state file. Use appropriate access controls:

Unix file permissions or ACLs for local state files
IAM roles and policies for cloud-based backends, such as S3, GCS, or Azure Blob Storage
RBAC policies for Terraform Cloud or Terraform Enterprise

Use secure backends

Use a secure backend for your Terraform state file to prevent unauthorized modifications to the state file, such as:

S3 bucket with server-side encryption (SSE)
Google Cloud Storage with encryption
Azure Blob Storage with encryption
Terraform Cloud with built-in encryption

For configuration examples, see Secure backend configuration.

Rotate credentials regularly

Rotate sensitive credentials regularly and store them in secure locations, such as a secrets manager or encrypted file storage. Set up automated rotation when possible using your cloud provider’s tools.

Monitor Terraform state changes

Monitor Terraform state changes to detect unauthorized access or modifications. Use logging and auditing features available in your backend storage system:

AWS CloudTrail for S3 backends
Google Cloud Audit Logs for GCS backends
Azure Activity Logs for Azure Storage backends

Use short-lived secrets

Use short-lived credentials such as OAuth tokens or temporary security credentials, such as STS in AWS, instead of long-lived API keys when possible.

Use fine-grained permissions

Use fine-grained permissions to limit access to specific resources. For Confluent, use:

Confluent Cloud Identity Pools
Service accounts with minimal required permissions

Use the `sensitive` parameter

Mark sensitive variables and outputs as sensitive to prevent Terraform from displaying them in logs and console output.

variable "api_secret" {
  description = "API secret for Confluent Cloud"
  type        = string
  sensitive   = true
}

output "connection_string" {
  description = "Database connection string"
  value       = local.connection_string
  sensitive   = true
}

Review and deauthorize unused secrets

Regularly review when secrets are used and deauthorize or delete unused secrets. Implement processes to:

Audit secret usage
Remove secrets associated with decommissioned resources
Clean up orphaned API keys and service accounts

Encrypted Terraform state files

Terraform state files can contain sensitive data, including resource attributes that may include passwords, keys, and other secrets. Encrypting state files protects this information.

Best practices for encrypted state files

This section provides best practices for encrypted state files.

Use strong encryption

Use a strong password-based key derivation function (PBKDF2) to generate encryption keys with:

Sufficient block size of at least 128 bits
High iteration count, for example, at least 100,000 iterations, or higher in line with current security guidance

Store encryption keys securely

Store encryption keys in a secure location, such as:

Hardware Security Module (HSM)
Cloud-based key management service, such as AWS KMS, GCP KMS, or Azure Key Vault
Secrets manager, such as HashiCorp Vault or AWS Secrets Manager

Never commit encryption keys to version control.

Example: Encrypt state files

# Generate an encryption key
key=$(openssl rand -base64 32)

# Pull the current |tf| state to a local file
terraform state pull > terraform.tfstate

# Encrypt the local |tf| state file with OpenSSL (AES-256)
openssl enc -aes-256-cbc -salt -pbkdf2 \
  -in terraform.tfstate \
  -out terraform.tfstate.enc \
  -k "$key"

# Decrypt the |tf| state file when needed
openssl enc -d -aes-256-cbc -salt -pbkdf2 \
  -in terraform.tfstate.enc \
  -out terraform.tfstate \
  -k "$key"

Note

This example shows encrypting the output of terraform state pull with an external tool (openssl). When you use a remote backend such as Amazon S3 or Google Cloud Storage, you should rely on the backend’s native encryption features, for example, S3 server-side encryption or GCS CMEK, instead of managing encrypted state files yourself.

Secure backend configuration

Configure your Terraform backend to use encryption and access controls.

AWS S3 backend with encryption

Enable server-side encryption (SSE) with AWS-managed keys or customer-managed keys:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "prod/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-west-2:123456789012:key/12345678-1234-1234-1234-123456789012"
    dynamodb_table = "terraform-state-lock"
  }
}

Best practices for S3 backends

Limit access to the Terraform state file by using IAM roles and policies. Use a secure bucket policy to restrict access:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-terraform-state-bucket",
        "arn:aws:s3:::my-terraform-state-bucket/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

Enable versioning to maintain state history
Enable logging to track access
Use DynamoDB for state locking to prevent concurrent modifications

Google Cloud Storage backend

terraform {
  backend "gcs" {
    bucket  = "my-terraform-state-bucket"
    prefix  = "terraform/state"
    encryption_key = "your-base64-encoded-encryption-key"
  }
}

Azure Storage backend

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "terraformstatestorage"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Secrets management integration

Integrate Terraform with secrets management systems to retrieve sensitive values at runtime.

HashiCorp Vault

HashiCorp Vault provides secure secrets storage with dynamic secrets, encryption as a service, and fine-grained access controls.

provider "vault" {
  address = "https://vault.example.com:8200"
}

data "vault_generic_secret" "confluent_api_key" {
  path = "secret/confluent/api-key"
}

provider "confluent" {
  cloud_api_key    = data.vault_generic_secret.confluent_api_key.data["key"]
  cloud_api_secret = data.vault_generic_secret.confluent_api_key.data["secret"]
}

For more information, see HashiCorp Vault |tf| integration.

AWS Secrets Manager

AWS Secrets Manager provides native integration with Terraform for secure storage and automatic rotation of secrets.

# Read a JSON-formatted secret from |aws| Secrets Manager
data "aws_secretsmanager_secret_version" "confluent_cloud" {
  secret_id = "confluent/cloud/api-credentials"
}

# Secret string is expected to be JSON, for example:
# {
#   "api_key":    "....",
#   "api_secret": "...."
# }
locals {
  confluent_creds = jsondecode(data.aws_secretsmanager_secret_version.confluent_cloud.secret_string)
}

provider "confluent" {
  cloud_api_key    = local.confluent_creds.api_key
  cloud_api_secret = local.confluent_creds.api_secret
}

In this example, the Confluent Cloud API key and secret are stored securely in AWS Secrets Manager as a JSON object. Terraform reads and decodes the secret at apply time, so the credentials are not hardcoded in your configuration.

AWS Systems Manager Parameter Store

Use AWS Systems Manager Parameter Store for simpler secret storage needs:

data "aws_ssm_parameter" "confluent_api_key" {
  name = "/confluent/api-key"
}

data "aws_ssm_parameter" "confluent_api_secret" {
  name            = "/confluent/api-secret"
  with_decryption = true
}

provider "confluent" {
  cloud_api_key    = data.aws_ssm_parameter.confluent_api_key.value
  cloud_api_secret = data.aws_ssm_parameter.confluent_api_secret.value
}

Create parameters as SecureString type:

variable "confluent_api_secret" {
  type        = string
  description = "Confluent Cloud API secret"
}

resource "aws_ssm_parameter" "confluent_api_secret" {
  name  = "/confluent/api-secret"
  type  = "SecureString"
  value = var.confluent_api_secret
}

Google Cloud Secret Manager

Google Cloud Secret Manager integrates with Terraform for managing secrets at scale:

data "google_secret_manager_secret_version" "confluent_api_key" {
  secret = "confluent-api-key"
}

provider "confluent" {
  cloud_api_key    = jsondecode(data.google_secret_manager_secret_version.confluent_api_key.secret_data)["key"]
  cloud_api_secret = jsondecode(data.google_secret_manager_secret_version.confluent_api_key.secret_data)["secret"]
}

Azure Key Vault

Azure Key Vault provides secure storage for keys, certificates, and secrets:

data "azurerm_key_vault" "example" {
  name                = "mykeyvault"
  resource_group_name = "my-resource-group"
}

data "azurerm_key_vault_secret" "confluent_api_key" {
  name         = "confluent-api-key"
  key_vault_id = data.azurerm_key_vault.example.id
}

data "azurerm_key_vault_secret" "confluent_api_secret" {
  name         = "confluent-api-secret"
  key_vault_id = data.azurerm_key_vault.example.id
}

provider "confluent" {
  cloud_api_key    = data.azurerm_key_vault_secret.confluent_api_key.value
  cloud_api_secret = data.azurerm_key_vault_secret.confluent_api_secret.value
}

Terraform Cloud

Terraform Cloud provides built-in secrets management with environment variables and workspace-level variable encryption:

Store sensitive variables in Terraform Cloud workspace settings
Mark variables as “Sensitive” to prevent display in logs and UI
Use variable sets to share secrets across multiple workspaces

For more information, see |tf| Cloud documentation.

Open-source alternatives

Sops is an open-source tool that enables you to manage encrypted secrets in version control. It can be used together with a Terraform provider or decrypt-at-runtime pattern to supply secrets securely to Terraform without storing them in plaintext.

Confluent-specific security considerations

This section provides security settings that are specific to Confluent.

Sensitive connector properties

When configuring custom connector plugins, mark all sensitive properties appropriately. For more information, see the Confluent |tf| provider documentation.

Note

The sensitive_config_properties parameter should include all connector configuration properties that must be hidden after a user enters the value, such as passwords, keys, and tokens. Marking a property as sensitive ensures proper handling within Confluent infrastructure, including masking in exception logs and encrypting values in the underlying data store.

Only add connector-specific sensitive properties. Do not include Kafka keys, passwords, or service account information.

Identity Pools for fine-grained access

Use Identity Pools to implement fine-grained, role-based access control for Terraform-managed resources:

resource "confluent_identity_pool" "terraform_pool" {
  display_name  = "|tf| Automation Pool"
  description   = "Identity pool for |tf| CI/CD pipeline"
  identity_claim = "claims.sub"
  filter         = "claims.sub == 'terraform-automation'"
}

resource "confluent_identity_provider" "example" {
  display_name = "|tf| Identity Provider"
  description  = "Identity provider for |tf| automation"
  issuer       = "https://your-identity-provider.com"
  jwks_uri     = "https://your-identity-provider.com/.well-known/jwks.json"
}

Service accounts with minimal permissions

Create service accounts with only the permissions needed for Terraform operations:

resource "confluent_service_account" "terraform_sa" {
  display_name = "terraform-automation"
  description  = "Service account for |tf| automation"
}

resource "confluent_role_binding" "terraform_env_admin" {
  principal   = "User:${confluent_service_account.terraform_sa.id}"
  role_name   = "EnvironmentAdmin"
  crn_pattern = confluent_environment.staging.resource_name
}

Selecting a secrets management tool

When choosing a secrets management tool or service, consider:

Integration: Compatibility with your existing infrastructure and CI/CD pipelines
Security features: Encryption, access controls, audit logging, and compliance certifications
Scalability: Ability to handle your current and future secrets volume
Performance: Latency and throughput for secrets retrieval
Cost: Pricing model and total cost of ownership
Support: Vendor support, documentation quality, and community resources
Multi-cloud: Support for multiple cloud providers if you use multi-cloud deployments

Terraform Security Best Practices for Confluent Cloud

General security best practices

Don’t hardcode sensitive values

Use environment variables

Use secrets managers

Use encrypted state files

Limit access to Terraform state

Use secure backends

Rotate credentials regularly

Monitor Terraform state changes

Use short-lived secrets

Use fine-grained permissions

Use the sensitive parameter

Review and deauthorize unused secrets

Encrypted Terraform state files

Best practices for encrypted state files

Use strong encryption

Store encryption keys securely

Example: Encrypt state files

Secure backend configuration

AWS S3 backend with encryption

Best practices for S3 backends

Google Cloud Storage backend

Azure Storage backend

Secrets management integration

HashiCorp Vault

AWS Secrets Manager

AWS Systems Manager Parameter Store

Google Cloud Secret Manager

Azure Key Vault

Terraform Cloud

Open-source alternatives

Confluent-specific security considerations

Sensitive connector properties

Identity Pools for fine-grained access

Service accounts with minimal permissions

Selecting a secrets management tool

Related content

Use the `sensitive` parameter