Terraform Security Best Practices for Confluent Cloud

Protect API keys, credentials, and other secrets in your Terraform configurations for Confluent Cloud by keeping sensitive values out of source files, encrypting state, and integrating with a dedicated secrets manager. The following sections cover securing Terraform state files, managing secrets with external tools, using encrypted backends, configuring access controls, and integrating with secrets management systems.

General security best practices

Follow these security practices when working with Terraform configurations: don’t hardcode secrets, use environment variables or a secrets manager, encrypt and restrict access to state files, rotate credentials, monitor state changes, prefer short-lived secrets, mark sensitive values, and apply fine-grained permissions.

Don’t hardcode sensitive values

Never hardcode sensitive values such as API keys, database passwords, or other secrets into your Terraform configuration files. Hardcoded secrets are credentials embedded directly in Terraform configuration files (.tf) instead of being supplied through variables, environment variables, or a secrets manager. They can be accidentally committed to version control systems, exposing them to unauthorized access.

Bad example

provider "confluent" {
  cloud_api_key    = "ABC123XYZ"           # Don't do this!
  cloud_api_secret = "secret_password123"  # Don't do this!
}

Good example

provider "confluent" {
  cloud_api_key    = var.confluent_cloud_api_key
  cloud_api_secret = var.confluent_cloud_api_secret
}

Use environment variables

The Confluent Terraform provider automatically reads the CONFLUENT_CLOUD_API_KEY and CONFLUENT_CLOUD_API_SECRET environment variables when the cloud_api_key and cloud_api_secret fields are not set in the provider configuration. Store sensitive values as environment variables on the host machine to keep credentials out of your .tf files.

export CONFLUENT_CLOUD_API_KEY="<your-api-key>"
export CONFLUENT_CLOUD_API_SECRET="<your-api-secret>"

Example Terraform configuration:

# Provider automatically reads CONFLUENT_CLOUD_API_KEY and
# CONFLUENT_CLOUD_API_SECRET from the environment
provider "confluent" {
}

resource "confluent_environment" "staging" {
  display_name = "Staging"
}

Alternatively, you can explicitly reference variables that you set as environment variables by using the TF_VAR_ prefix:

export TF_VAR_confluent_cloud_api_key="<your-api-key>"
export TF_VAR_confluent_cloud_api_secret="<your-api-secret>"

variable "confluent_cloud_api_key" {
  type      = string
  sensitive = true
}

variable "confluent_cloud_api_secret" {
  type      = string
  sensitive = true
}

provider "confluent" {
  cloud_api_key    = var.confluent_cloud_api_key
  cloud_api_secret = var.confluent_cloud_api_secret
}

Use secrets managers

A secrets manager, such as HashiCorp Vault, AWS Secrets Manager, Google Cloud Secret Manager, or Azure Key Vault, stores credentials centrally and serves them to Terraform at apply time, so secrets never appear in configuration files. For integration examples, see Integrate Terraform with secrets managers.

Use encrypted, secure backends for state

Terraform state files can contain sensitive information, so store state in a backend that encrypts it at rest and prevents unauthorized modification, rather than in plaintext. Use a backend such as:

AWS S3 with server-side encryption, using Amazon S3 managed keys (SSE-S3) or AWS KMS keys (SSE-KMS)
Google Cloud Storage with encryption
Azure Blob Storage with encryption
Terraform Cloud or Terraform Enterprise with built-in encryption

For local state files, rely on operating system or disk-level encryption, for example, full-disk encryption, to protect the files on disk.

For more information, see Encrypt Terraform state files and Configure a secure Terraform backend.

Limit access to Terraform state

Only allow authorized users to access the Terraform state file. Use appropriate access controls:

Unix file permissions or ACLs for local state files
IAM roles and policies for cloud-based backends, such as S3, Google Cloud Storage (GCS), or Azure Blob Storage
RBAC policies for Terraform Cloud or Terraform Enterprise

Rotate credentials regularly

Rotate sensitive credentials regularly and store them in secure locations, such as a secrets manager or encrypted file storage. Use your cloud provider’s tools to automate rotation where supported.

Monitor Terraform state changes

Monitor Terraform state changes to detect unauthorized access or modifications. Use logging and auditing features available in your backend storage system:

AWS CloudTrail for S3 backends
Google Cloud Audit Logs for GCS backends
Azure Activity Logs for Azure Storage backends

Use short-lived secrets

Short-lived secrets, such as OAuth tokens or temporary security credentials (for example, STS in AWS), expire automatically and reduce the blast radius if they leak. Prefer them over long-lived API keys whenever your authentication flow supports it.

Use fine-grained permissions

Use fine-grained permissions to limit access to specific resources. For Confluent, use:

Confluent Cloud Identity Pools
Service accounts with minimal required permissions

Mark sensitive values with the `sensitive` parameter

Mark sensitive variables and outputs as sensitive to prevent Terraform from displaying them in logs and console output.

variable "api_secret" {
  description = "API secret for Confluent Cloud"
  type        = string
  sensitive   = true
}

output "connection_string" {
  description = "Database connection string"
  value       = local.connection_string
  sensitive   = true
}

For Confluent-specific sensitive connector properties, see Confluent-specific security settings.

Review and revoke unused secrets

Regularly review secret usage. Revoke or delete secrets you no longer need. Implement processes to:

Audit secret usage
Remove secrets associated with decommissioned resources
Clean up orphaned API keys and service accounts

Encrypt Terraform state files

Encrypted state files protect the sensitive resource data, such as passwords, keys, and connection strings, that Terraform persists in its state file. Use a backend’s native encryption, such as SSE-S3 or GCS CMEK, when it is available, and use external encryption, such as openssl with PBKDF2, only for local state files.

Best practices for encrypted state files

Apply these practices when you encrypt Terraform state files outside a backend’s native encryption.

Use strong encryption

Use a strong password-based key derivation function (PBKDF2) to generate encryption keys with:

Sufficient block size of at least 128 bits.
High iteration count: at least 100,000 iterations, or higher to align with current security guidance.

Store encryption keys securely

Store encryption keys in a secure location, such as:

Hardware Security Module (HSM)
Cloud-based key management service, such as AWS KMS, Google Cloud KMS, or Azure Key Vault
Secrets manager, such as HashiCorp Vault or AWS Secrets Manager

Important

Never commit encryption keys to version control.

Example: Encrypt state files

# Generate an encryption key
key=$(openssl rand -base64 32)

# Pull the current Terraform state to a local file
terraform state pull > terraform.tfstate

# Encrypt the local Terraform state file with OpenSSL (AES-256)
openssl enc -aes-256-cbc -salt -pbkdf2 \
  -in terraform.tfstate \
  -out terraform.tfstate.enc \
  -k "$key"

# Decrypt the Terraform state file when needed
openssl enc -d -aes-256-cbc -salt -pbkdf2 \
  -in terraform.tfstate.enc \
  -out terraform.tfstate \
  -k "$key"

Note

This example shows encrypting the output of terraform state pull with an external tool (openssl). When you use a remote backend such as Amazon S3 or Google Cloud Storage, you should rely on the backend’s native encryption features, for example, S3 server-side encryption or GCS CMEK, instead of managing encrypted state files yourself.

Configure a secure Terraform backend

Configure your Terraform backend to use encryption and access controls.

AWS S3 backend with encryption

Enable server-side encryption (SSE) with AWS-managed keys or customer-managed keys:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "prod/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-west-2:123456789012:key/12345678-1234-1234-1234-123456789012"
    dynamodb_table = "terraform-state-lock"
  }
}

Best practices for S3 backends

Limit access to the Terraform state file by using IAM roles and policies. Use a secure bucket policy to restrict access:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::my-terraform-state-bucket",
        "arn:aws:s3:::my-terraform-state-bucket/*"
      ],
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

Enable versioning to maintain state history.
Enable logging to track access.
Use DynamoDB for state locking to prevent concurrent modifications.

Google Cloud Storage backend with encryption

terraform {
  backend "gcs" {
    bucket  = "my-terraform-state-bucket"
    prefix  = "terraform/state"
    encryption_key = "<base64_encoded_encryption_key>"
  }
}

Azure Storage backend with encryption

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "terraformstatestorage"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Selecting a secrets management tool

When choosing a secrets management tool or service, consider:

Integration: Compatibility with your existing infrastructure and CI/CD pipelines
Security features: Encryption, access controls, audit logging, and compliance certifications
Scalability: Ability to handle your current and future secrets volume
Performance: Latency and throughput for secrets retrieval
Cost: Pricing model and total cost of ownership
Support: Vendor support, documentation quality, and community resources
Multi-cloud: Support for multiple cloud providers if you use multi-cloud deployments

Integrate Terraform with secrets managers

Integrate Terraform with secrets management systems to retrieve sensitive values at runtime.

HashiCorp Vault

Integrate HashiCorp Vault with the Confluent Terraform provider to retrieve Confluent Cloud API credentials at apply time instead of storing them in configuration files. Configure the Vault provider, read the credentials with vault_generic_secret, and pass them to the confluent provider.

provider "vault" {
  address = "https://vault.example.com:8200"
}

data "vault_generic_secret" "confluent_api_key" {
  path = "secret/confluent/api-key"
}

provider "confluent" {
  cloud_api_key    = data.vault_generic_secret.confluent_api_key.data["key"]
  cloud_api_secret = data.vault_generic_secret.confluent_api_key.data["secret"]
}

In this example, the Confluent Cloud API key and secret are stored at the Vault path secret/confluent/api-key and read at apply time, so no credentials are stored in the configuration.

For more information, see HashiCorp Vault |tf| integration.

AWS Secrets Manager

AWS Secrets Manager provides native integration with Terraform for secure storage and automatic rotation of secrets.

# Read a JSON-formatted secret from AWS Secrets Manager
data "aws_secretsmanager_secret_version" "confluent_cloud" {
  secret_id = "confluent/cloud/api-credentials"
}

# Secret string is expected to be JSON, for example:
# {
#   "api_key":    "....",
#   "api_secret": "...."
# }
locals {
  confluent_creds = jsondecode(data.aws_secretsmanager_secret_version.confluent_cloud.secret_string)
}

provider "confluent" {
  cloud_api_key    = local.confluent_creds.api_key
  cloud_api_secret = local.confluent_creds.api_secret
}

In this example, the Confluent Cloud API key and secret are stored securely in AWS Secrets Manager as a JSON object. Terraform reads and decodes the secret at apply time, so the credentials are not hardcoded in your configuration.

AWS Systems Manager Parameter Store

Use AWS Systems Manager Parameter Store for simpler secret storage needs:

data "aws_ssm_parameter" "confluent_api_key" {
  name = "/confluent/api-key"
}

data "aws_ssm_parameter" "confluent_api_secret" {
  name            = "/confluent/api-secret"
  with_decryption = true
}

provider "confluent" {
  cloud_api_key    = data.aws_ssm_parameter.confluent_api_key.value
  cloud_api_secret = data.aws_ssm_parameter.confluent_api_secret.value
}

Create parameters as SecureString type:

variable "confluent_api_secret" {
  type        = string
  description = "Confluent Cloud API secret"
}

resource "aws_ssm_parameter" "confluent_api_secret" {
  name  = "/confluent/api-secret"
  type  = "SecureString"
  value = var.confluent_api_secret
}

Google Cloud Secret Manager

Google Cloud Secret Manager integrates with Terraform for managing secrets at scale:

data "google_secret_manager_secret_version" "confluent_api_key" {
  secret = "confluent-api-key"
}

provider "confluent" {
  cloud_api_key    = jsondecode(data.google_secret_manager_secret_version.confluent_api_key.secret_data)["key"]
  cloud_api_secret = jsondecode(data.google_secret_manager_secret_version.confluent_api_key.secret_data)["secret"]
}

Azure Key Vault

Azure Key Vault provides secure storage for keys, certificates, and secrets:

data "azurerm_key_vault" "example" {
  name                = "my-key-vault"
  resource_group_name = "my-resource-group"
}

data "azurerm_key_vault_secret" "confluent_api_key" {
  name         = "confluent-api-key"
  key_vault_id = data.azurerm_key_vault.example.id
}

data "azurerm_key_vault_secret" "confluent_api_secret" {
  name         = "confluent-api-secret"
  key_vault_id = data.azurerm_key_vault.example.id
}

provider "confluent" {
  cloud_api_key    = data.azurerm_key_vault_secret.confluent_api_key.value
  cloud_api_secret = data.azurerm_key_vault_secret.confluent_api_secret.value
}

Terraform Cloud

Terraform Cloud provides built-in secrets management with environment variables and workspace-level variable encryption:

Store sensitive variables in Terraform Cloud workspace settings
Mark variables as “Sensitive” to prevent display in logs and UI
Use variable sets to share secrets across multiple workspaces

For more information, see Terraform Cloud workspace variables documentation.

Manage secrets with Sops

Sops is an open-source tool that enables you to manage encrypted secrets in version control. You can use Sops together with a Terraform provider or a decrypt-at-runtime pattern to supply secrets securely to Terraform without storing them in plaintext.

Confluent-specific security settings

The following settings apply specifically to Confluent Cloud resources managed with Terraform: sensitive connector properties, identity pools, and service accounts.

Sensitive connector properties

When configuring custom connector plugins, mark all sensitive properties appropriately. For more information, see the Confluent |tf| provider documentation.

Note

The sensitive_config_properties parameter should include all connector configuration properties that must be hidden after a user enters the value, such as passwords, keys, and tokens. Marking a property as sensitive ensures proper handling within Confluent infrastructure, including masking in exception logs and encrypting values in the underlying data store.

Only add connector-specific sensitive properties. Do not include Kafka keys, passwords, or service account information.

Identity pools for fine-grained access

Use identity pools to implement fine-grained, role-based access control for Terraform-managed resources:

resource "confluent_identity_pool" "terraform_pool" {
  display_name  = "Terraform Automation Pool"
  description   = "Identity pool for Terraform CI/CD pipeline"
  identity_claim = "claims.sub"
  filter         = "claims.sub == 'terraform-automation'"
}

resource "confluent_identity_provider" "example" {
  display_name = "Terraform Identity Provider"
  description  = "Identity provider for Terraform automation"
  issuer       = "https://your-identity-provider.com"
  jwks_uri     = "https://your-identity-provider.com/.well-known/jwks.json"
}

Service accounts with minimal permissions

Create service accounts with only the permissions needed for Terraform operations:

resource "confluent_service_account" "terraform_sa" {
  display_name = "terraform-automation"
  description  = "Service account for Terraform automation"
}

resource "confluent_role_binding" "terraform_env_admin" {
  principal   = "User:${confluent_service_account.terraform_sa.id}"
  role_name   = "EnvironmentAdmin"
  crn_pattern = confluent_environment.staging.resource_name
}

Terraform Security Best Practices for Confluent Cloud

General security best practices

Don’t hardcode sensitive values

Use environment variables

Use secrets managers

Use encrypted, secure backends for state

Limit access to Terraform state

Rotate credentials regularly

Monitor Terraform state changes

Use short-lived secrets

Use fine-grained permissions

Mark sensitive values with the sensitive parameter

Review and revoke unused secrets

Encrypt Terraform state files

Best practices for encrypted state files

Use strong encryption

Store encryption keys securely

Example: Encrypt state files

Configure a secure Terraform backend

AWS S3 backend with encryption

Best practices for S3 backends

Google Cloud Storage backend with encryption

Azure Storage backend with encryption

Selecting a secrets management tool

Integrate Terraform with secrets managers

HashiCorp Vault

AWS Secrets Manager

AWS Systems Manager Parameter Store

Google Cloud Secret Manager

Azure Key Vault

Terraform Cloud

Manage secrets with Sops

Confluent-specific security settings

Sensitive connector properties

Identity pools for fine-grained access

Service accounts with minimal permissions

Related content

Mark sensitive values with the `sensitive` parameter