Create a Kafka Cluster on Confluent Cloud from a Template Using Terraform
What is Terraform?
HashiCorp Terraform is an open source infrastructure-as-code tool that lets you build, change, and version your cloud data infrastructure in a safe, efficient way. You program Terraform with human-readable configuration files that you can version, reuse, share, and deploy in your CI/CD pipelines.
Get Started for Free
Sign up for a Confluent Cloud trial and get $400 of free credit.
Resources you can manage
Why Terraform and Kafka?
Use the Confluent Terraform provider to deploy and manage Confluent Cloud infrastructure. The Confluent Terraform provider automates the workflow for managing environments, Apache Kafka® clusters, Kafka topics, and other resources in Confluent Cloud.
These are some of the benefits you get with the Confluent Terraform provider:
Human Readable Configuration: Define infrastructure resources declaratively in human-readable configuration files.
Manage Critical Confluent Cloud Resources: Manage API keys, environments, Kafka clusters, topics, ACLs, RBAC, Private Networking, and more.
Consistent Deployability: Provision and manage your infrastructure safely and efficiently throughout its lifecycle. Package reusable modules to provision multi-cloud resources.
Multi-Cloud With Ease: Deploy Confluent Cloud seamlessly across cloud providers.
Scale Quickly: Provision complex and dependent infrastructure quickly.
Industry Standard: Enable industry standard GitOps and infrastructure-as-code practices.
Confluent Terraform tutorial
The following tutorials show how to get started with the Confluent Terraform provider:
Using the Confluent Terraform provider
The following example shows a simple Terraform configuration file that provisions a Confluent Cloud environment, a basic Kafka cluster, and a service account.
# Configure the Confluent Provider
terraform {
required_providers {
confluent = {
source = "confluentinc/confluent"
version = "|terraform_provider_version|"
}
}
}
provider "confluent" {
cloud_api_key = var.confluent_cloud_api_key # optionally use CONFLUENT_CLOUD_API_KEY env var
cloud_api_secret = var.confluent_cloud_api_secret # optionally use CONFLUENT_CLOUD_API_SECRET env var
}
resource "confluent_environment" "development" {
display_name = "Development"
lifecycle {
prevent_destroy = true
}
}
resource "confluent_kafka_cluster" "basic" {
display_name = "basic_kafka_cluster"
availability = "SINGLE_ZONE"
cloud = "AWS"
region = "us-east-2"
basic {}
environment {
id = confluent_environment.development.id
}
lifecycle {
prevent_destroy = true
}
}
resource "confluent_service_account" "app-manager" {
display_name = "app-manager"
description = "Service account to manage Kafka cluster"
}
# Create more resources ...
You must provide appropriate Confluent Cloud and Kafka cluster credentials to use the provider.
For more information about the resources that you can interact with, see the Confluent Terraform Provider documentation in the Terraform registry.
Sample project
For realistic examples that create multiple Confluent Cloud resources, see the Sample Project for Confluent Terraform Provider. The sample project shows configurations that provision Confluent Cloud resources like these:
Basic, Standard, Enterprise, Dedicated, and Freight Kafka clusters
PrivateLink connections
RBAC and ACLs for access control
Source and sink connectors
VPC Peering connections
Source code
For the provider source code, see the Confluent Terraform Provider repository on GitHub.
For sample configuration files, see the examples directory in the source repo.
Resource Importer
The Resource Importer
enables importing your existing Confluent Cloud resources to Terraform Configuration
(main.tf) and Terraform State (terraform.tfstate) files.
Resources you can manage
You can provision the following Confluent Cloud resources and get data from these data sources in your Terraform configuration files.
Confluent
Resource |
Data Source |
|---|---|
Connect
Resource |
Data Source |
|---|---|
– |
|
– |
|
– |
Flink
Resource |
Data Source |
|---|---|
– |
|
– |
Kafka cluster
Resource |
Data Source |
|---|---|
– |
|
– |
|
– |
|
ksqlDB
Resource |
Data Source |
|---|---|
Metadata
Resource |
Data Source |
|---|---|
Network
Resource |
Data Source |
|---|---|
– |
|
– |
|
Schema management and Stream Governance
Resource |
Data Source |
|---|---|
– |
|
– |
|
– |
|
– |
|
– |
|
Security, access control, and identity
Resource |
Data Source |
|---|---|
– |
|
– |
|
– |
|
– |
|
– |
|
– |
Tableflow
Resource |
Data Source |
|---|---|