AWS PrivateLink for Enterprise Clusters¶
Confluent Cloud supports private connectivity for Enterprise Kafka clusters using PrivateLink Attachment. When you use PrivateLink Attachment, your Enterprise cluster is only accessible from tenant-specific private endpoints. Public access is blocked with PrivateLink Attachment.
Confluent Cloud uses the following private networking resources for Enterprise clusters. These resources are regional and do not have a mapping to specific availability zones.
- PrivateLink Attachment
The PrivateLink Attachment (
PrivateLinkAttachment
) resource represents a reservation to establish a PrivateLink connection from your VPC to regional services in a Confluent Cloud environment.A PrivateLink Attachment belongs to an Environment in the Confluent resource hierarchy.
- PrivateLink Attachment Connection
- A PrivateLink Attachment Connection (
PrivateLinkAttachmentConnection
) is a registration of VPC interface endpoints that are allowed to connect to Confluent Cloud. A PrivateLink Attachment Connection belongs to a specific PrivateLink Attachment.
You can use Confluent Cloud UI, Confluent REST API, or Terraform to establish a PrivateLink connectivity for an Enterprise cluster.
The high-level workflow is:
In AWS, create a VPC Endpoint to the PrivateLinkAttachment service.
In Confluent Cloud, create a PrivateLinkAttachmentConnection.
Create a Kafka client in your VPC using the bootstrap endpoint of your Enterprise Kafka cluster. This Kafka client can live in Virtual Machine or similar compute infrastructure.
Validate produce/consume traffic is successful.
Once you create a PrivateLinkAttachment resource and establish a PrivateLink, you can securely send and receive traffic through the PrivateLink between your VPC and Confluent Cloud.
Requirements and considerations¶
- You can connect to only one environment from a single VPC or from an on-prem network.
- The following regions are supported:
us-east-1
us-east-2
us-west-2
eu-west-1
eu-central-1
ap-south-1
ap-southeast-2
ap-southeast-1
af-south-1
- Fully-managed Confluent Cloud connectors can connect to sources or sinks using public IP addresses. Sources or sinks in the customer network with private IP addresses are not supported. An exception to this is the Amazon S3 Sink connector which can connect to an Amazon S3 bucket from a private network.
- Confluent Cloud Console components, like topic management, require additional configuration to function as they use cluster endpoints. To use all features of the Confluent Cloud Console with AWS PrivateLink, see Use Confluent Cloud with Private Networking.
Create a PrivateLink Attachment¶
When you create a PrivateLink Attachment in an environment and in a region, the PrivateLink Attachment resource provides connectivity to all Enterprise Kafka clusters within the environment for the specific cloud region.
- In the Confluent Cloud Console, select an environment for the PrivateLink Attachment.
- In the Network management tab in the environment, click Add network configuration.
- Select For Enterprise Clusters and click Continue.
- Select AWS, select Region, and click Continue.
- Provide the PrivateLink Attachment name in the Network name field and click Add network configuration.
The PrivateLink Attachment will be provisioned and move to the Waiting for
connection
state.
A PrivateLink Attachment can be in one of the following states:
WAITING FOR CONNECTION
: The PrivateLink Attachment is waiting for a connection to be created.READY
: AWS PrivateLink connectivity is ready to be used.EXPIRED
: A valid connection has not been provisioned within the allotted time. A new PrivateLink Attachment must be provisioned.
Send a request to create a PrivateLink Attachment resource:
REST request
POST https://api.confluent.cloud/networking/v1/private-link-attachments
REST request body
{ "spec": { "display_name": "<name of this resource>", "cloud": "<provider type>", "region": "<region>", "environment": { "id": "<environement id>" } } }
In the REST response,
status.phase
should be set toPROVISIONING
because a VPC Endpoint Service has not yet been allocated.Check the status of the new PrivateLink Attachment:
REST request
GET https://api.confluent.cloud/networking/v1/private-link-attachments/<platt-id>
REST response example
{ "status": { "phase": "WAITING_FOR_CONNECTIONS", "error_code": "", "error_message": "", "cloud": { "kind": "AwsPrivateLinkAttachmentStatus", "vpc_endpoint_service_id": "com.amazonaws.vpce.us-east-1.vpce-svc-123abcc1298abc123", } } }
status.phase
isWAITING_FOR_CONNECTIONS
because a VPC Endpoint has not been associated with this PrivateLink Attachment resource yet.The
status.cloud
object has information about thevpc_endpoint_service
that you must connect your PrivateLink Attachment endpoint to.
Create a VPC Endpoint¶
In AWS, create an endpoint that is associated with the PrivateLink Service ID of the PrivateLink Attachment you created in Create a PrivateLink Attachment.
- In the AWS Management Console, go to the the VPC dashboard.
- Verify that DNS hostnames and DNS resolution are enabled.
- In the navigation menu under VIRTUAL PRIVATE CLOUD, click Your VPCs.
- Select your VPC and click Edit VPC settings.
- Under DNS settings, verify that Enable DNS resolution and Enable DNS hostnames are selected and then click Save.
- In the navigation menu under VIRTUAL PRIVATE CLOUD, click Endpoints, and click Create endpoint.
- In Name tag, specify the name of the endpoint.
- In Service category, select Other endpoint services.
- In the Service name field, specify the PrivateLink Service ID of the PrivateLink Attachment you created in Create a PrivateLink Attachment.
- In the VPC field, specify the ID of this VPC.
- Uncheck the Enable DNS name setting under Additional settings (only appearing after the VPC is selected).
- In Subnets, select the subnets in which to create an endpoint network interface.
- Select or create a security group for the VPC Endpoint.
- Add three inbound rules for each of ports
80
,443
, and9092
from your desired source (your VPC CIDR). The Protocol should beTCP
for all three rules. - Port
80
is not required, but is available as a redirect only tohttps/443
, if desired.
- Add three inbound rules for each of ports
- Click Create endpoint.
- Note that the VPC endpoint ID created.
aws ec2 create-vpc-endpoint --vpc-id <id of this VPC> \
--service-name <privatelink attachment service id> \
--subnet-ids <subnet IDs for the endpoint> \
--region <region to use> \
--private-dns-enabled false \
--vpc-endpoint-type Interface
Note that the VPC endpoint ID is created.
For example, using the information in
status.cloud.vpc_endpoint_service_i
in the PrivateLink Attachment status:
aws ec2 create-vpc-endpoint --vpc-id vpc-097799943f9fc059d \
--service-name com.amazonaws.vpce.us-east-1.vpce-svc-123abcc1298abc123 \
--subnet-ids subnet-7b16de0c \
--region us-east-1 \
--private-dns-enabled false \
--vpc-endpoint-type Interface
Create a PrivateLink Attachment Connection¶
Create a PrivateLink Attachment Connection (PrivateLinkAttachmentConnection) resource in Confluent Cloud. A PrivateLink Attachment Connection represents a VPC Interface Endpoint in your VPC.
The name of the VPC Endpoint Service is not required. Confluent will check which VPC Endpoint Service is associated with the PrivateLink Attachment that has a pending VPC Endpoint with the given ID.
In Confluent Cloud Console, in the Network management tab in the environment, click the PrivateLink Attachment you want to add a connection to.
Make sure the PrivateLink Attachment is in the correct region of the VPC Private Endpoint.
Click + Add connection.
Specify the connection name and the Private Endpoint ID.
The Private Endpoint ID is the id of the VPC Endpoint that was created in Create a VPC Endpoint.
Click Finish.
The PrivateLink Attachment and PrivateLink Attachment Connection should now move to the READY
state once the VPC
Endpoint connection is accepted.
Send a request to create a PrivateLink Attachment Connection resource:
REST request
POST https://api.confluent.cloud/networking/v1/private-link-attachment-connections
REST request body
{ "spec": { "display_name": "<PrivateLinkAttachmentEndpoint name>", "cloud": { "kind": "AwsPrivateLinkAttachmentConnection", "vpc_endpoint_id": "<VPC Private Endpoint ID>", }, "environment": { "id": "<Environment ID>", }, "private_link_attachment": { "id": "<PrivateLinkAttachment>", } } }
REST response example
{ "api_version": "networking/v1", "kind": "PrivateLinkAttachmentConnection", "id": "plattc-xyzuvw", "status": { "phase": "PROVISIONING", "error_code": "", "error_message": "", } }
status.phase
isPROVISIONING
because a VPC Endpoint connection has not yet been accepted.Check the status of the new PrivateLink Attachment Connection:
REST request
GET https://api.confluent.cloud/networking/v1/private-link-attachment-connections/<platt-id>
REST response example
{ "api_version": "networking/v1", "kind": "PrivateLinkAttachmentConnection", "id": "plattc-xyzuvw", "status": { "phase": "READY", "error_code": "", "error_message": "", "cloud": { "kind": "AwsPrivateLinkAttachmentConnectionStatus", "phase": "READY", "vpc_endpoint_service": "com.amazonaws.vpce.us-east-1.vpce-svc-123abcc1298abc123", "vpc_endpoint_id": "vpce-bbbbbb2222222333" } } }
status.phase
isREADY
because a VPC Endpoint connection has been accepted.status.cloud
has an object of kindAwsPrivateLinkConnectionStatus
.vpc_endpoint_id
reflects the valuevpce-bbbbbb2222222333
that the customer registered.
Set up DNS resolution¶
Set up a Route53 Private Hosted Zone in your AWS VPC for DNS resolution.
In Confluent Cloud, verify that the status of the PrivateLink Attachment Connection is
READY
.In Confluent Cloud, browse to Cluster Overview > Networking of the Enterprise cluster, and click the PrivateLink Attachment to get the DNS domain value of Confluent Cloud.
The value is in the
<region>.aws.private.confluent.cloud
pattern.In the AWS Route 53 console, create a Route53 Private Hosted Zone:
- Specify the following values:
- Domain name: Confluent Cloud DNS domain value from the previous step
- Type:
Private hosted zone
- VPC ID: VPC ID where you added the VPC Endpoint
- Click Create hosted zone to associate the Private Hosted Zone with your VPC.
- Specify the following values:
Create a DNS record for the Hosted Zone you created above.
- Click Create Record from within the previously created Hosted Zone.
- Specify the following values:
- Record name:
*
- Record type:
CNAME
- Value: DNS Name of your VPC Interface Endpoint
- Record name:
- Click Create Record.
Connectivity scenarios¶
Below are a few connectivity scenarios that are supported for Enterprise clusters in Confluent Cloud.
Scenario: Access one environment from one VPC¶

The following resources are configured:
PLATT-prod
as a PrivateLink Attachment for accessing Kafka clusters in the env-prod environment.PLATTC-123
as a PrivateLink Attachment Connection for thevpce-1
VPC endpoint in VPC-1ProdApp
as a Kafka client bootstrapped withlkc-123.us-west-2.aws.private.confluent.cloud
phz-1
as a Route53 Private Hosted Zone with the regional wildcard*.us-west-2.aws.private.confluent.cloud
The following steps are performed:
ProdApp
attempts to accesslkc-123
in the env-prod environment. A DNS query forlkc-123.us-west-2.aws.private.confluent.cloud
resolves againstphz-1
and returnsvpce-1
.- Application sends traffic to
vpce-1
. vpce-1
forwards traffic toPLATT-prod
, andlkc-123
can be accessed sincePLATTC-123
is associated withvpce-1
.
Scenario: Access one environment from many VPC’s¶

The following resources are configured:
PLATT-abc
as a PrivateLink Attachment for accessing Kafka clusters in the env-prod environmentPLATTC-123
as a PrivateLink Attachment Connection for thevpce-1
VPC endpoint in VPC-1PLATTC-456
for thevpce-2
VPC endpoint in VPC-2ProdApp-1
as a Kafka client bootstrapped withlkc-123.us-west-2.aws.private.confluent.cloud
ProdApp-2
as a Kafka client bootstrapped withlkc-456.us-west-2.aws.private.confluent.cloud
phz-1
as a Route53 Private Hosted Zone with the regional wildcard*.us-west-2.aws.private.confluent.cloud
phz-2
as a Route53 Private Hosted Zone with the regional wildcard*.us-west-2.aws.private.confluent.cloud
The following steps are performed:
ProdApp-1
attempts to accesslkc-123
in the env-prod environment. A DNS query forlkc-123.us-west-2.aws.private.confluent.cloud
resolves againstphz-1
and returnsvpce-1
.ProdApp-1
sends traffic tovpce-1
.vpce-1
forwards traffic toPLATT-abc
, andlkc-123
can be accessed sincePLATTC-123
is associated withvpce-1
.ProdApp-2
attempts to accesslkc-456
in the env-prod environment. A DNS query forlkc-456.us-west-2.aws.private.confluent.cloud
resolves againstphz-2
and returnsvpce-2
.ProdApp-2
sends traffic tovpce-2
.vpce-2
forwards traffic toPLATT-abc
, andlkc-456
can be accessed sincePLATTC-456
is associated withvpce-2
.
Scenario: Access one environment from an on-premise network¶

The following resources are configured:
PLATT-abc
as a PrivateLink Attachment for accessing Kafka clusters in the env-abc environmentPLATTC-123
as a PrivateLink Attachment Connection for thevpce-1
endpoint in VPC-1On-Prem-1
as a Kafka client bootstrapped withlkc-123.us-west-2.aws.private.confluent.cloud
ProdApp-1
as a Kafka client bootstrapped withlkc-123.us-west-2.aws.private.confluent.cloud
phz-fwd
as a DNS forwarding rule with the regional wildcard*.us-west-2.aws.private.confluent.cloud
phz-1
as a Route53 Private Hosted Zone with the regional wildcard*.us-west-2.aws.private.confluent.cloud
The following steps are performed:
On-Prem-1
attempts to accesslkc-123
in the env-abc environment. A DNS query forlkc-123.us-west-2.aws.private.confluent.cloud
forwards tophz-1
and returnsvpce-1
.On-Prem-1
sends traffic tovpce-1
over AWS DirectConnect.vpce-1
forwards traffic toPLATT-abc
andlkc-123
can be accessed sincePLATTC-123
is associated withvpce-1
.