Use Private Network Interface on Confluent Cloud

Confluent Private Network Interface (PNI) enables private communication between your AWS resources, such as Kafka clients, and your Confluent Cloud clusters, specifically Freight and Enterprise clusters, through the Elastic Network Interface (ENI) that resides in your AWS account.

This capability is based on the AWS Multi-VPC ENI attachment feature. The feature can be used to attach a secondary Elastic Network Interface (ENI) in one VPC to a virtual machine (VM) running in another, even when those VPCs are in different AWS accounts or organizations. Key characteristics of ENIs for PNI are:

  • Located in Customer’s VPC: ENIs for PNI are associated with the subnets in customer VPC.

  • Owned and fully controlled by the Customer who can:

    • create, update and delete such ENIs

    • create and attach Security Group (SG) on such ENIs to control origination of connections

    • give limited permissions to Confluent to attach Confluent Cloud VMs to such ENIs

  • Used by Confluent: with limited ability to attach ENIs for PNI to Confluent Cloud VMs (after the customer gives permission)

../_images/aws-pni.png

Confluent PNIs retain all properties and behaviors of AWS ENIs, including the security rules you defined through security groups.

The security groups applied to PNIs control the source and type of inbound and outbound traffic allowed between your environment and Confluent Cloud services. You can leverage your existing security operations to create, apply, and track changes to these security groups.

Confluent Cloud uses the following private networking resources for PNI.

Gateway

A gateway is a resource that represents a connectivity type to and from Confluent Cloud services. It is created within an environment for the region and zone(s) you choose. The PNI gateway allows you to connect to Confluent Cloud services, such as Freight and Enterprise clusters, hosted in a given environment and region, from your network.

Access point

An access point is a resource that represents a connection instance to a gateway and must match the type of the gateway it connects to. An access point of type PNI consists of a set of AWS ENIs in the same cloud region as the gateway, carrying traffic to and from Confluent Cloud services. The PNI access point provides you with a connection to services like Freight and Enterprise clusters, hosted in a specific environment and region, from your network.

The high-level workflow is:

  1. In Confluent Cloud, create a gateway.

  2. Update the security group to block incoming connections from Confluent Cloud to customer network (Recommended).

  3. In AWS, create ENIs.

    Confluent Cloud provides a script that you can use to automate this process. The script is downloadable at the beginning of the next step, where you create an access point.

  4. In Confluent Cloud, create an access point by sharing the ENIs created in the previous step.

Requirements and considerations

  • Currently, PNI is supported for Freight and Enterprise Kafka clusters on AWS.

  • Enterprise clusters on AWS now support simultaneous connectivity using both PrivateLink Attachment and PNI.

  • Switching from PrivateLink to PNI, or vice versa, is fully supported and does not require redeploying the cluster.

  • You need to create 51 ENIs and assign network interface permissions to them as described in Create ENIs. This ensures that the network layer can support scaling operations without becoming a bottleneck for the service.

  • Currently, ENIs need to be created by the user with the minimum EC2 permissions. Create or use an IAM role in your AWS account and attach policies that allow lifecycle management of network interfaces.

    Expand the following to view an example policy:

    pni-eni-policy.json 
    [
   {
      "Version": "2012-10-17",
      "Statement":
      [
         {
            "Effect": "Allow",
            "Action": [
               "ec2:CreateNetworkInterface",
               "ec2:CreateNetworkInterfacePermission",
               "ec2:DescribeNetworkInterfaces",
               "ec2:DeleteNetworkInterface",
               "ec2:DescribeSubnets",
               "ec2:DescribeAvailabilityZones",
               "ec2:DescribeSecurityGroups"
            ],
            "Resource": [
               "arn:aws:ec2:*:*:network-interface/*",
               "arn:aws:ec2:*:*:subnet/*",
               "arn:aws:ec2:*:*:security-group/*"
            ]
         }
      ]
   }
]

  • The ENIs you create in your AWS account must be in the same Availability Zone (AZ) and have the same AZ ID as the zones in Confluent Cloud.

  • Each ENI (or set of ENIs) can be associated with only one PNI access point.

  • A single PNI gateway can be used to access multiple Enterprise and Freight clusters in the same region/environment.

  • You can currently set up two PNI gateways per Confluent Cloud environment. If you need more than two gateways, contact Confluent Support.

  • Confluent Cloud Console components, such as topic management, require additional configuration to function as they use cluster endpoints. To use the Confluent Cloud Console with PNI, see Use the Confluent Cloud Console with Private Networking.

  • For enhanced security, it is recommended that you block incoming connections from Confluent Cloud to customer network (i.e., Confluent Cloud–initiated requests) by attaching the security group with appropriate rules to PNI. This effectively denies all egress connections from Confluent Cloud infrastructure to customer network and mirrors the connectivity behavior of AWS PrivateLink. See Update the security group for steps.

  • Egress over PNI is not yet supported. Use an Egress PrivateLink Endpoint instead.

For some of the common questions about PNI, see PNI FAQs.

Network security

An ENI for PNI is a network attachment point on your VPC subnet, capable of handling all network traffic independent of its origination direction.

../_images/aws-pni-connectivity.png

PNI can offer:

  • Security posture at par with PrivateLink: While PNI uses a different shared responsibility model than PrivateLink, you can achieve the same strong security posture. As shown in the figure above, you can use standard AWS Security Groups to control traffic flowing through the ENI. You can customize these Security Groups to mimic the specific security behaviors of PrivateLink, by blocking all connections from Confluent into the customer network.

    We strongly recommend such Security Group configuration.

  • More flexibility than PrivateLink: ENI for PNI offers greater flexibility as compared to PrivateLink, which is designed to allow traffic originating from one direction to specific network ports. PNI offers an optional capability to the user to open up select connections from Confluent into the customer network to allow legitimate use cases by services like Fully Managed Connectors.

Create a gateway

In order for Confluent Cloud clusters to be able to leverage PNI for inbound connectivity, create a gateway to enable connectivity.

To create a gateway:

  1. In the Confluent Cloud Console, select an environment for the PNI.

  2. In the Network management tab in the environment, click For serverless products.

  3. Click + Add gateway configuration (if you have one or more existing gateways in the environment) or + Create gateway configuration (if you are creating the first gateway in the environment).

  4. Select the Private Network Interface card to select the type of gateway configuration.

  5. Click + Create configuration.

  6. On the Configure gateway sliding panel, enter the following information.

    • Gateway name

    • Cloud provider: AWS

    • Region

    • Zones

  7. Click Submit.

Send a request to create a gateway:

HTTP POST request

POST https://api.confluent.cloud/networking/v1/gateways

Authentication

See Authentication.

Request specification

{
  "spec": {
    "display_name": "<The name of the gateway>",
    "config": {
      "kind": "AwsPrivateNetworkInterfaceGatewaySpec",
      "region": "<AWS region of the gateway>",
      "zones": "<AWS availability zone ids of the gateway>"
    },
    "environment": {
      "id": "<The environment ID where the gateway belongs to>",
      "environment": "<The environment of the gateway>"
    }
  }
}

Use the confluent network gateway create Confluent CLI command to create a PNI gateway:

confluent network gateway create [name] [flags]

The following are the command-specific flags:

  • --cloud: Required. The cloud provider. Set to aws.

  • --region: Required. AWS region of the gateway.

  • --type: Required. Set to private-network-interface.

  • --zones: Required. Set to a comma-separated list of availability zones.

You can specify additional optional CLI flags described in the Confluent CLI command reference, such as --environment.

Use the confluent_gateway resource to create a PNI gateway.

An example snippet of Terraform configuration:

resource "confluent_environment" "development" {
  display_name = "Development"
}

resource "confluent_gateway" "staging" {
    display_name = "test-gateway"
    environment {
        id = confluent_environment.staging.id
    }
    aws_private_network_interface_gateway {
        region = "us-west-2"
        zones = ["usw2-az1", "usw2-az2", "usw2-az3"]
    }
}

Create ENIs

Create the ENIs using Terraform or using AWS CLI. For PNI, you need a minimum of 51 ENIs.

Create ENIs using Terraform

The following is a sample code to create an ENI and the permission to be attached.

  • The code revokes the egress rule from the security group.

create_enis.tf

main.tf:

# Data sources to get VPC and subnet information
data "aws_vpc" "main" {
  id = var.vpc_id
}

data "aws_subnet" "pni_subnets" {
  count = length(var.subnet_ids)
  id    = var.subnet_ids[count.index]
}

# Dynamic private IP calculation using actual subnet CIDR blocks
# Starts at offset 10 in each subnet and increments for each ENI

# Create network interfaces
resource "aws_network_interface" "pni" {
  count = length(var.subnet_ids) * var.num_pni_per_subnet

  subnet_id = var.subnet_ids[floor(count.index / var.num_pni_per_subnet)]
  security_groups = [aws_security_group.main.id]

  # Conditional IP assignment based on auto_assign_ips variable
  # When auto_assign_ips = false (default): Use sequential IP assignment starting at offset 10
  # When auto_assign_ips = true: Let AWS auto-assign IP addresses from available addresses

  private_ips = var.auto_assign_ips ? [] : [
    cidrhost(
      data.aws_subnet.pni_subnets[floor(count.index / var.num_pni_per_subnet)].cidr_block,
      10 + (count.index % var.num_pni_per_subnet) + 1
    )
  ]

  description = "Confluent PNI-sub-${floor(count.index / var.num_pni_per_subnet)}-eni-${(count.index % var.num_pni_per_subnet) + 1}"

  tags = {
    Name = "Confluent-PNI-sub-${floor(count.index / var.num_pni_per_subnet)}-eni-${(count.index % var.num_pni_per_subnet) + 1}"
  }
}

resource "aws_network_interface_permission" "pni_permit" {
  count = length(aws_network_interface.pni)

  network_interface_id = aws_network_interface.pni[count.index].id
  permission           = "INSTANCE-ATTACH"
  aws_account_id       = var.aws_account_id
}

resource "aws_security_group" "main" {
  name        = "pni-demo"
  description = "Demo security group for PNI test (EC2 + ENIs)"
  vpc_id      = data.aws_vpc.main.id

  # Block all outbound traffic.
  # See https://docs.confluent.io/cloud/current/networking/aws-pni.html#update-the-security-group-recommended.
  egress = []

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

# Required:Kafka REST API access
resource "aws_vpc_security_group_ingress_rule" "allow_kafka_broker_access" {
  security_group_id = aws_security_group.main.id
  from_port     = 9092
  to_port       = 9092
  ip_protocol   = "tcp"
  cidr_ipv4     = data.aws_vpc.main.cidr_block

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

# Required:Kafka broker access for ENIs
resource "aws_vpc_security_group_ingress_rule" "allow_kafka_REST_access" {
  security_group_id = aws_security_group.main.id
  from_port     = 443
  to_port       = 443
  ip_protocol   = "tcp"
  cidr_ipv4     = data.aws_vpc.main.cidr_block

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

variables.tf:

variable "subnet_ids" {
  type = list(string)
  description = "List of subnet IDs where network interfaces will be created"
}

variable "aws_account_id" {
  type = string
  description = "Confluent AWS Account ID for setting network interface permission"
}

variable "aws_region" {
  type = string
  description = "The AWS region where resources will be created"
}

variable "num_pni_per_subnet" {
  type = number
  description = "Number of private network interfaces per subnet"
}

variable "vpc_id" {
  type = string
  description = "VPC ID where the subnets are located"
}

variable "auto_assign_ips" {
  type = bool
  description = "Whether to let AWS auto-assign IP addresses (true) or use sequential assignment (false)"
  default = true
}

Example terraform.tfvars:

subnet_ids = ["subnet-00000000000000001", "subnet-00000000000000002", "subnet-00000000000000003"]
aws_account_id = "012345678901"
aws_region = "eu-west-1"
num_pni_per_subnet = 17
vpc_id = "vpc-00000000000000001"
auto_assign_ips = true

Create ENIs using AWS CLI

The following is an AWS CLI script to create ENIs. You need to have AWS CLI installed to run the script.

create_aws_enis_with_permission.sh

Run the script with the following 10 arguments. The script creates 17 ENIs in each subnet, 51 total ENIs.

  • 3 pairs of subnets and its base IP addresses

  • 1 security group ID

  • 1 Confluent AWS account ID from the PNI gateway creation response. Note that this is NOT your AWS account ID.

  • 1 AWS region

  • 17 ENIs per subnet

#!/bin/bash

# Check if 10 arguments are given (3 subnets, 3 base IPs, 1 security group, 1 AWS account, 1 AWS region, 17 pnis per subnet)
if [ "$#" -ne 10 ]; then
    echo "Usage: $0 <subnet-id1> <base-ip1> <subnet-id2> <base-ip2> <subnet-id3> <base-ip3> <security-group-id> <confluent-aws-account-id> <aws-region> <num_pni_per_subnet>>"
    echo
    echo "Example: ./create_aws_enis_with_permission.sh subnet-00000000000000001 100.251.1.10 subnet-00000000000000002 100.251.2.10 subnet-00000000000000003 100.251.3.10 sg-9999999999999999 012345678901 eu-west-1 17"
    exit 1
fi

# Assigning arguments to readable variables
subnet_ids=($1 $3 $5)
base_ips=($2 $4 $6)
security_group_id=$7
aws_account_id=$8
aws_region=$9
num_pni=${10}

# Initialize an array to store successful NetworkInterfaceIds
created_network_interface_ids=()

for i in 0 1 2; do # Adjusted to correct array indexing
    subnet_id=${subnet_ids[$i]}
    base_ip=${base_ips[$i]}
    ip_prefix=$(echo $base_ip | cut -d '.' -f 1-3)
    last_octet=$(echo $base_ip | cut -d '.' -f 4)

    for j in $(seq 1 $num_pni); do
        next_ip_last_octet=$((last_octet + j))
        if [ $next_ip_last_octet -le 255 ]; then
            next_ip="$ip_prefix.$next_ip_last_octet"
        else
            echo "Skipping IP $ip_prefix.$next_ip_last_octet - exceeds valid range."
            continue
        fi

        description="Confluent PNI-sub-${i}-eni-$j"
        tag="Confluent-PNI-sub-${i}-eni-$j"

        create_ni_output=$(aws ec2 create-network-interface \
          --subnet-id $subnet_id \
          --description "$description" \
          --groups $security_group_id \
          --private-ip-address $next_ip \
          --tag-specifications "ResourceType=network-interface,Tags=[{Key=Name,Value=$tag}]" \
          --region $aws_region)

        if [ $? -eq 0 ]; then
            network_interface_id=$(echo $create_ni_output | jq -r '.NetworkInterface.NetworkInterfaceId')
            created_network_interface_ids+=("$network_interface_id")
            echo "Created network interface $network_interface_id with IP $next_ip"

            aws ec2 create-network-interface-permission \
              --network-interface-id $network_interface_id \
              --permission INSTANCE-ATTACH \
              --aws-account $aws_account_id \
              --region $aws_region

            if [ $? -eq 0 ]; then
                echo "Permission set for $network_interface_id"
            else
                echo "Failed to set permission for $network_interface_id"
            fi

        else
            echo "Failed to create network interface for $next_ip"
        fi
    done
done

# Convert the array of NetworkInterfaceIds to a comma-separated string
network_interface_ids_str=$(IFS=, ; echo "${created_network_interface_ids[*]}")


# Describing the security group rules
echo "Describing security group rules for group ID: $security_group_id"
aws ec2 describe-security-groups --group-ids $security_group_id --region $aws_region --output table --no-cli-pager

# Emit the comma-separated list of NetworkInterfaceIds
echo "All created NetworkInterfaceIds with above security group: $network_interface_ids_str"

For example:

./create_aws_enis_with_permission.sh \
  subnet-00000000000000001 100.251.1.10 \
  subnet-00000000000000002 100.251.2.10 \
  subnet-00000000000000003 100.251.3.10 \
  sg-9999999999999999 \
  012345678901 \
  eu-west-1 \
  17

Create an access point

Create a Confluent Cloud access point that represents a PNI connection.

  1. In the Network Management tab of the desired Confluent Cloud environment, click the For serverless products tab.

  2. Click the gateway to which you want to add this access point.

  3. In the Access points tab, click Add access point.

  4. Follow the guided steps to specify the field values:

    1. Specify the name of the access point.

    2. Create ENIs.

    3. Specify the IDs of the ENIs that you created in the previous step.

    4. Enter the AWS account ID where the ENIs were created.

    5. Click Submit.

Send a request to create an access point:

HTTP POST request

POST https://api.confluent.cloud/networking/v1/access-points

Authentication

See Authentication.

Request specification

{
  "spec": {
    "display_name": "<The name for the access point>",
    "config": {
      "kind": "AwsPrivateNetworkInterface",
      "network_interfaces": "<List of 51 or more ENI IDs>",
      "account": "<The AWS account ID associated with the ENIs you are using for the Confluent PNI>",
    },
    "environment": {
      "id": "<The environment ID where the access point belongs to>",
      "environment": "<The environment of the access point>"
    },
    "gateway": {
      "id": "<The gateway ID to which this belongs>",
      "environment": "<The environment of the gateway>"
    }
  }
}

Use the confluent network access-point private-network-interface create Confluent CLI command to create a PNI access point.

confluent network access-point private-network-interface create [name] [flags]

The following are the command-specific flags:

  • --cloud: Required. The cloud provider. Set to aws.

  • --gateway: Required. Gateway ID.

  • --network-interfaces: Required. A comma-separated list of the IDs of the ENIs you created in Create ENIs. You need to specify a minimum of 51 ENI IDs.

  • --account: Required. The AWS account ID associated with the ENIs you created in Create ENIs.

You can specify additional optional CLI flags described in the Confluent CLI command reference, such as --environment.

The following is an example Confluent CLI command to create a PNI access point:

confluent network access-point private-network-interface create \
  --cloud aws \
  --gateway gw-123456 \
  --network-interfaces eni-00000000000000000,eni-00000000000000001,... \
  --account 000000000000

Use the confluent_access_point resource to create an access point.

An example snippet of Terraform configuration, using the code from Create ENIs using Terraform:

resource "confluent_environment" "development" {
    display_name = "Development"
}

resource "confluent_access_point" "staging" {
    display_name = "test-access-point"
    environment {
        id = confluent_environment.staging.id
    }
    gateway {
        id = confluent_gateway.staging.id
    }
    aws_private_network_interface {
       network_interfaces = aws_network_interface.pni[*].id
       account            = var.aws_account_id
    }
}

DNS resolution

Confluent Cloud hosts and manages the DNS records for endpoints which use PNI connectivity. In order for your Kafka clients to resolve these DNS names, they must be able to resolve records from public DNS servers.

Confluent does not provide a supported private DNS solution for the clusters that use PNI connectivity.