Use Private Network Interface on Confluent Cloud

Confluent Private Network Interface (PNI), powered by AWS Elastic Network Interface (ENI), enables you to attach an ENI from your AWS account to a network service in the Confluent AWS account. This allows you to access Confluent Cloud services, such as Freight and Enterprise clusters, through the ENI that resides in your AWS account and offers PrivateLink-like one way connectivity with user controlled security groups.

Confluent PNIs retain all properties and behaviors of AWS ENIs, including the security rules you defined through security groups.

The security groups applied to PNIs control the source and type of inbound and outbound traffic allowed between your environment and Confluent Cloud services. You can leverage your existing security operations to create, apply, and track changes to these security groups.

../_images/aws-pni.png

Confluent Cloud uses the following private networking resources for PNI.

Gateway
A gateway is a resource that represents a connectivity type to and from Confluent Cloud services. It is created within an environment for the region and zone(s) you choose. The PNI gateway allows you to connect to Confluent Cloud services, such as Freight and Enterprise clusters, hosted in a given environment and region, from your network.
Access point
An access point is a resource that represents a connection instance to a gateway and must match the type of the gateway it connects to. An access point of type PNI consists of a set of AWS ENIs in the same cloud region as the gateway, carrying traffic to and from Confluent Cloud services. The PNI access point provides you with a connection to services like Freight and Enterprise clusters, hosted in a specific environment and region, from your network.

The high-level workflow is:

  1. In Confluent Cloud, create a gateway.

  2. Update the security group to block incoming connections from Confluent Cloud to customer network (Recommended).

  3. In AWS, create ENIs.

    Confluent Cloud provides a script that you can use to automate this process. The script is downloadable at the beginning of the next step, where you create an access point.

  4. In Confluent Cloud, create an access point by sharing the ENIs created in the previous step.

Requirements and considerations

  • Currently, PNI is supported for Freight and Enterprise clusters on AWS.

  • Enterprise clusters on AWS now support simultaneous connectivity using both PrivateLink Attachment and PNI.

  • Switching from PrivateLink to PNI, or vice versa, is fully supported and does not require redeploying the cluster.

  • You need to create 51 ENIs and assign network interface permissions to them as described in Create ENIs. This ensures that the network layer can support scaling operations without becoming a bottleneck for the service.

  • Currently, ENIs need to be created by the user with the minimum EC2 permissions. Create or use an IAM role in your AWS account and attach policies that allow lifecycle management of network interfaces.

    Expand the following to view an example policy:

    pni-eni-policy.json 
    [
   {
      "Version": "2012-10-17",
      "Statement":
      [
         {
            "Effect": "Allow",
            "Action": [
               "ec2:CreateNetworkInterface",
               "ec2:CreateNetworkInterfacePermission",
               "ec2:DescribeNetworkInterfaces",
               "ec2:DeleteNetworkInterface",
               "ec2:DescribeSubnets",
               "ec2:DescribeAvailabilityZones",
               "ec2:DescribeSecurityGroups"
            ],
            "Resource": [
               "arn:aws:ec2:*:*:network-interface/*",
               "arn:aws:ec2:*:*:subnet/*",
               "arn:aws:ec2:*:*:security-group/*"
            ]
         }
      ]
   }
]

  • The ENIs you create in your AWS account must be in the same Availability Zone (AZ) and have the same AZ ID as the zones in Confluent Cloud.
  • Each ENI (or set of ENIs) can be associated with only one PNI access point.
  • A single PNI gateway can be used to access multiple Enterprise and Freight clusters in the same region/environment.
  • You can currently set up two PNI gateways per Confluent Cloud environment. If you need more than two gateways, contact Confluent Support.
  • Confluent Cloud Console components, such as topic management, require additional configuration to function as they use cluster endpoints. To use the Confluent Cloud Console with PNI, see Use the Confluent Cloud Console with Private Networking.
  • For enhanced security, it is recommended that you block incoming connections from Confluent Cloud to customer network (i.e., Confluent Cloud–initiated requests) by attaching the security group with appropriate rules to PNI. This effectively denies all egress connections from Confluent Cloud infrastructure to customer network and mirrors the connectivity behavior of AWS PrivateLink. See Update the security group for steps.

Create a gateway

In order for Confluent Cloud clusters to be able to leverage PNI for inbound connectivity, create a gateway to enable connectivity.

To create a gateway:

  1. In the Confluent Cloud Console, select an environment for the PNI.
  2. In the Network management tab in the environment, click For serverless products.
  3. Click + Add gateway configuration (if you have one or more existing gateways in the environment) or + Create gateway configuration (if you are creating the first gateway in the environment).
  4. Select the Private Network Interface card to select the type of gateway configuration.
  5. Click + Create configuration.
  6. On the Configure gateway sliding panel, enter the following information.
    • Gateway name
    • Cloud provider: AWS
    • Region
    • Zones
  7. Click Submit.

Create ENIs

Create the ENIs using Terraform or using AWS CLI. For PNI, you need a minimum of 51 ENIs.

Create ENIs using Terraform

The following is a sample code to create an ENI and the permission to be attached.

  • The code revokes the egress rule from the security group.
create_enis.tf

main.tf:

# Data sources to get VPC and subnet information
data "aws_vpc" "main" {
  id = var.vpc_id
}

data "aws_subnet" "pni_subnets" {
  count = length(var.subnet_ids)
  id    = var.subnet_ids[count.index]
}

# Dynamic private IP calculation using actual subnet CIDR blocks
# Starts at offset 10 in each subnet and increments for each ENI

# Create network interfaces
resource "aws_network_interface" "pni" {
  count = length(var.subnet_ids) * var.num_pni_per_subnet

  subnet_id = var.subnet_ids[floor(count.index / var.num_pni_per_subnet)]
  security_groups = [aws_security_group.main.id]

  # Conditional IP assignment based on auto_assign_ips variable
  # When auto_assign_ips = false (default): Use sequential IP assignment starting at offset 10
  # When auto_assign_ips = true: Let AWS auto-assign IP addresses from available addresses

  private_ips = var.auto_assign_ips ? [] : [
    cidrhost(
      data.aws_subnet.pni_subnets[floor(count.index / var.num_pni_per_subnet)].cidr_block,
      10 + (count.index % var.num_pni_per_subnet) + 1
    )
  ]

  description = "Confluent PNI-sub-${floor(count.index / var.num_pni_per_subnet)}-eni-${(count.index % var.num_pni_per_subnet) + 1}"

  tags = {
    Name = "Confluent-PNI-sub-${floor(count.index / var.num_pni_per_subnet)}-eni-${(count.index % var.num_pni_per_subnet) + 1}"
  }
}

resource "aws_network_interface_permission" "pni_permit" {
  count = length(aws_network_interface.pni)

  network_interface_id = aws_network_interface.pni[count.index].id
  permission           = "INSTANCE-ATTACH"
  aws_account_id       = var.aws_account_id
}

resource "aws_security_group" "main" {
  name        = "pni-demo"
  description = "Demo security group for PNI test (EC2 + ENIs)"
  vpc_id      = data.aws_vpc.main.id

  # Block all outbound traffic.
  # See https://docs.confluent.io/cloud/current/networking/aws-pni.html#update-the-security-group-recommended.
  egress = []

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

# Required:Kafka REST API access
resource "aws_vpc_security_group_ingress_rule" "allow_kafka_broker_access" {
  security_group_id = aws_security_group.main.id
  from_port     = 9092
  to_port       = 9092
  ip_protocol   = "tcp"
  cidr_ipv4     = data.aws_vpc.main.cidr_block

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

# Required:Kafka broker access for ENIs
resource "aws_vpc_security_group_ingress_rule" "allow_kafka_REST_access" {
  security_group_id = aws_security_group.main.id
  from_port     = 443
  to_port       = 443
  ip_protocol   = "tcp"
  cidr_ipv4     = data.aws_vpc.main.cidr_block

  tags = {
    Name = "enterprise-pni-aws-kafka-rbac"
  }
}

variables.tf:

variable "subnet_ids" {
  type = list(string)
  description = "List of subnet IDs where network interfaces will be created"
}

variable "aws_account_id" {
  type = string
  description = "Confluent AWS Account ID for setting network interface permission"
}

variable "aws_region" {
  type = string
  description = "The AWS region where resources will be created"
}

variable "num_pni_per_subnet" {
  type = number
  description = "Number of private network interfaces per subnet"
}

variable "vpc_id" {
  type = string
  description = "VPC ID where the subnets are located"
}

variable "auto_assign_ips" {
  type = bool
  description = "Whether to let AWS auto-assign IP addresses (true) or use sequential assignment (false)"
  default = true
}

Example terraform.tfvars:

subnet_ids = ["subnet-00000000000000001", "subnet-00000000000000002", "subnet-00000000000000003"]
aws_account_id = "012345678901"
aws_region = "eu-west-1"
num_pni_per_subnet = 17
vpc_id = "vpc-00000000000000001"
auto_assign_ips = true

Create ENIs using AWS CLI

The following is an AWS CLI script to create ENIs. You need to have AWS CLI installed to run the script.

create_aws_enis_with_permission.sh

Run the script with the following 10 arguments. The script creates 17 ENIs in each subnet, 51 total ENIs.

  • 3 pairs of subnets and its base IP addresses
  • 1 security group ID
  • 1 Confluent AWS account ID from the PNI gateway creation response. Note that this is NOT your AWS account ID.
  • 1 AWS region
  • 17 ENIs per subnet
#!/bin/bash

# Check if 10 arguments are given (3 subnets, 3 base IPs, 1 security group, 1 AWS account, 1 AWS region, 17 pnis per subnet)
if [ "$#" -ne 10 ]; then
    echo "Usage: $0 <subnet-id1> <base-ip1> <subnet-id2> <base-ip2> <subnet-id3> <base-ip3> <security-group-id> <confluent-aws-account-id> <aws-region> <num_pni_per_subnet>>"
    echo
    echo "Example: ./create_aws_enis_with_permission.sh subnet-00000000000000001 100.251.1.10 subnet-00000000000000002 100.251.2.10 subnet-00000000000000003 100.251.3.10 sg-9999999999999999 012345678901 eu-west-1 17"
    exit 1
fi

# Assigning arguments to readable variables
subnet_ids=($1 $3 $5)
base_ips=($2 $4 $6)
security_group_id=$7
aws_account_id=$8
aws_region=$9
num_pni=${10}

# Initialize an array to store successful NetworkInterfaceIds
created_network_interface_ids=()

for i in 0 1 2; do # Adjusted to correct array indexing
    subnet_id=${subnet_ids[$i]}
    base_ip=${base_ips[$i]}
    ip_prefix=$(echo $base_ip | cut -d '.' -f 1-3)
    last_octet=$(echo $base_ip | cut -d '.' -f 4)

    for j in $(seq 1 $num_pni); do
        next_ip_last_octet=$((last_octet + j))
        if [ $next_ip_last_octet -le 255 ]; then
            next_ip="$ip_prefix.$next_ip_last_octet"
        else
            echo "Skipping IP $ip_prefix.$next_ip_last_octet - exceeds valid range."
            continue
        fi

        description="Confluent PNI-sub-${i}-eni-$j"
        tag="Confluent-PNI-sub-${i}-eni-$j"

        create_ni_output=$(aws ec2 create-network-interface \
          --subnet-id $subnet_id \
          --description "$description" \
          --groups $security_group_id \
          --private-ip-address $next_ip \
          --tag-specifications "ResourceType=network-interface,Tags=[{Key=Name,Value=$tag}]" \
          --region $aws_region)

        if [ $? -eq 0 ]; then
            network_interface_id=$(echo $create_ni_output | jq -r '.NetworkInterface.NetworkInterfaceId')
            created_network_interface_ids+=("$network_interface_id")
            echo "Created network interface $network_interface_id with IP $next_ip"

            aws ec2 create-network-interface-permission \
              --network-interface-id $network_interface_id \
              --permission INSTANCE-ATTACH \
              --aws-account $aws_account_id \
              --region $aws_region

            if [ $? -eq 0 ]; then
                echo "Permission set for $network_interface_id"
            else
                echo "Failed to set permission for $network_interface_id"
            fi

        else
            echo "Failed to create network interface for $next_ip"
        fi
    done
done

# Convert the array of NetworkInterfaceIds to a comma-separated string
network_interface_ids_str=$(IFS=, ; echo "${created_network_interface_ids[*]}")


# Describing the security group rules
echo "Describing security group rules for group ID: $security_group_id"
aws ec2 describe-security-groups --group-ids $security_group_id --region $aws_region --output table --no-cli-pager

# Emit the comma-separated list of NetworkInterfaceIds
echo "All created NetworkInterfaceIds with above security group: $network_interface_ids_str"

For example:

./create_aws_enis_with_permission.sh \
  subnet-00000000000000001 100.251.1.10 \
  subnet-00000000000000002 100.251.2.10 \
  subnet-00000000000000003 100.251.3.10 \
  sg-9999999999999999 \
  012345678901 \
  eu-west-1 \
  17

Create an access point

Create a Confluent Cloud access point that represents a PNI connection.

  1. In the Network Management tab of the desired Confluent Cloud environment, click the For serverless products tab.
  2. Click the gateway to which you want to add this access point.
  3. In the Access points tab, click Add access point.
  4. Follow the guided steps to specify the field values:
    1. Specify the name of the access point.
    2. Create ENIs.
    3. Specify the IDs of the ENIs that you created in the previous step.
    4. Enter the AWS account ID where the ENIs were created.
    5. Click Submit.