Enable Private Networking on Confluent Cloud with Schema Registry PrivateLink (Limited Availability)

Confluent Cloud Schema Registry now supports private networking on Amazon Web Services, currently in “Limited Access” for production use to a subset of Confluent customers. If you are interested in the Limited Access release, sign up here to get started: Sign up for AWS Private Link on Schema Registry (Limited Availability)

This feature enables your client applications in the virtual private cloud (VPC) to securely access Schema Registry without egressing to the public internet from your VPC.

This capability is available for new environments. Schema Registry will be automatically provisioned to use private networking if the first Apache Kafka® cluster within the environment is provisioned to use private link connectivity. Currently, Schema Registry private link supports only AWS Dedicated clusters and Enterprise clusters. Therefore, the first Kafka cluster in the new environment must be either an AWS Dedicated cluster or Enterprise cluster configured to use Private Link.

All Confluent Cloud internal access to a Schema Registry with private connectivity (Schema Validation, Connectors using schemas, Flink, and ksqlDB) is also via private network. Therefore, your traffic is never exposed to the public internet when connecting to Schema Registry.

How does it work?

Schema Registry Private Networking requires a Private Link Attachment (PLATT). Private Link Attachments are resources that enable you to connect to Confluent serverless products, like Enterprise Clusters and Apache Flink®.

For Schema Registry, the new Private Link Attachment is used only to establish a connection between your client applications (including Cloud Console, Confluent CLI, Terraform) and Schema Registry. As a result, this PLATT is used by your clients to access Schema Registry from your VPC.

Prerequisites

• Access to Confluent Cloud
• The OrganizationAdmin to provision a new environment with Dedicated clusters or Enterprise clusters on AWS configured to use Schema Registry PrivateLink
• The OrganizationAdmin, EnvironmentAdmin, or NetworkAdmin to create a PLATT
• A VPC in AWS

Provision Schema Registry to use private networking

Once you create the new environment, provision an AWS Dedicated or Enterprise cluster with private link configuration as the first Kafka cluster in the environment, the Schema Registry will automatically be provisioned to use Private Link.

After you provision the Enterprise (ESKU) or Dedicated (DSKU) cluster, you will need to create a PLATT to access the Schema Registry.

• For Dedicated clusters, Schema Registry requires that you define a PLATT in the same region and environment of the cluster, even if you create a Private Link for the Dedicated cluster.
• For Enterprise clusters, the same PLATT is automatically used by Schema Registry and vice-versa.

Create a PrivateLink Attachment overview

This walkthrough guides you through the following steps to create a PLATT.

1. Create PLATT/PLATTC:
   1. In Confluent Cloud, create a PrivateLink Attachment as shown in Step 1.
   2. In AWS, create a PrivateLink Attachment Connection, and create a VPC Endpoint linked to the PrivateLink Attachment service.
   3. In AWS Route53, set up a DNS resolution for this endpoint.
2. (Optional step) If your client is not in the VPC, enable the Confluent Cloud Console or Confluent CLI to connect to your private network as shown in Step 2.

Step 1: Create a PrivateLink Attachment

Add the network configuration

The following steps show how to create a PrivateLink Attachment using the Cloud Console.

If a PrivateLink Attachment exists already, you don’t need to create another, because the existing PrivateLink Attachment provides connectivity for the current environment and region.

1. Log in to the Cloud Console at https://confluent.cloud, and navigate to an environment with the Schema Registry you want to use.

2. In the environment details page, click Network management and ensure that For serverless products is selected.

3. Click Add network configuration.

4. In the Add network configuration page, select your cloud service provider and region.

   Note

   Currently, Schema Registry supports only the AWS PrivateLink Attachment.

5. Click Continue.

6. In the Network name textbox, type the name of your PrivateLink Attachment network.

7. Click Add network configuration.

   The environment details page opens and shows your network. Provisioning the network may take a few seconds.

Create a PrivateLink Attachment Connection

1. Click Add connection and follow the instructions in Create a VPC Endpoint.
2. Set the routes by following the instructions in Set up DNS resolution.

When you have successfully created the PLATT, you should see the private connectivity endpoint for Schema Registry on the Cloud Console as shown:

At this point, Schema Registry is ready for use on the privatelink. The next step is optional.

(Optional) Step 2: Connect to the network with Confluent Cloud Console

This step is required only if you want to view and manage schemas on the Cloud Console outside of your VPC. If you don’t connect from a machine within the VPC, you see the following error when attempting to access schemas.

To connect to Confluent Cloud with PrivateLink Attachment, see Use Confluent Cloud with Private Networking. The Resource Metadata access option is not currently supported for Schema Registry.

One way to connect is to set up a reverse proxy:

1. Create an EC2 instance

2. Connect to the instance with SSH

3. Install NGINX

4. Configure Routing Table

5. Set up DNS resolution: point to the Schema Registry regional endpoints you use, as described in Step 6 of Configure a proxy.

   <Public IP Address of VM instance> <schema-registry-private-endpoint>
   

   <schema-registry-private-endpoint> will resemble lsrc-1234.<region>.<cloud>.private.confluent.cloud, for example: lsrc-axliw12p.us-east-2.aws.private.confluent.cloud.

   Find the DNS part of the PrivateLink Attachment by navigating to your environment’s Network management page and finding the DNS domain setting.

   

Once networking is set up in Cloud Console, the interface uses the correct endpoint automatically, and you should be able to view your schemas on the Schema Registry page.

Limitations

• Make sure all your resources in the environment belong to the region where the Schema Registry exists. Confluent Cloud resources outside this region cannot access the Schema Registry with private connectivity enabled.
• Resource Metadata access is not applicable for Schema Registry.
• To view the schemas on the Schema Registry tab you have to connect to the Schema Registry through the private network containing the Schema Registry private endpoints or set up a proxy as described in Step 2.
• Schema Linking is not supported between two fully managed registries on Confluent Cloud.
• Existing registries using public endpoints cannot be configured to use Private Connectivity.

Supported AWS Regions (required for limited access)

The following AWS regions are supported under the current “Limited Availability” release.

The Americas and Canada

Code	Region
ca-central-1	Canada (Central)
us-west-2	US West (Oregon)
us-east-1	US East (N. Virginia)
us-east-2	US East (Ohio)
sa-east-1	South America (São Paulo)

Asia Pacific

Code	Region
ap-southeast-1	Asia Pacific (Singapore)
ap-south-1	Asia Pacific (Mumbai)
ap-east-1	Asia Pacific (Hong Kong)
ap-northeast-2	Asia Pacific (Seoul)
ap-southeast-2	Asia Pacific (Sydney)

Europe

Code	Region
eu-central-1	Europe (Frankfurt)
eu-west-1	Europe (Ireland)
eu-west-2	Europe (London)
eu-west-3	Europe (Paris)

Africa

Code	Region
af-south-1	Africa (Cape Town)