Enable Private Networking on Confluent Cloud with Schema Registry PrivateLink (Early Access)

Note

The Schema Registry PrivateLink feature is available in a limited Early Access release, and not available by default. Contact your Confluent Cloud team if you want to join the Early Access program, and gain access to Schema Registry PrivateLink. Early Access Program features are intended for evaluation use in development and testing environments only, and not suitable for production use. Early Access Program features are considered to be a Proof of Concept as defined in the Confluent Cloud Terms of Service.

Confluent Cloud Schema Registry now supports private networking on Amazon Web Services. This feature enables your client applications in the virtual private cloud (VPC) to securely access Schema Registry without egressing to the public internet from your VPC.

This capability is available for new environments. Schema Registry will be automatically provisioned to use private networking if the first Apache Kafka® cluster within the environment is provisioned to use private link connectivity. Currently, Schema Registry private link supports only AWS Dedicated clusters and Enterprise clusters. Therefore, the first Kafka cluster in the new environment must be either an AWS Dedicated cluster or Enterprise cluster configured to use Private Link.

All Confluent Cloud internal access to a Schema Registry with private connectivity (Schema Validation, Connectors using schemas, Flink, and ksqlDB) is also via private network. Therefore, your traffic is never exposed to the public internet when connecting to Schema Registry.

How does it work?

Schema Registry Private Networking requires a Private Link Attachment (PLATT). Private Link Attachments are resources that enable you to connect to Confluent serverless products, like Enterprise Clusters and Apache Flink®.

For Schema Registry, the new Private Link Attachment is used only to establish a connection between your client applications (including Cloud Console, Confluent CLI, Terraform) and Schema Registry. As a result, this PLATT is used by your clients to access Schema Registry from your VPC.

Prerequisites

• Access to Confluent Cloud
• The OrganizationAdmin to provision a new environment with Dedicated clusters or Enterprise clusters on AWS configured to use Schema Registry PrivateLink
• The OrganizationAdmin, EnvironmentAdmin, or NetworkAdmin to create a PLATT
• A VPC in AWS

Provision Schema Registry to use private networking

Once you create the new environment, provision an AWS Dedicated or Enterprise cluster with private link configuration as the first Kafka cluster in the environment, the Schema Registry will automatically be provisioned to use Private Link.

After you provision the Enterprise (ESKU) or Dedicated (DSKU) cluster, you will need to create a PLATT to access the Schema Registry.

• For Dedicated clusters, Schema Registry requires that you define a PLATT in the same region and environment of the cluster, even if you create a Private Link for the Dedicated cluster.
• For Enterprise clusters, the same PLATT is automatically used by Schema Registry and vice-versa.

Create a PrivateLink Attachment overview

This walkthrough guides you through the following steps to create a PLATT.

1. Create PLATT/PLATTC:
   1. In Confluent Cloud, create a PrivateLink Attachment as shown in Step 1.
   2. In AWS, create a PrivateLink Attachment Connection, and create a VPC Endpoint linked to the PrivateLink Attachment service.
   3. In AWS Route53, set up a DNS resolution for this endpoint.
2. (Optional step) If your client is not in the VPC, enable the Confluent Cloud Console or Confluent CLI to connect to your private network as shown in Step 2.

Step 1: Create a PrivateLink Attachment

Add the network configuration

The following steps show how to create a PrivateLink Attachment using the Cloud Console.

If a PrivateLink Attachment exists already, you don’t need to create another, because the existing PrivateLink Attachment provides connectivity for the current environment and region.

1. Log in to the Cloud Console at https://confluent.cloud, and navigate to an environment with the Schema Registry you want to use.

2. In the environment details page, click Network management and ensure that For serverless products is selected.

3. Click Add network configuration.

4. In the Add network configuration page, select your cloud service provider and region.

   Note

   Currently, Schema Registry supports only the AWS PrivateLink Attachment.

5. Click Continue.

6. In the Network name textbox, type the name of your PrivateLink Attachment network.

7. Click Add network configuration.

   The environment details page opens and shows your network. Provisioning the network may take a few seconds.

Create a PrivateLink Attachment Connection

1. Click Add connection and follow the instructions in Create a VPC Endpoint.
2. Set the routes by following the instructions in Set up DNS resolution.

When you have successfully created the PLATT, you should see the private connectivity endpoint for Schema Registry on the Cloud Console as shown:

At this point, Schema Registry is ready for use on the privatelink. The next step is optional.

(Optional) Step 2: Connect to the network with Confluent Cloud Console

This step is required only if you want to view and manage schemas on the Cloud Console outside of your VPC. If you don’t connect from a machine within the VPC, you see the following error when attempting to access schemas.

To connect to Confluent Cloud with PrivateLink Attachment, see Use Confluent Cloud with Private Networking. The Resource Metadata access option is not currently supported for Schema Registry.

One way to connect is to set up a reverse proxy:

1. Create an EC2 instance

2. Connect to the instance with SSH

3. Install NGINX

4. Configure Routing Table

5. Set up DNS resolution: point to the Schema Registry regional endpoints you use, as described in Step 6 of Configure a proxy.

   <Public IP Address of VM instance> <schema-registry-private-endpoint>
   

   <schema-registry-private-endpoint> will resemble lsrc-1234.<region>.<cloud>.private.confluent.cloud, for example: lsrc-axliw12p.us-east-2.aws.private.confluent.cloud.

   Find the DNS part of the PrivateLink Attachment by navigating to your environment’s Network management page and finding the DNS domain setting.

   

Once networking is set up in Cloud Console, the interface uses the correct endpoint automatically, and you should be able to view your schemas on the Schema Registry page.

Limitations

• Make sure all your resources in the environment belong to the region where the Schema Registry exists. Confluent Cloud resources outside this region cannot access the Schema Registry with private connectivity enabled.
• Resource Metadata access is not applicable for Schema Registry.
• To view the schemas on the Schema Registry tab you have to connect to the Schema Registry through the private network containing the Schema Registry private endpoints or set up a proxy as described in Step 2.
• Schema Linking is not supported between two fully managed registries on Confluent Cloud.
• Existing registries using public endpoints cannot be configured to use Private Connectivity.
• Schema Registry with private connectivity is not resilient to AWS zonal failures.
• Service level agreements (SLAs) provided by the Governance package will not be applicable.