<a id="usm-network-azure"></a>

# Configure Azure Private Networking for USM Agents

This page describes how to set up Azure Private Link connectivity for the
USM Agent to send Confluent Platform metadata to Confluent Cloud. You configure this
connectivity by creating a gateway and an access point in your Confluent Cloud
environment. For general Confluent Cloud PrivateLink setup that’s not specific to
USM, see [Use Azure Private Link for Serverless Products on Confluent Cloud](../../networking/azure-platt.md#cloud-networking-privatelink-azure-esku).

Unified Stream Manager (USM) requires a secure, private network connection between your
Confluent Platform cluster and Confluent Cloud that uses Azure Private Link. With Azure Private Link,
your Confluent Cloud resources are only accessible from private endpoints in your
VNet, so all metadata is transmitted over a private channel without exposure
to the public internet.

You can create a new private network or use an existing one.

## Private networking resources

Confluent Cloud uses the following private networking resources for registering Confluent Platform clusters.
These resources are regional and do not map to specific availability zones.

* **Ingress PrivateLink Gateway**: A reservation to establish a Private Link
  connection from your virtual network (VNet) to regional services in a
  Confluent Cloud environment.
* **Ingress PrivateLink Access Point**: A registration of a private endpoint
  that’s allowed to connect to a Confluent Cloud environment. An access point
  belongs to a specific gateway.

You can create the gateway and access point by using the Confluent Cloud Console,
Confluent REST API, Confluent CLI, or Terraform.

The process consists of the following steps:

1. [In Confluent Cloud, create a gateway](#usm-azure-gate).
2. [In Azure, create a private endpoint that connects to the gateway’s service](#usm-azure-endpoint).
3. [In Confluent Cloud, create an access point for your endpoint](#usm-azure-access).
4. [Set up your DNS resolution](#usm-azure-dns).

The setup wizard in the Confluent Cloud guides you through this process.

If you have an existing Azure Private Link connection for your Confluent Cloud account and have
configured a private network using these [Azure Private Link instructions](../../networking/azure-platt.md#cloud-networking-privatelink-azure-esku), you can select that
existing network from the wizard and proceed to the next section.

## Requirements and considerations

* You can connect to several Confluent Cloud environments in the same region from a single VNet
  by creating a separate gateway for each environment.
* Cross-region Azure access point resources are not supported.

<a id="usm-azure-gate"></a>

## Create an Ingress PrivateLink Gateway in Confluent Cloud

To establish a private network connection, first create a gateway in the
Confluent Cloud wizard. This process generates the unique service name that you need
to create the private endpoint in your Azure account.

### Confluent Cloud Console

1. On the **Select network configuration** page of the wizard, click **Add network configuration**.
2. In the **Configure gateway** panel, enter the following information:
   * **Gateway name**: A descriptive name for your network connection.
   * **Cloud provider**: Select **Microsoft Azure**.
   * **Region**: Select the Azure region that matches your VNet.
3. Click **Continue**.
4. The next panel displays the **Private Link Service ID** or **Private
   Link Service Alias**. Copy this value to use it in the next step in
   the Azure portal.
5. Keep this Confluent Cloud browser tab open. Confluent Cloud provisions the gateway, and its state changes to `CREATED`.

   A gateway can have one of the following states:
   * `CREATED`: You provisioned the gateway and can now add an access point.
   * `READY`: You created the access point, and it can accept connections.
   * `EXPIRED`: You didn’t provision a valid access point in the allotted time. Create a new gateway to continue.

### Confluent REST API

1. Send a request to create a gateway resource:

   **REST request**
   ```rest
   POST https://api.confluent.cloud/networking/v1/gateways
   ```

   **REST request body**
   ```json
   {
     "spec": {
       "display_name": "<A custom name for the gateway>",
       "config": {
         "kind": "AzureIngressPrivateLinkGatewaySpec",
         "region": "<Azure region of the gateway>"
       },
       "environment": {
         "id": "<The ID of the environment to add the gateway to>"
       }
     }
   }
   ```

   For more information, see [Gateways](https://docs.confluent.io/cloud/current/api.html#tag/Gateways-(networkingv1))
   in the Confluent REST API reference.

### Confluent CLI

Use the following Confluent CLI command to create an Azure gateway:

```bash
confluent network gateway create <gateway-name> <flags>
```

The following command-specific flags are supported:

* `--cloud`: Required. The cloud provider. Set to `azure`.
* `--region`: Required. The Azure region of the gateway.
* `--type`: Required. The type of gateway configuration. Set to
  `ingress-privatelink`.

For more information, see [confluent network gateway create](https://docs.confluent.io/confluent-cli/current/command-reference/network/gateway/confluent_network_gateway_create.html).

The following is an example Confluent CLI command to create a gateway:

```bash
confluent network gateway create my-ingress-gateway \
  --cloud azure \
  --region centralus \
  --type ingress-privatelink
```

### Terraform

Use the [confluent_gateway](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_gateway)
resource from the Confluent Terraform Provider to create a gateway.

An example snippet of Terraform configuration for a gateway:

```terraform
resource "confluent_gateway" "azure_ingress" {
  display_name = "my-gateway"
  environment {
    id = "env-123abc"
  }
  azure_ingress_private_link_gateway {
    region = "centralus"
  }
}
```

<a id="usm-azure-endpoint"></a>

## Create a private endpoint in Azure

In a new browser tab, log in to your Azure portal. Use the **Private Link
Service ID** or **Private Link Service Alias** that you copied in
[Create an Ingress PrivateLink Gateway in Confluent Cloud](#usm-azure-gate) to create and configure the private endpoint.

### Azure Portal

1. On the [Private Endpoint page in the Azure portal](https://portal.azure.com/#view/Microsoft_Azure_Network/PrivateLinkCenterBlade/~/privateendpoints),
   click **+ Create**.
2. In the **Basics** pane, specify the following:
   * **Subscription**: The subscription that contains the VNet you want
     to connect from.
   * **Resource group**: The resource group for this VNet.
   * **Name**: A name for the private endpoint.
   * **Network interface name**: A network interface name.
   * **Region**: The same region as the gateway you created in
     [Create an Ingress PrivateLink Gateway in Confluent Cloud](#usm-azure-gate).
3. Click **Next: Resource**, and specify the following:
   * **Connection method**: Select **Connect to an Azure resource by
     resource ID or alias**.
   * **Resource ID or alias**: Paste in the **Private Link Service ID**
     or **Private Link Service Alias** from [Create an Ingress PrivateLink Gateway in Confluent Cloud](#usm-azure-gate).
4. Click **Next: Virtual Network**, and specify the following:
   * **Virtual network**: The VNet where the private endpoint is
     created.
   * **Subnet**: The subnet where the private endpoint is created.
   * **Network policy for private endpoints**: Select the
     organization-approved or mandated policy. The default is
     **Disabled**.
   * **Private IP configuration**: Select **Dynamically allocate IP
     address**.
5. Click **Next: DNS** and accept the default values.
6. Click **Next: Tags**, and optionally, add tags.
7. Click **Next: Review + create**. Review the details and click
   **Create**.
8. Wait for the Azure deployment to complete, and then note the private
   endpoint resource ID. You use this value in the next step.

### Azure CLI

```bash
az network private-endpoint create \
  --connection-name <connection name> \
  --name <endpoint name> \
  --private-connection-resource-id <resource ID or alias from the gateway> \
  --resource-group <resource group name> \
  --vnet-name <VNet name> \
  --subnet <subnet name> \
  --manual-request true
```

Specify both `--vnet-name` and `--subnet`, or pass the full subnet
resource ID as `--subnet`. A bare subnet name is not sufficient for the
Azure CLI to resolve which VNet it belongs to.

Include `--manual-request true`. Without it, the Azure CLI attempts to
auto-approve the connection against the Private Link Service, which
requires permissions on the Confluent’s subscription that you do not have,
causing the command to fail. With this flag, the connection is created in a
`PENDING` state. Creating the access point in the next step accepts the
connection and transitions both the gateway and the access point to
`READY`.

From the command output, note the private endpoint resource ID. Use this value
in the next step.

### Azure Terraform

Use the [azurerm_private_endpoint](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/private_endpoint)
Azure Terraform Provider resource to create a private endpoint in Azure.

<a id="usm-azure-access"></a>

## Create an Ingress PrivateLink Access Point

Return to Confluent Cloud and create an access point resource.
An access point represents a private endpoint in your VNet.

You don’t need to provide the Private Link Service name. Confluent checks
which Private Link Service is associated with the gateway that has a pending
private endpoint with the given ID.

### Confluent Cloud Console

1. In the **Access points** tab, click **Create access point**.
   1. [Create a private endpoint in Azure](#usm-azure-endpoint) using
      the Private Link Service ID or Alias shown in step 4 on the sliding panel.
   2. In **Private Endpoint ID**, specify the Azure resource ID of the private endpoint you created in the previous step.
   3. In the **Access point name** field, enter a name for the access point.
   4. Click **Create access point**.

      After the private endpoint connection is accepted, the statuses for the gateway and the access point change to `READY`.

### Confluent REST API

1. Send a request to create an access point resource:

   **REST request**
   ```rest
   POST https://api.confluent.cloud/networking/v1/access-points
   ```

   **REST request body**
   ```json
   {
     "spec": {
       "display_name": "<A custom name for the access point>",
       "config": {
         "kind": "AzureIngressPrivateLinkEndpoint",
         "private_endpoint_resource_id": "<The Azure resource ID of your private endpoint>"
       },
       "environment": {
         "id": "<The ID of the environment that has the gateway for this access point>"
       },
       "gateway": {
         "id": "<The ID of the gateway to add the access point to>"
       }
     }
   }
   ```

   For more information, see [Access Points](https://docs.confluent.io/cloud/current/api.html#tag/Access-Points-(networkingv1))
   in the Confluent REST API reference.

### Confluent CLI

Use the following Confluent CLI command to create an access point:

```bash
confluent network access-point private-link ingress-endpoint create <access-point-name> <flags>
```

The following command-specific flags are supported:

* `--cloud`: Required. The cloud provider. Set to `azure`.
* `--gateway`: Required. The ID of the gateway to add the access point to.
* `--private-endpoint-resource-id`: Required. The Azure resource ID of your private endpoint.

For more information, see [confluent network access-point private-link ingress-endpoint create](https://docs.confluent.io/confluent-cli/current/command-reference/network/access-point/private-link/ingress-endpoint/confluent_network_access-point_private-link_ingress-endpoint_create.html).

The following is an example Confluent CLI command to create an access point:

```bash
confluent network access-point private-link ingress-endpoint create my-ingress-access-point \
  --cloud azure \
  --gateway gw-123abc \
  --private-endpoint-resource-id /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Network/privateEndpoints/my-endpoint
```

### Terraform

Use the [confluent_access_point](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_access_point)
Confluent Terraform Provider resource to create an access point.

<a id="usm-azure-dns"></a>

## Set up DNS resolution

Confluent Cloud requires that you set up private DNS records for each access point
pointing its DNS domain to the private endpoint you created.

For access points, when connecting to Confluent Cloud using access-point-specific
hostnames, you must allow public DNS resolution from your network or VNet.
Confluent Cloud advertises these hostnames in the public DNS resolver, and these
hostnames then redirect to match the domains that you input to your private
DNS resolver.

DNS resolution happens in two steps:

1. The Confluent Cloud Global DNS Resolver returns a CNAME for your hostnames,
   removing the `glb` subdomain and converting your access point ID to be a
   subdomain.

   For example, with the given hostname:
   ```text
   api-<accessPointId>.<region>.azure.accesspoint.glb.confluent.cloud
   ```

   The CNAME returned is:
   ```text
   api.<accessPointId>.<region>.azure.accesspoint.confluent.cloud
   ```
2. The CNAME then resolves to your private endpoint based on the private
   DNS configuration.

To use an Azure Private DNS Zone as your private DNS resolver:

1. In Confluent Cloud, verify that the status of the access point is `READY`.
2. In Confluent Cloud, open the newly created gateway to get the DNS domain value of
   Confluent Cloud.

   Take the FrontDoor URL shown in the Confluent Cloud Console and remove `.glb`
   and the `api-` prefix. Use the resulting
   `<accessPointId>.<region>.azure.accesspoint.confluent.cloud` as the
   **Name** for your Azure Private DNS Zone.

   For example, if the FrontDoor URL is
   `api-ap123.centralus.azure.accesspoint.glb.confluent.cloud`,
   the Private DNS Zone name is
   `ap123.centralus.azure.accesspoint.confluent.cloud`.
3. In the [Private DNS Zones page in the Azure portal](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.Network%2FprivateDnsZones),
   create a Private DNS Zone with the following settings:
   * **Subscription** and **Resource group**: The subscription and resource
     group that you used when you created the VNet.
   * **Name**: Enter the Confluent Cloud DNS domain value that you copied in the previous step.

   Click **Review + create**, and then click **Create**.
4. Create a DNS record for the Private DNS Zone you created in the previous step.

   This record is regional DNS and is used for all the target Confluent Cloud resources
   in the region.
   1. Go to the Private DNS Zone resource you created, and click **+ Record Set**.
   2. Specify the following values:
      * **Name**: `*`

        Enter `*` as the subdomain name.

        The record name consists of the subdomain and the DNS domain name. The
        DNS domain name is filled in with the Confluent Cloud DNS domain value you
        specified when you created the Private DNS Zone in the previous step.

        If you are creating DNS resolution for Schema Registry for a single VNet connecting
        to several Schema Registry clusters in the same region across different
        environments, enter the id of the Schema Registry, `lsrc-xxxxx` in the **Name**
        field to connect to a specific Schema Registry.
      * **Type**: Select `A`.
      * **TTL** and **TTL unit**: `1` `Minute`.
      * **IP address**: Enter the IP address of the private endpoint that you
        created in [Create a private endpoint in Azure](#usm-azure-endpoint).

        You can find this value under the private endpoint’s associated
        **Network interface** in the Azure portal.

      #### NOTE
      In Confluent Cloud with private linking, Kafka broker names you retrieve from the
      metadata are not static. Do not hardcode the broker names in DNS records.
   3. Click **OK**.
5. Attach the Private DNS Zone to the VNets where clients or applications are
   present.
   1. Go to the Private DNS Zone resource and click **Virtual network links**
      under **Settings**.
   2. Click **+ Add**.
   3. Specify the required values and click **OK** to create a virtual network
      link.

## What’s next

* [Configure a Service Account for the USM Agent](usm-service-account.md#usm-sa)

## Related content

* [Unified Stream Manager in Confluent Cloud](../overview.md#cloud-usm-overview)
* [Register Your Confluent Platform Cluster](overview.md#cloud-usm-register)
* [Set Up a Payment Method](payment-method.md#usm-payment)
* [Configure AWS Private Networking for USM Agents](usm-network.md#usm-network)
* [Configure a Service Account for the USM Agent](usm-service-account.md#usm-sa)
* [Deploy the USM Agent](deploy-agent.md#usm-agent)
* [Complete the Cluster Registration](complete-registration.md#complete-registration)