Quick Start for Schema Management on Confluent Cloud

You can use Confluent Cloud Schema Registry to manage schemas in Confluent Cloud. You enable a single Schema Registry per Confluent Cloud environment. Schema Registry is accessed over port 443.

You can create and edit schemas in a schema editor and associate them with Kafka topics.

Important

Log in to the Confluent Cloud UI

  1. Log in to Confluent Cloud at https://confluent.cloud.
  2. Select an environment or create a new one.

Enable Schema Registry for Confluent Cloud

  1. Select an environment.

  2. Click the Schema Registry tab.

  3. Click Set up on my own (or choose the guided path).

  4. Choose a cloud provider and geography (which will map to a region under the hood).

    • Select Amazon Web Services (AWS), Google Cloud Platform (GCP), or Azure
    • Choose a Region. US, Europe, and APAC are the available choices.
    • Click Enable Schema Registry.
    Screenshot of Cloud provider and region settings

    Tip

    For each cloud provider, geographies are mapped under the hood to the following regions:

    • GCP US: us-central1
    • GCP EU: europe-west3
    • GCP APAC: australia-southeast1
    • AWS US: us-east-2
    • AWS EU: eu-central-1
    • AWS APAC: ap-southeast-2
    • Azure US: westus2
    • Azure EU: westeurope
    • Azure APAC: southeastasia
  5. Review your settings, and click Continue to create the Schema Registry with these cloud provider and region settings.

    This creates a Schema Registry to serve all clusters in the current environment. The environment level Schema Registry settings page is displayed.

    Screenshot of Schema Registry settings

    An empty schemas list is shown, indicating no schemas have yet been created.

    Screenshot of empty schemas list at start of enabling Schema Registry for a Confluent Cloud environment

Create an API Key for Confluent Cloud Schema Registry

To use Confluent Cloud Schema Registry for managing Kafka clusters, you need an API key specific to Schema Registry.

Note

  • The API key for Confluent Cloud Schema Registry is distinct from the API key you created for Kafka clusters in the same environment, per the Quick Start for Apache Kafka using Confluent Cloud.
  • Confluent Cloud provides one Schema Registry per environment, and each environment can contain multiple Kafka clusters. You need an API key/secret pair for each Kafka cluster and another for the Schema Registry cluster.
  1. In the environment for which you want to set up Schema Registry, click the Schema Registry tab, then click Edit on the API credentials card.

    API credentials setup for Schema Registry
  2. Click Create key to create a new Schema Registry API key.

    Screenshot of a new Schema Registry API access options
  3. When the API Key and API Secret are saved, click the checkbox next to I have saved my API key and secret and am ready to continue, and click Continue.

    Screenshot of a new Schema Registry key and secret in Confluent Cloud

    Your new Schema Registry key is shown on the Schema Registry API access key list.

    Screenshot of Schema Registry in Confluent Cloud listing newly created key

Schema Registry API Keys for Multi-Tenant Clusters

Confluent Cloud supports one Schema Registry per environment. In multi-tenant deployments, the registries for different Kafka clusters in different environments will have the same URL as the Confluent Cloud Schema Registry. However, Confluent Cloud uses the Schema Registry API keys to store schemas in different logical clusters.

Therefore, if you have a Schema Registry API key (prod-sr-api-1) for Schema Registry in production 1 cluster in environment A and a Schema Registry API key (for example, prod-sr-api-2) for a Schema Registry in production 2 cluster in Environment B, when a Kafka producer sends messages with prod-sr-api-1, Confluent Cloud appropriately registers a schema in Schema Registry in prod-1 .

Tip

To learn more about other environment level schema management options available on the Environment settings page, see Configure and Manage Schemas for an Environment.

Create a Topic in Confluent Cloud

You may already have topics available on this cluster, having worked through the prerequisite Quick Start for Apache Kafka using Confluent Cloud, but create a new topic designed to test out Schema Registry.

  1. Select the environment.

    Tip

    If you have only one environment, default, it will be selected already.

  2. Select the cluster.

  3. Click Topics on the left menu, then click Create topic.

    Screenshot of new topic dialog in Confluent Cloud

    Name the new topic employees, and click Create with defaults to add it.

Create a Schema

You can create schemas for topics from the web UI, or from the Confluent Cloud CLI. This section shows you how to create a schema from the Confluent Cloud CLI, just to provide a 360 view of cloud tools and demonstrate how you can use the web UI and the CLI to interact with the same entities, such as environments, clusters, topics, and schemas.

If you need more help with the basic CLI commands, type ccloud --help, ccloud schema-registry --help to drill down, or see Quick Start for Apache Kafka using Confluent Cloud and the Confluent Cloud command reference.

  1. Log in to the Confluent Cloud CLI.

    ccloud login
    

    Follow the prompts to enter your credentials (email and password).

    Upon login, Confluent Cloud indicates your current environment.

    Enter your Confluent credentials:
    Email:
    Password:
    
    Using environment t2814 ("my-cloud-demos")
    
  2. Select the environment where you created the employees topic. (It may already be selected upon login.)

    In order to verify this, use ccloud environment list to get the environment IDs, followed by ccloud environment use <ID> to select the environment you want (if it is not already the current environment).

    For example:

    ccloud environment list
    

    Your output should resemble:

          Id      |        Name
    +-------------+--------------------+
      * t2703     | default
        env-vnywz | ccloud-demo
        env-qzrg2 | data-lineage-demo
        env-250o2 | my-new-environment
    

    To use “ccloud-demo”:

    ccloud environment use env-vnywz
    

    Tip

    • To list environments: ccloud environment list
    • To select an environment: ccloud environment use <environment ID>
    • To list clusters: ccloud kafka cluster list
    • To select a cluster: ccloud kafka cluster use <cluster ID>
    • For the full list of ccloud commands see Confluent Cloud command reference.
  3. Create a file containing the following JSON and name it employees.json.

    {
       "type" : "record",
       "namespace" : "Example",
       "name" : "Employee",
       "fields" : [
          { "name" : "Name" , "type" : "string" },
          { "name" : "Age" , "type" : "int" }
       ]
    }
    
  4. Create a schema that uses employees.json.

    ccloud schema-registry schema create --subject employees-value --schema employees.json --type AVRO
    

    For example:

    ccloud schema-registry schema create --subject employees-value --schema employees.json --type AVRO
    
    Enter your Schema Registry API Key:
    xxxx
    Enter your Schema Registry API Secret:
    xxxxyyyyzzzz
    Successfully registered schema with ID: 100001
    

    Tip

View the Topic and Associated Schema

Return to the Confluent Cloud UI to view the new schema for the employees topic.

  1. Navigate to the topics list for your cluster.

    Screenshot of sample topic in Confluent Cloud
  2. Click the employees topic you created in a previous step.

  3. Click Schema.

    Screenshot of sample schema in Confluent Cloud

    And there you have it! You have a new schema (employees-value) for the employees topic in Confluent Cloud Schema Registry.

Configure and Manage Schemas for an Environment

Several tasks related to schemas on Confluent Cloud are managed at the environment level. This includes creating and managing Schema Registry API keys along with options, as described below, to view and search schemas, monitor usage, and set a compatibility mode for schemas.

View and Search All Schemas in an Environment

To view a searchable list of all schemas available in a Confluent Cloud environment:

  1. Select an environment.

  2. Click the Schemas Registry tab.

  3. On the Schemas card, click View and manage schemas.

    Screenshot of schemas card for an environment Confluent Cloud

    The schemas list is shown.

    Screenshot of schemas list in Confluent Cloud
    • You can use Search to find schemas in longer lists
    • Click a schema to drill down for more options, including view, download, duplicate, or delete a schema; navigate to compatibility settings; traverse the schema version history, view each version, and compare versions.
    • Also shown at the top of this page is the number of schemas available in the environment compared to the usage limit. See the next section to learn more.

View Allowed Usage for Schemas on Confluent Cloud

Schema Registry in Confluent Cloud limits the number of schema versions supported in the registry for basic clusters, standard clusters, and dedicated clusters as described in Confluent Cloud Cluster Types. You can view the number of schemas you have in an environment compared to the allowed usage limit.

  1. Select an environment.

  2. Click the Schemas Registry tab.

  3. On the Schemas card, click View and manage schemas.

    The schemas list is shown. The number of schemas in use compared to the number of available schemas is shown at the top.

    Screenshot of schemas list in Confluent Cloud

To learn more about how to manage schema usage on Confluent Cloud, see how to free up space for new schemas.

View and Edit Global Compatibility Settings on Schemas

The default compatibility mode is Backward. From the environment level Schema Registry settings, you can change the mode, which will apply globally to all schemas in an environment. You can also change compatibility settings for a schema on a specific topic (at the subject level). Subject-level compatibility settings override the global settings.

Caution

If you change the compatibility mode of existing schemas already in production use, be aware of any possible breaking changes to your applications.

  1. Select an environment.

  2. Click the Schema Registry tab.

  3. Click Edit on the Compatibility setting card.

    Screenshot of schema compatibility settings on Confluent Cloud

    The Compatibility settings are shown.

    Screenshot of schema compatibility settings on Confluent Cloud
  4. Select a mode option:

    Descriptions indicate the compatibility behavior for each option. For more information, including the changes allowed for each option, see Schema Evolution and Compatibility.

  5. Click Save.

Use curl to View and Manage Schemas

You can also use curl commands to view and manage schemas on Confluent Cloud.

Schema Registry on Confluent Cloud requires that you pass the API Key and Secret with the --user (or -u) flag. For example, to view all subjects in the registry:

curl --user <schema-registry-api-key>:<schema-registry-api-secret> \
<schema-registry-url>/subjects

For more information about using curl commands with Schema Registry, see Schema Registry API Usage Examples. With the exception of this one same example, the examples are for on-premises Schema Registry but many of the commands are relevant to Confluent Cloud, once you know how to pass your API Key and Secret as shown here.