Manage Schemas in Confluent Platform and Control Center

Use the Schema Registry feature in Control Center to create, view, edit, compare, and download schemas for Confluent Platform topics.

You can:

• create, edit, and view schemas

• compare schema versions

• download schemas

The Schema Registry performs validations and compatibility checks on schemas.

Tip

• Review the step-by-step tutorial for using Schema Registry.

• For comprehensive information on Schema Registry, see the Schema Registry Documentation.

The Schema Registry feature in Control Center is enabled by default. Disabling the feature disables both viewing and editing of schemas.

Prerequisites

If you don’t have a Confluent Platform instance running, you can still follow these workflows. Use the single-node developer test version of Confluent Platform with a separately installed Control Center.

1. Install Confluent Platform 8 or later on your local machine. To do this, see Install Confluent Platform using ZIP and TAR Archives, and complete the first two steps, Get the software and Configure CONFLUENT_HOME and PATH.

2. Run the following commands to check that you installed the Confluent CLI and that it’s up to date.

   confluent --version
   

   confluent update
   

3. Follow the steps to download, extract, and configure Control Center to run a Single-node manual installation using the archives (not Docker). Control Center runs in its own directory, $CONTROL_CENTER_HOME, at the same level as $CONFLUENT_HOME. In previous versions of Confluent Platform, Control Center was part of Confluent Platform and resided in a subdirectory under $CONFLUENT_HOME.

4. Start the Confluent Platform cluster by running the following command, preferably in its own terminal window.

   confluent local services start
   

5. Configure the $CONTROL_CENTER_HOME environment variable for Control Center.

   1. Change to your local Control Center install directory.

   2. Run the following command to set $CONTROL_CENTER_HOME.

      export CONTROL_CENTER_HOME=`pwd`
      

6. Start Control Center by running the following command, preferably in its own terminal window.

   confluent local services control-center start
   

7. Open http://localhost:9021/ in your browser to verify that Control Center is running.

Create a schema for a topic in Control Center

Create key and value schemas. Value schemas are typically created more frequently than a key schema. These best practices are recommended:

• Provide default values for fields to facilitate backward-compatibility if pertinent to your schema.

• Document at least the more obscure fields for human-readability of a schema.

Create a topic

If you don’t already have a topic, create one with either of the following methods.

• Create a topic using the Confluent CLI:

  kafka-topics --create --topic my-transactions --bootstrap-server localhost:9092
  

• Create a topic using the Control Center web interface:

  1. Navigate to the Topics page in the Control Center web interface at http://localhost:9021/.

  2. Click Add a topic, and rename it from the default name to my-transactions.

  3. Click Create with defaults.

Create a topic value schema

1. Select a cluster.

2. Click Topics on the menu.

3. Select a topic.

   

   The topic overview page appears.

   

4. Click the Schema tab.

   You are prompted to set a message value schema.

   

5. Click Set a schema. The Schema editor appears pre-populated with the basic structure of an Avro schema as a starting point.

   

6. For this example, accept the default schema context. You can also create a new context or select an existing one. To learn more, see Work with schema contexts.

   

7. Select a schema format type:

   • Avro

   • JSON

   • Protobuf

   

   To try the code examples in the next steps, choose Avro.

   Tip

   To learn more about each of the schema types, see Supported formats.

8. Enter the schema in the editor:

   • name: Enter a name for the schema if you do not want to accept the default, which is determined by the subject name strategy. The default is schema_type_topic_name. Required.

   • type: Either record, enum, union, array, map, or fixed. (The type record is specified at the schema’s top level and can include multiple fields of different data types.) Required.

   • namespace: Fully-qualified name to prevent schema naming conflicts. String that qualifies the schema name. Optional but recommended.

   • fields: JSON array listing one or more fields for a record. Required.

     Each field can have the following attributes:

     • name: Name of the field. Required.

     • type: Data type for the field. Required.

     • doc: Field metadata. Optional but recommended.

     • default: Default value for a field. Optional but recommended.

     • order: Sorting order for a field. Valid values are ascending, descending, or ignore. Default: Ascending. Optional.

     • aliases: Alternative names for a field. Optional.

   Copy and paste the following example schema.

   {
    "type": "record",
    "namespace": "my.examples",
    "name": "Payment",
    "fields": [
      {
       "name": "id",
       "type": "string"
      },
      {
       "name": "amount",
       "type": "double"
      }
              ]
   }
   

   

9. In edit mode, you can:

   • Validate the schema for syntax and structure before you create it.

   • Add schema references with a guided wizard.

   • Choose a context for the schema.

10. Click Create.

    • If the schema is valid, the banner area briefly displays the Schema updated message.

    • If the schema isn’t valid, the banner area displays an Input schema is an invalid Avro schema error.

    If applicable, repeat the procedure for the topic key schema.

Work with schema contexts

A schema context is a grouping of subject names and schema IDs. Contexts give you more flexibility in subject naming, schema IDs, and how clients reference schemas.

Specify schema contexts

When you create a schema in Control Center, you can accept the default context or specify a custom context. To create a schema under a new context:

1. Create a topic or choose an existing topic.

2. Click the Schema tab.

3. With Value selected, click Set a schema.

4. Add the schema with the appropriate schema format selected. This example shows an Avro schema.

5. Next to the Schema context field, select the drop-down menu and choose Define a new context.

   

6. Enter a name for the new context in the dialog, and then click Create on the confirmation dialog.

   

Control Center creates the schema under the new context.

Learn more about schema contexts

Schema Registry lets you group schemas logically by specifying schema contexts. By default, schemas live in the default context. When you give schemas qualified names, you group them into sub-registries with context-specific paths. Schemas with the same subject names and IDs can then exist as unique entities in different contexts. Qualified names also let you provide specific contexts for different clients. Schema Registry uses schema contexts extensively for Schema Linking, and you can also use them independently of that feature.

Compatibility applies per context, not globally across all contexts. To learn more, see Schema Evolution and Compatibility for Schema Registry on Confluent Platform.

To learn more about schema contexts, see Contexts within Schema Linking. The Schema Linking Quick Start includes examples of working with unqualified subjects in the default context and qualified subjects in named contexts.

Work with schema references

Use the wizard to locate available schemas and versions and add a reference to another schema.

The Reference name must match the target schema, based on the guidelines for your schema format:

• In JSON Schema, the name is the value on the $ref field of the referenced schema

• In Avro, the name is the value on the type field of the referenced schema

• In Protobuf, the name is the value on the Import statement referenced schema

Add a schema reference to the current schema in the editor

1. Locate the schema you want to reference and get its reference name.

2. Click Add reference.

3. Enter a Reference name that follows the preceding rules.

4. Select the schema from the Subject list.

5. Select the Version of the schema to use.

6. Click Validate to check whether the reference passes.

7. Click Save to save the reference.

For example, to reference the schema for the my-transactions topic (my-transactions-value) from the widget schema, configure a reference to type record.

To learn more, see Schema references in the schema formats developer documentation.

View, edit, or delete schema references for a topic

Existing schema references appear on editable versions of the schema where you configured them.

1. Go to a topic, such as the widget-value schema associated with the widget topic in the previous example.

2. Click into the editor as if to edit the schema.

   If the schema has references to other schemas, they appear in the Schema references list below the editor.

   You can also add references, change existing references, or delete references from this view.

Create a topic key schema

1. Click the Key option. You are prompted to set a message key schema.

   

2. Click Set a schema.

3. Choose Avro format, or delete the sample formatting and paste in a string UUID.

4. Enter the schema into the editor and click Save.

   Copy and paste the following example schema, and save it.

   {
     "namespace": "io.confluent.examples.clients.basicavro",
     "name": "key_my_transactions",
     "type": "string"
   }
   

   

5. In edit mode, you can:

   • Validate the schema for syntax and structure before you create it.

   • Add schema references with a guided wizard.

Best practices and pitfalls for key values

Kafka messages are key-value pairs. You can serialize message keys and message values independently. For example, the value might use an Avro record, while the key is a primitive such as string or integer. Message keys, if used, are typically primitives. How you set the key depends on your implementation requirements.

As a best practice, keep key value schema complexity low. Use a simple, non-serialized data type, such as a string UUID or long ID. You can also use an Avro record that doesn’t use maps or arrays as fields. Don’t use Protobuf messages and JSON objects for key values. Avro doesn’t guarantee deterministic serialization for maps or arrays, and Protobuf and JSON schema formats don’t guarantee deterministic serialization for any object. Using these formats for key values breaks topic partitioning. If you decide to use a complex format for a key value schema, set auto.register.schemas=false. This setting prevents the registration of new valid and compatible schemas that break your partitioning because of the complex key value format. To learn more, see Auto schema registration in the On-Premises Schema Registry Tutorial, and Partitioning gotchas in the Confluent Community Forum.

For detailed examples of key and value schemas, see the discussion under Formats, Serializers, and Deserializers.

View a schema in Control Center

View the schema details for a specific topic.

1. Select a cluster from the navigation bar.

2. Click the Topics menu. The Manage Topics Using Control Center for Confluent Platform appears.

3. Select a topic.

   

   The topic overview page appears.

4. Click the Schema tab.

   The Value schema is displayed by default.

   

   Tip

   Control Center can reorder the metadata of a schema in its display. Compared with the example in the previous section, the schema name, namespace, and record type appear below the field definitions. The reordering is only a display artifact, and the schema definition is the same.

5. Click the Key tab to view the key schema if present.

Edit a schema in Control Center

Edit an existing schema for a topic.

1. Select a cluster from the navigation bar.

2. Click the Topics menu. The Manage Topics Using Control Center for Confluent Platform appears.

3. Select a topic.

4. Click the Schema tab.

5. Select the Value or Key tab for the schema.

6. Click anywhere in the schema to enable edit mode and make changes in the schema editor.

   To follow along with the example:

   1. Select the topic my-transactions.

   2. Click Schema, and then select the Value tab.

   3. Edit the schema by pasting the following definition for a new region field after the id and amount fields. Precede the new definition with a comma.

      {
       "name": "region",
       "type": "string",
       "default": ""
      }
      

   The new region field includes a default value, which makes it backward compatible. With the default value, consumers can use the new schema to read data from producers that use the older schema without the region field.

7. Click Save.

   • If the update is valid and compatible with prior versions, Control Center updates the schema and increments the version count. This outcome assumes a backward-compatible mode. You can compare the different versions of a schema.

     

   • If the schema update is invalid or incompatible with an earlier schema version, Control Center displays an error.

     The following example adds another new field, country, without a default value.

     

Tip

You can also add schema references as a part of editing a schema, as described in Work with schema references.

Compare schema versions in Control Center

Compare versions of a schema to view its evolutionary differences.

1. Select a cluster from the navigation bar.

2. Click the Topics menu. The Manage Topics Using Control Center for Confluent Platform appears.

3. Select a topic.

4. Click the Schema tab.

5. Select the Key or Value tab for the schema.

6. Select Version history from the inline menu.

   

   The version menu shows the current version number of the schema.

   

7. Select the Turn on version diff check box.

8. Select the versions to compare from each version menu. Control Center highlights the differences.

   

Change the compatibility mode of a schema in Control Center

The default compatibility mode is Backward. The mode can be changed for the schema of any topic if necessary.

Caution

If you change the compatibility mode of an existing schema already in production use, be aware of any possible breaking changes to your applications.

1. Select a cluster from the navigation bar.

2. Click the Topics menu. The Manage Topics Using Control Center for Confluent Platform appears.

3. Select a topic.

4. Click the Schema tab.

5. Select the Key or Value tab for the schema.

6. Select Compatibility setting from the inline menu.

   The compatibility settings appear.

   

7. Select a mode option:

   • Backward (Confluent Schema Registry default)

   • Transitive backward

   • Forward

   • Transitive forward

   • Full

   • Transitive full

   • None (not recommended)

   Descriptions indicate the compatibility behavior for each option. For more information, including the changes allowed for each option, see Schema Evolution and Compatibility for Schema Registry on Confluent Platform.

8. Click Save.

Download a schema from Control Center

1. Select a cluster from the navigation bar.

2. Click the Topics menu. The Manage Topics Using Control Center for Confluent Platform appears.

3. Select a topic.

4. Click the Schema tab.

5. Select the Key or Value tab for the schema.

6. Click Download. Control Center downloads a schema JSON file for the topic into your Downloads directory.

   Example filename: schema-transactions-v1-Ry_XaOGvTxiZVZ5hbBhWRA.json

   Example contents:

   {"subject":"transactions-value","version":1,"id":2,"schema":"{\"type\":\"record\",\"name\":\"Payment\",
   \"namespace\":\"io.confluent.examples.clients.basicavro\",
   \"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"amount\",\"type\":\"double\"},
   {\"name\":\"region\",\"type\":\"string\"}]}"}
   

   This is the first version of the schema, and it has an id of 2. The schema is escaped JSON. A backslash precedes double-quotes.

Troubleshoot error “Schema Registry is not set up”

If Control Center shows the “Schema Registry is not set up” error when you try to access a topic schema, make sure that Schema Registry is running. Then verify that the Schema Registry listeners configuration matches the Control Center confluent.controlcenter.schema.registry.url configuration. Also check the HTTPS configuration parameters.

For more information, see A schema for message values has not been set for this topic and the start-up procedures for Quick Start for Confluent Platform or Install Confluent Platform On-Premises, depending on which one you use to run Confluent Platform.

Enable and disable Schema Registry in Control Center

Control Center enables the feature that lets you work with schemas by default. You can turn off the feature if your organization doesn’t want any users to access it. After you turn off the feature, the Topics Schema menu and the Schema tab no longer appear in the Control Center UI. Users can’t view or edit schemas.

To disable the edit schema feature in Control Center:

1. Set the confluent.controlcenter.schema.registry.enable option in your control-center.properties file to false.

   confluent.controlcenter.schema.registry.enable=false
   

   Note

   Make the change in the Control Center properties files that you configured for your environments, such as control-center-dev.properties or control-center-production.properties. The properties files are under $CONTROL_CENTER_HOME, where you installed Control Center.

2. Restart Control Center and pass in the properties file so the configuration takes effect:

   ./bin/control-center-stop
   ./bin/control-center-start ../etc/confluent-control-center/control-center.properties
   

   Tip

   If you use a Confluent Platform development environment with

   confluent local, stop Control Center and then start it as follows:

   |confluent_local_stop_control_center|
   |confluent_local_stop_control_center| ../etc/confluent-control-center/control-center-dev.properties
   

To enable the feature again, set the option back to true and restart Control Center with the updated properties file.

Enable multi-cluster Schema Registry

Confluent Platform supports running more than one schema registry and associating a unique Schema Registry with each Kafka cluster in multi-cluster environments.

Scaling up schema registries along with Kafka clusters is useful for evolving businesses. It also supports data governance, organization, and management across departments in large enterprises.

When multi-cluster Schema Registry is configured and running, you can create and manage schemas for topics in Control Center as usual.

Configuration properties and files

You can specify more than one Schema Registry cluster with confluent.controlcenter.schema.registry.{name}.url in the Control Center properties file. To use a Schema Registry cluster identified this way, add or verify these broker and Control Center configurations.

Kafka exposes the /v1/metadata/schemaRegistryUrls endpoint, which returns the confluent.schema.registry.url field from the Kafka broker configurations. Control Center uses this field to look up the registries. To use it, configure unique listener endpoints for each cluster:

• In the broker server.properties files (unique for each Kafka cluster), specify the REST endpoint with the confluent.http.server.listeners field, which defaults to http://0.0.0.0:8090.

• In the appropriate Control Center properties file, use confluent.controlcenter.streams.cprest.url to define the REST endpoint for controlcenter.cluster.

• For more clusters, define REST endpoints using confluent.controlcenter.kafka.{name}.cprest.url. Keep this consistent with the Kafka cluster name in other Control Center configurations, such as confluent.controlcenter.kafka.{name}.bootstrap.servers.

A minimal configuration touches the following files and includes settings for these properties. The names and ports are examples:

Control Center properties file

The Control Center Configuration Examples for Confluent Platform file includes:

• confluent.controlcenter.schema.registry.url=http://localhost:8081

• confluent.controlcenter.schema.registry.sr-1.url=http://localhost:8082

• confluent.controlcenter.streams.cprest.url=http://localhost:8090

• confluent.controlcenter.kafka.AK1.cprest.url=http://localhost:8091

See Control Center configuration reference for a full description of confluent.controlcenter.schema.registry.url.

Broker configuration file for the Control Center cluster

The Kafka broker configuration file for controlcenter.cluster, such as server0.properties, includes:

• confluent.http.server.listeners=http://localhost:8090

• confluent.schema.registry.url=http://localhost:8081

Broker configuration file for the Kafka cluster

The Kafka broker configuration file for AK1, server1.properties, includes:

• confluent.http.server.listeners=http://localhost:8091

• confluent.schema.registry.url=http://localhost:8082

With these configurations, editing the schema through the Control Center UI connects to http://localhost:8081 for controlcenter.cluster and http://localhost:8082 for AK1.

Defaults and fallback

If confluent.schema.registry.url fields are not specified for any brokers, the confluent.controlcenter.schema.registry.url Schema Registry URL is applied. For example, if the Schema Registry URL was not provided for AK1, AK1’s associated Schema Registry cluster would also be specified at http://localhost:8081. If confluent.controlcenter.schema.registry.url is not explicitly specified in the Control Center properties file, it defaults to http://localhost:8081.

Example

The following example shows a functional multi-cluster Schema Registry setup with two Kafka clusters connected to Control Center. One is the controlcenter.cluster and the other is AK1, each with one broker. The example instructions refer to the location of your Confluent Platform installation as $CONFLUENT_HOME.

To run the example, copy the default configuration files to new files with the example filenames. Add or change properties as shown, and run the components as described in Run the example.

Important

Confluent deprecated ZooKeeper for new deployments in Confluent Platform 7.5 and recommends KRaft mode for new deployments. To learn more about running Kafka in KRaft mode, see KRaft Overview for Confluent Platform, KRaft Configuration for Confluent Platform, the Platform Quick Start, and Settings for other Kafka and Confluent Platform components. The following example provides KRaft (combined mode) and ZooKeeper configurations. Another example of running multi-cluster Schema Registry in KRaft mode is shown in the Schema Linking Quick Start for Confluent Platform. KRaft combined mode is for local experimentation only and isn’t supported by Confluent.

Key configurations

KRaft mode

The example properties files are based on the defaults. Copy the default properties files to use as a basis for the specialized versions shown here. The example assumes the new files are in the same directories as the originals.

• KRaft server file is in $CONFLUENT_HOME/etc/kafka/server.properties for KRaft combined mode. KRaft combined mode is for local experimentation only and isn’t supported by Confluent. Copy this to create server0.properties and server1.properties.

• Schema Registry properties file is $CONFLUENT_HOME/etc/schema-registry/schema-registry.properties. Copy this to create schema-registry0.properties and schema-registry1.properties.

• Control Center properties file is $CONFLUENT_HOME/etc/confluent-control-center/control-center-dev.properties. Copy this to create $CONFLUENT_HOME/etc/confluent-control-center/control-center-multi-sr.properties. In addition to the configurations shown for this file, comment out the zookeeper.connect line because it doesn’t apply in this mode.

File	Properties
server0.properties	node.id=1 listeners=PLAINTEXT://:9092, CONTROLLER://:9094 log.dirs=/tmp/kraft-combined-logs (fresh logging directory) confluent.metrics.reporter.bootstrap.servers=localhost:9092 The following configurations are specific to multi-cluster Schema Registry setup for this broker: confluent.http.server.listeners=http://0.0.0.0:8090 confluent.schema.registry.url=http://localhost:8081
server1.properties	node.id=1 listeners=PLAINTEXT://:9093, CONTROLLER://:9095 og.dirs=/tmp/kraft-combined-logs-1(fresh logging directory) confluent.metrics.reporter.bootstrap.servers=localhost:9093 The following configurations are specific to multi-cluster Schema Registry setup for this broker: confluent.http.server.listeners=http://0.0.0.0:8091 confluent.schema.registry.url=http://localhost:8082
schema-registry0.properties	listeners=http://0.0.0.0:8081 kafkastore.bootstrap.servers=localhost:9092
schema-registry1.properties	listeners=http://0.0.0.0:8082 kafkastore.bootstrap.servers=localhost:9093 kafkastore.topic=_schemas1 (with both Schema Registry clusters on localhost, this value must be different from the default _schemas in schema-registry0.properties so that the registries do not overwrite each other) schema.registry.group.id=schema-registry-dest (with both Schema Registry clusters on localhost, this value must be different from the group ID used by schema-registry0.properties, which takes the default ID schema-registry)
control-center-multi-sr.properties	bootstrap.servers=localhost:9092 confluent.controlcenter.kafka.AK1.bootstrap.servers=localhost:9093 confluent.controlcenter.streams.cprest.url=http://0.0.0.0:8090 confluent.controlcenter.schema.registry.url=http://localhost:8081 confluent.controlcenter.kafka.AK1.cprest.url=http://0.0.0.0:8091 confluent.controlcenter.schema.registry.SR-AK1.url=http://localhost:8082 The configurations for cpcrest.url, confluent.controlcenter.kafka.AK1.cprest.url, and confluent.controlcenter.schema.registry.SR-AK1.url are new properties, specific to multi-cluster Schema Registry.

Tip

The two registries are colocated on localhost. The values for kafkastore.topic and schema.registry.group.id must be unique for each Schema Registry properties file. If the Schema Registry clusters were on different hosts, you wouldn’t need to make these changes.

ZooKeeper mode

The example properties files are based on the defaults. Copy the default properties files to use as a basis for the specialized versions shown here. The example assumes the new files are in the same directories as the originals.

• ZooKeeper zookeeper.properties file is in $CONFLUENT_HOME/etc/kafka/zookeeper.properties. Copy this to create zookeeper0.properties and zookeeper1.properties.

• KRaft server.properties file is in $CONFLUENT_HOME/etc/kafka/. Copy this to create server0.properties and server1.properties.

• Schema Registry properties file is $CONFLUENT_HOME/etc/schema-registry/schema-registry.properties. Copy this to create schema-registry0.properties and schema-registry1.properties.

• Control Center properties file is $CONFLUENT_HOME/etc/confluent-control-center/control-center-dev.properties. Copy this to create $CONFLUENT_HOME/etc/confluent-control-center/control-center-multi-sr.properties.

File	Properties
zookeeper0.properties	dataDir=/tmp/zookeeper/zk-0 (fresh logging directory) clientPort=2181 (same as zookeeper.properties)
zookeeper1.properties	dataDir=/tmp/zookeeper/zk-1(fresh logging directory) clientPort=2182 (since zookeeper0 is already using 2181)
server0.properties	broker.id=0 listeners=PLAINTEXT://:9092 log.dirs=/tmp/kafka-logs-bk-0 (fresh logging directory) zookeeper.connect=localhost:2181 confluent.metrics.reporter.bootstrap.servers=localhost:9092 The following configurations are specific to multi-cluster Schema Registry setup for this broker: confluent.http.server.listeners=http://0.0.0.0:8090 confluent.schema.registry.url=http://localhost:8081
server1.properties	broker.id=1 listeners=PLAINTEXT://:9093 log.dirs=/tmp/kafka-logs-bk-1(fresh logging directory) zookeeper.connect=localhost:2182 confluent.metrics.reporter.bootstrap.servers=localhost:9093 The following configurations are specific to multi-cluster Schema Registry setup for this broker: confluent.http.server.listeners=http://0.0.0.0:8091 confluent.schema.registry.url=http://localhost:8082
schema-registry0.properties	listeners=http://0.0.0.0:8081 kafkastore.bootstrap.servers=localhost:9092
schema-registry1.properties	listeners=http://0.0.0.0:8082 kafkastore.bootstrap.servers=localhost:9093 kafkastore.topic=_schemas1 (with both Schema Registry clusters on localhost, this value must be different from the default _schemas in schema-registry0.properties so that the registries do not overwrite each other) schema.registry.group.id=schema-registry-dest (with both Schema Registry clusters on localhost, this value must be different from the group ID used by schema-registry0.properties, which takes the default ID schema-registry)
control-center-multi-sr.properties	bootstrap.servers=localhost:9092 zookeeper.connect=localhost:2181 confluent.controlcenter.kafka.AK1.bootstrap.servers=localhost:9093 confluent.controlcenter.kafka.AK1.zookeeper.connect=localhost:2182 confluent.controlcenter.streams.cprest.url=http://0.0.0.0:8090 confluent.controlcenter.schema.registry.url=http://localhost:8081 confluent.controlcenter.kafka.AK1.cprest.url=http://0.0.0.0:8091 confluent.controlcenter.schema.registry.SR-AK1.url=http://localhost:8082 The configurations for cpcrest.url, confluent.controlcenter.kafka.AK1.cprest.url, and confluent.controlcenter.schema.registry.SR-AK1.url are new properties, specific to multi-cluster Schema Registry.

Tip

The two registries are colocated on localhost. The values for kafkastore.topic and schema.registry.group.id must be unique for each Schema Registry properties file. If the Schema Registry clusters were on different hosts, you wouldn’t need to make these changes.

Run the example

KRaft mode

To run the example in KRaft mode:

1. Configure cluster IDs and format log directories for the Kafka clusters.

2. Start the Kafka brokers in dedicated command windows, one per broker.

3. Start the Schema Registry clusters in dedicated command windows, one per Schema Registry cluster.

4. Start Confluent Control Center in its own dedicated command window.

Configure KRaft specific settings for ``server0`` and ``server1``

Run the following configuration commands from $CONFLUENT_HOME, the top-level directory where you installed Confluent Platform. If you configured your example KRaft servers in the same directory as the default server.properties file, this directory is $CONFLUENT_HOME/etc/kafka.

1. In a new command window where you plan to run server0, generate a random-uuid for server0 using the kafka-storage tool.

   KAFKA_CLUSTER_ID="$(bin/kafka-storage random-uuid)"
   

2. Format the log directories for server0:

   ./bin/kafka-storage format -t $KAFKA_CLUSTER_ID -c $CONFLUENT_HOME/etc/kafka/server0.properties --ignore-formatted
   

   Run server0 in this dedicated window.

3. In a new command window where you plan to run server1, generate a random-uuid for server1 using the kafka-storage tool.

   KAFKA_CLUSTER_ID="$(bin/kafka-storage random-uuid)"
   

4. Format the log directories for server1:

   ./bin/kafka-storage format -t $KAFKA_CLUSTER_ID -c $CONFLUENT_HOME/etc/kafka/server1.properties --ignore-formatted
   

   Run server1 in this dedicated window.

Start the Kafka brokers

kafka-server-start etc/kafka/server0.properties

kafka-server-start etc/kafka/server1.properties

Start Schema Registry clusters

schema-registry-start etc/schema-registry/schema-registry0.properties

schema-registry-start etc/schema-registry/schema-registry1.properties

Start Control Center

control-center-start etc/confluent-control-center/control-center-multi-sr.properties

ZooKeeper mode

To run the example in ZooKeeper mode:

1. Start the ZooKeepers in dedicated command windows, one per ZooKeeper.

2. Start the Kafka brokers in dedicated command windows, one per broker.

3. Start the Schema Registry clusters in dedicated command windows, one per Schema Registry cluster.

4. Start Confluent Control Center in its own dedicated command window.

Start ZooKeepers

zookeeper-server-start etc/kafka/zookeeper0.properties

zookeeper-server-start etc/kafka/zookeeper1.properties

Start the Kafka brokers

kafka-server-start etc/kafka/server0.properties

kafka-server-start etc/kafka/server1.properties

Start Schema Registry clusters

schema-registry-start etc/schema-registry/schema-registry0.properties

schema-registry-start etc/schema-registry/schema-registry1.properties

Start Control Center

control-center-start etc/confluent-control-center/control-center-multi-sr.properties

Manage schemas for both clusters on Control Center

1. When the example clusters are running and Control Center finishes initialization, open Control Center in your web browser. By default, Control Center runs at http://localhost:9021/, as described in Install and access.

2. Select a cluster from the navigation bar, click the Topics menu, and explore the schema management options for one or both clusters.

Security

You can apply any other Schema Registry client configuration used with Control Center to another Schema Registry cluster. Append the Schema Registry cluster’s name to the confluent.controlcenter.schema.registry prefix.

For example, for HTTP Basic authentication with multi-cluster Schema Registry, specify the following in the Confluent Control Center configuration file:

• Use confluent.controlcenter.schema.registry.basic.auth.credentials.source and confluent.controlcenter.schema.registry.basic.auth.user.info to define authentication for the confluent.controlcenter.schema.registry.url cluster.

• Use confluent.controlcenter.schema.registry.{name}.basic.auth.credentials.source and confluent.controlcenter.schema.registry.{name}.basic.auth.user.info for additional Schema Registry clusters (associated with the URL fields by {name}).

Some Schema Registry client configurations also include a schema.registry prefix. For TLS/SSL security settings, specify the following in the Confluent Control Center configuration file:

• Use confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.location and confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.password for the confluent.controlcenter.schema.registry.url cluster.

• Use confluent.controlcenter.schema.registry.{name}.schema.registry.ssl.truststore.location and confluent.controlcenter.schema.registry.{name}.schema.registry.ssl.truststore.password for additional Schema Registry clusters (associated with the URL fields by {name}).

To learn more, see Schema Registry authentication properties in the Control Center Configuration Reference under Security for Confluent Platform components settings and the section on Schema Registry in HTTP Basic authentication. For client configuration, see How to configure clients to Schema Registry in the Schema Registry Security Overview.

Errors and troubleshooting

If the brokers for the cluster have matching Schema Registry URLs that aren’t defined in the Control Center properties file, the cluster overview page displays an error message.

Version compatibility

The confluent.controlcenter.schema.registry.url configuration in the Control Center properties file acts as a default if a cluster’s broker configurations don’t contain confluent.schema.registry.url fields. You can specify more than one Schema Registry cluster with confluent.controlcenter.schema.registry.{name}.url fields.

Suggested reading

• See Tutorial: Use Schema Registry on Confluent Platform to define schemas for a client application for hands-on examples of developing schemas, mapping to topics, and sending messages.

• See Schema Registry for an overview of Schema Registry and schema management.

• Control Center Configuration Reference for Confluent Platform

• Control Center Configuration Examples for Confluent Platform

• A schema for message values has not been set for this topic in Troubleshoot Control Center for Confluent Platform

• Validate Broker-side Schema IDs in Confluent Platform

• Change the subject naming strategy for a topic