Schema Registry Maven Plugin for Confluent Platform

A Maven plugin for Confluent Schema Registry is available to help throughout the development process, with configuration options as listed below.

Tip

Confluent Platform doesn’t provide an official Gradle plugin for Schema Registry. You can use any of these Gradle plugins in your build.gradle file:

Imflog Kafka Schema Registry Gradle Plugin Source: GitHub repository.
com.commercehub.gradle.plugin.avro Source: GitHub repository.
The Maven plugin, used in the pom.xml example file in the Schema Registry tutorial.

configs for all goals

All goals of the Schema Registry Maven plugin accept the configs option. Use configs to add any valid configuration to the CachedSchemaRegistryClient.

configs

Type: Map<String, String>
Required: false

To set up SSL for the Maven plugin, add the following configuration to the plugin. It specifies the keystore and truststore locations and passwords:

<configuration>
   <configs>
     <schema.registry.ssl.keystore.location>path-to-keystore.jks</schema.registry.ssl.keystore.location>
     <schema.registry.ssl.keystore.password>password</schema.registry.ssl.keystore.password>
     <schema.registry.ssl.truststore.location>path-to-truststore.jks</schema.registry.ssl.truststore.location>
     <schema.registry.ssl.truststore.password>password</schema.registry.ssl.truststore.password>
    </configs>
</configuration>

Use the schema.registry. prefix, as with other Schema Registry client configurations for HTTPS.

Authentication

To connect to Confluent Cloud Schema Registry, add credentials under configs with the schema.registry. prefix. The plugin accepts the same Schema Registry client configurations as other clients. It supports API key authentication and OAuth/OpenID Connect (OIDC). For the full list of authentication properties, see Configuration Reference for Schema Registry Clients on Confluent Platform.

API key authentication

Set basic.auth.credentials.source to USER_INFO and provide the Schema Registry API key and secret in basic.auth.user.info as <api-key>:<api-secret>:

<configuration>
   <configs>
     <schema.registry.basic.auth.credentials.source>USER_INFO</schema.registry.basic.auth.credentials.source>
     <schema.registry.basic.auth.user.info>${SR_API_KEY}:${SR_API_SECRET}</schema.registry.basic.auth.user.info>
   </configs>
</configuration>

Goals that connect to Schema Registry also accept these credentials through the dedicated userInfoConfig field.

OAuth/OIDC

Set bearer.auth.credentials.source to OAUTHBEARER and add the identity provider endpoint, client credentials, logical cluster, and identity pool:

<configuration>
   <configs>
     <schema.registry.bearer.auth.credentials.source>OAUTHBEARER</schema.registry.bearer.auth.credentials.source>
     <schema.registry.bearer.auth.issuer.endpoint.url>https://provider.example.com/oauth2/token</schema.registry.bearer.auth.issuer.endpoint.url>
     <schema.registry.bearer.auth.client.id>${OAUTH_CLIENT_ID}</schema.registry.bearer.auth.client.id>
     <schema.registry.bearer.auth.client.secret>${OAUTH_CLIENT_SECRET}</schema.registry.bearer.auth.client.secret>
     <schema.registry.bearer.auth.scope>${OAUTH_SCOPE}</schema.registry.bearer.auth.scope>
     <schema.registry.bearer.auth.logical.cluster>${SR_LOGICAL_CLUSTER}</schema.registry.bearer.auth.logical.cluster>
     <schema.registry.bearer.auth.identity.pool.id>${IDENTITY_POOL_ID}</schema.registry.bearer.auth.identity.pool.id>
   </configs>
</configuration>

schema-registry:download

The download goal downloads schemas for the requested subjects from a Schema Registry server and writes them to a folder on the local file system. If the versions array is empty, the goal downloads the latest schemas for all subjects. If the array has values, its length must match the length of the subjectPatterns array. Use versions to compare a new schema against a specific schema or to get older versions for local compatibility testing.

schemaRegistryUrls

Schema Registry URLs to connect to.

Type: String[]
Required: true

userInfoConfig

User credentials for connecting to Schema Registry, of the form user:password. This is required if connecting to Confluent Cloud Schema Registry.

Type: String[]
Required: false
Default: null

outputDirectory

Output directory to write the schemas to.

Type: File
Required: true

schemaExtension

The file extension to use for the output file name. This must begin with a . character.

Type: File
Required: false
Default: .avsc

subjectPatterns

The subject patterns to download. This is a list of regular expressions. Patterns must match the entire subject name.

Type: String[]
Required: true

Example 1: Download the latest version of a schema

The following example downloads the latest version of schemas with the subject pattern ^TestSubject000-(key|value)$:

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.6.15</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://192.168.99.100:8081</param>
        </schemaRegistryUrls>
        <outputDirectory>src/main/avro</outputDirectory>
        <subjectPatterns>
            <param>^TestSubject000-(key|value)$</param>
        </subjectPatterns>
    </configuration>
</plugin>

Example 2: Specify a schema version to download

The following example downloads version 1 of schemas with the subject pattern ^topic1-(key|value)$, and the latest version of schemas with the subject pattern ^topic2-(key|value)$:

<plugin>
  <groupId>io.confluent</groupId>
  <artifactId>kafka-schema-registry-maven-plugin</artifactId>
  <version>7.6.15</version>
  <configuration>
      <schemaRegistryUrls>
          <param>http://127.0.0.1:8081</param>
      </schemaRegistryUrls>
      <outputDirectory>outputDir/target/</outputDirectory>
      <subjectPatterns>
          <param>^topic1-(key|value)$</param>
          <param>^topic2-(key|value)$</param>
      </subjectPatterns>
      <versions>
          <param>1</param>
          <param>latest</param>
      </versions>
  </configuration>
</plugin>

schema-registry:set-compatibility

The set-compatibility goal updates the configuration of a subject, or the global configuration, directly from the plugin. Use it to change compatibility levels as schemas evolve and to centralize subject and schema management.

schemaRegistryUrls

Schema Registry URLs to connect to.

Type: String[]
Required: true

compatibilityLevels

Map of subjects and the compatibility types to be set in them, respectively.

Type: Map<String,String> (Subject,CompatibilityLevel)
Required: true

If the subject is NULL or __GLOBAL, the goal updates the global configuration. If CompatibilityLevel is NULL, the goal deletes the configuration.

The following example makes these changes:

Sets the compatibility of the subject order to BACKWARD.
Sets the compatibility of the subject product to FORWARD_TRANSITIVE.
Deletes the compatibility of the subject customer.
Changes the global compatibility to BACKWARD_TRANSITIVE.

<plugin>
  <groupId>io.confluent</groupId>
  <artifactId>kafka-schema-registry-maven-plugin</artifactId>
  <version>7.6.15</version>
  <configuration>
        <schemaRegistryUrls>
            <param>http://192.168.99.100:8081</param>
        </schemaRegistryUrls>
        <compatibilityLevels>
            <order>BACKWARD</order>
            <product>FORWARD_TRANSITIVE</product>
            <customer>null</customer>
            <__GLOBAL>BACKWARD_TRANSITIVE</__GLOBAL>
        </compatibilityLevels>
    </configuration>
</plugin>

Example usage

Example usage of schema-registry:set-compatibility:

For a local Schema Registry, see the Schema Registry tutorial.
For Confluent Cloud Schema Registry, see GitHub example.

schema-registry:test-local-compatibility

This goal tests compatibility of a local schema with other existing local schemas during development and testing phases.

Use this goal to check the compatibility of a new schema without connecting to Schema Registry. Testing against Schema Registry requires registering all the schemas you want to check against.

Local testing avoids registration and supports quick, efficient compatibility checks during development.

Tip

For examples of testing schema compatibility using GitHub Actions, see Workflows and examples and the kafka-github-actions demo repo.

schemas

Map of schema names to the locations of the schemas to test for compatibility.

Type: Map<String, File>
Required: true

previousSchemaPaths

Map of schema names to the locations of previous schemas. The goal tests each schema for compatibility against the schemas in previousSchemaPaths. A location can be a directory or a filename. For a directory, the goal adds all files in the directory and ignores subdirectories.

Type: Map<String, File>
Required: true

compatibilityLevels

Map of schema names to the compatibility type to check. CompatibilityLevel is an enum with the values NONE, BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, and FULL_TRANSITIVE. For BACKWARD, FORWARD, or FULL, provide exactly one previousSchema per schema.

Type: Map<String, CompatibilityLevel>
Required: true

schemaTypes

String that specifies the schema type.

Type: Map<String, String> (one of AVRO (default), JSON, PROTOBUF)
Required: false
Default: AVRO

The following example configures three subjects (order, product, and customer) that use the AVRO schema type:

<configuration>
        <schemas>
            <order>src/main/avro/order.avsc</order>
            <product>src/main/avro/product.avsc</product>
            <customer>src/main/avro/customer.avsc</customer>
        </schemas>
        <schemaTypes>
            <order>AVRO</order>
            <product>AVRO</product>
            <customer>AVRO</customer>
        </schemaTypes>
        <compatibilityLevels>
            <order>BACKWARD</order>
            <product>FORWARD</product>
            <customer>NONE</customer>
        </compatibilityLevels>
        <previousSchemaPaths>
            <order>src/main/avro/order.avsc</order>
            <product>src/main/avro/products/</product>
            <customer>src/main/avro/customer.avsc</customer>
        </previousSchemaPaths>
</configuration>

schema-registry:test-compatibility

The test-compatibility goal reads schemas from the local file system and tests them for compatibility against the Schema Registry servers. Use it in a continuous integration pipeline to ensure that schemas in the project are compatible with the schemas in another environment.

Tip

For examples of testing schema compatibility using GitHub Actions, see Workflows and examples and the kafka-github-actions demo repo.

schemaRegistryUrls

Schema Registry URLs to connect to.

Type: String[]
Required: true

userInfoConfig

User credentials for connecting to Schema Registry, of the form user:password. This is required if connecting to Confluent Cloud Schema Registry.

Type: String[]
Required: false
Default: null

subjects

Map containing subject to schema path of the subjects to be registered.

Type: Map<String, File>
Required: true

Tip

To use a slash (/) or other special characters in a subject name, first URL-encode the subject name, and then replace each % with _x. For example, URL-encoding /path/to/my.proto produces %2Fpath%2Fto%2Fmy.proto, and the replacement produces _x2Fpath_x2Fto_x2Fmy.proto. The plugin requires subject names to be XML element names, and % isn’t valid in an XML name. This workaround lets you register a Protobuf schema that another schema references, using a subject such as /path/to/my.proto.

schemaTypes

String that specifies the schema type.

Type: String (one of AVRO (default), JSON, PROTOBUF)
Required: false
Default: AVRO

references

Map containing a reference name and a subject.

Type: Map<String, Reference[]>
Required: false

metadata

Map containing a subject and a Metadata object.

Type: Map<String, Metadata>
Required: false

ruleSet

Map containing a subject and a RuleSet object.

Type: Map<String, Ruleset>
Required: false

verbose

Whether to include in the output the reason a schema fails the compatibility test.

Type: Boolean
Required: false
Default: true

The following example configures three subjects (order, product, and customer) that use the AVRO schema type:

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.6.15</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://192.168.99.100:8081</param>
        </schemaRegistryUrls>
        <subjects>
            <order>src/main/avro/order.avsc</order>
            <product>src/main/avro/product.avsc</product>
            <customer>src/main/avro/customer.avsc</customer>
        </subjects>
        <schemaTypes>
            <order>AVRO</order>
            <product>AVRO</product>
            <customer>AVRO</customer>
        </schemaTypes>
        <references>
            <order>
              <reference>
                  <name>com.acme.Product</name>
                  <subject>product</subject>
              </reference>
              <reference>
                  <name>com.acme.Customer</name>
                  <subject>customer</subject>
              </reference>
            </order>
        </references>
    </configuration>
    <goals>
        <goal>test-compatibility</goal>
    </goals>
</plugin>

Example usage

Example usage of schema-registry:test-compatibility:

For a local Schema Registry, see the Schema Registry tutorial.
For Confluent Cloud Schema Registry, see GitHub example.

schema-registry:derive-schema

The derive-schema goal generates an Avro, JSON Schema, or Protobuf schema from a file that contains messages in JSON format. You can use the generated schema as is, or as a starting point to build on.

The goal takes the following three parameters.

messagePath

Location of the file that contains messages. Each message must be in JSON format and on a new line.

Type: File
Required: true

outputPath

Location of the file to write the result to.

Type: File
Required: true

schemaType

String that specifies the schema type.

Type: String (one of AVRO (default), JSON, PROTOBUF)
Required: false
Default: AVRO

The output of the derive-schema goal is JSON with one or more generated schemas. For each schema, the output includes the matched messages, as line numbers in the input file starting from 0.

{
  "schemas": [
    {
      "schema": "SCHEMA",
      "messagesMatched": [0,3,5]
    },
    {
      "schema": "SCHEMA",
      "messagesMatched": [1,2,4]
    }
  ]
}

The output varies by schema format in the number of schemas returned and in whether the format allows optional fields or arrays with multiple data types. The following sections show the output rules for each format, with example input messages and the resulting schemas.

Avro rules and examples

Avro output rules are as follows.

Multiple schemas can be returned.
Arrays must have a single data type. Any message that doesn’t follow this rule throws an error.
Avro doesn’t support optional fields. The goal can’t combine records.
Unions are supported.
Arrays containing multiple data types using unions are supported.

Example Avro messages:

{"name": "Foo", "Age": {"int": 12}}
{"name": "Bar", "Age": {"string": "12"}}
{"sport": "Football"}

The following output is the schema generated from these messages. Message 0 and Message 1 both have the field Age of type union. The goal merges them into one schema. Message 2 has a different field, sport, which can’t merge with the other messages. It gets its own schema.

{
  "schemas" : [ {
    "schema" : {
      "type" : "record",
      "name" : "Schema",
      "fields" : [ {
        "name" : "Age",
        "type" : [ "int", "string" ]
      }, {
        "name" : "name",
        "type" : "string"
      } ]
    },
    "messagesMatched" : [ 0, 1 ]
  }, {
    "schema" : {
      "type" : "record",
      "name" : "Schema",
      "fields" : [ {
        "name" : "sport",
        "type" : "string"
      } ]
    },
    "messagesMatched" : [ 2 ]
  } ]
}

JSON Schema rules and examples

JSON Schema output rules are as follows.

Exactly one schema is returned for all inputs.
The data type for an array is chosen by combining the data types of all its elements.
If the same name has multiple data types, the goal combines them using oneOf.
For records with the same name, the goal combines all their fields into one record.
For arrays with the same name, the goal combines their data types.

Example JSON Schema messages:

{"name": "Foo", "Age": 12}
{"name": "Bar", "Age": "12"}
{"sport": "Football"}

The following output is the schema generated from these messages. In JSON Schema, all fields are optional. The goal merges the field sport with the other messages. For the field Age, the goal combines the types string and number using oneOf.

{
  "schemas" : [ {
    "schema" : {
      "type" : "object",
      "properties" : {
        "Age" : {
          "oneOf" : [ {
            "type" : "number"
          }, {
            "type" : "string"
          } ]
        },
        "name" : {
          "type" : "string"
        },
        "sport" : {
          "type" : "string"
        }
      }
    },
    "messagesMatched" : [ 0, 1, 2 ]
  } ]
}

Protobuf rules and examples

Protobuf output rules are as follows.

Multiple schemas can be returned.
Arrays must have a single data type. Any message that doesn’t follow this rule throws an error.
For records with the same name, all their fields are combined into one record.

Example Protobuf messages:

{"name": "Foo", "Age": 12}
{"name": "Bar", "Age": "12"}
{"sport": "Football"}

The following output is the schema generated from these messages. In proto3, all fields are optional. The goal can merge Message 0 and Message 2, assuming sport is optional. The same applies to Message 1 and Message 2. Message 0 and Message 1 have conflicting data types for the field Age. The goal generates two schemas.

{
  "schemas" : [ {
    "schema" : "syntax = \"proto3\";\n\nmessage Schema {\n  int32 Age = 1;\n  string name = 2;\n  string sport = 3;\n}\n",
    "messagesMatched" : [ 0, 2 ]
  }, {
    "schema" : "syntax = \"proto3\";\n\nmessage Schema {\n  string Age = 1;\n  string name = 2;\n  string sport = 3;\n}\n",
    "messagesMatched" : [ 1, 2 ]
  } ]
}

Primitive data types mapping

The following table shows how the goal maps data types for each schema type when it creates a schema.

Actual Class	Avro Datatype Chosen	JSON Datatype Chosen	ProtoBuf Datatype Chosen
Long	long	number	int64
Short/Integer	int	number	int32
Null	null	null	google.protobuf.Any
BigInteger/BigDecimal	double	number	double
Float/Double	double	number	double
Boolean	boolean	boolean	boolean
String	string	string	string

Workflow and pom.xml example

Configure pom.xml with the required parameters for a schema. This example specifies Avro:

<plugins>
   <plugin>
       <groupId>io.confluent</groupId>
       <artifactId>kafka-schema-registry-maven-plugin</artifactId>
       <version>7.6.15</version>
       <configuration>
           <messagePath>messages/my-message-01.txt</messagePath>
           <schemaType>avro</schemaType>
           <outputPath>new-schema-01.json</outputPath>
        </configuration>
    </plugin>
</plugins>

Run the derive-schema goal:

mvn io.confluent:kafka-schema-registry-maven-plugin:derive-schema

View the generated schema.

schema-registry:validate

The validate goal reads schemas from the local file system and validates them locally, before registration. Use it to find and correct syntax errors before you submit schemas to Schema Registry with schema-registry:register.

schemaRegistryUrls

Schema Registry URLs to connect to.

Type: String[]
Required: true

userInfoConfig

User credentials for connecting to Schema Registry, of the form user:password. This is required if connecting to Confluent Cloud Schema Registry.

Type: String[]
Required: false
Default: null

subjects

Map containing subject to schema path of the subjects to be registered.

Type: Map<String, File>
Required: true

schemaTypes

String that specifies the schema type.

Type: String (one of AVRO (default), JSON, PROTOBUF)
Required: false
Default: AVRO

references

Map containing a reference name and a subject. (The referenced schema must be registered.)

Type: Map<String, Reference[]>
Required: false

metadata

Map containing a subject and a Metadata object.

Type: Map<String, Metadata>
Required: false

ruleSet

Map containing a subject and a RuleSet object.

Type: Map<String, Ruleset>
Required: false

schema-registry:register

The register goal reads schemas from the local file system and registers them on the target Schema Registry servers. Use it in a continuous deployment pipeline to push schemas to a new environment.

schemaRegistryUrls

Schema Registry URLs to connect to.

Type: String[]
Required: true

userInfoConfig

User credentials for connecting to Schema Registry, of the form user:password. This is required if connecting to Confluent Cloud Schema Registry.

Type: String[]
Required: false
Default: null

subjects

Map containing subject to schema path of the subjects to be registered.

Type: Map<String, File>
Required: true

schemaTypes

String that specifies the schema type.

Type: String (one of AVRO (default), JSON, PROTOBUF)
Required: false
Default: AVRO

normalizeSchemas

Normalizes schemas based on semantic equivalence during registration or lookup. To learn more, see Schema normalization in Formats, Serializers, and Deserializers.

Type: Boolean
Required: false
Default: false

references

Map containing a reference name and a subject.

Type: Map<String, Reference[]>
Required: false

metadata

Map containing a subject and a Metadata object.

Type: Map<String, Metadata>
Required: false

ruleSet

Map containing a subject and a RuleSet object.

Type: Map<String, Ruleset>
Required: false

The following example configures three subjects (order, product, and customer) that use the AVRO schema type:

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.6.15</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://192.168.99.100:8081</param>
        </schemaRegistryUrls>
        <subjects>
            <order>src/main/avro/order.avsc</order>
            <product>src/main/avro/product.avsc</product>
            <customer>src/main/avro/customer.avsc</customer>
        </subjects>
        <schemaTypes>
            <order>AVRO</order>
            <product>AVRO</product>
            <customer>AVRO</customer>
        </schemaTypes>
        <references>
            <order>
              <reference>
                  <name>com.acme.Product</name>
                  <subject>product</subject>
              </reference>
              <reference>
                  <name>com.acme.Customer</name>
                  <subject>customer</subject>
              </reference>
            </order>
        </references>
    </configuration>
    <goals>
        <goal>register</goal>
    </goals>
</plugin>

The following example uses the metadata and ruleSet parameters to add metadata and check the data when registering the schema.

<plugin>
    <groupId>io.confluent</groupId>
    <artifactId>kafka-schema-registry-maven-plugin</artifactId>
    <version>7.4.0</version>
    <configuration>
        <schemaRegistryUrls>
            <param>http://192.168.99.100:8081</param>
        </schemaRegistryUrls>
        <subjects>
            <customer>src/main/avro/customer.avsc</customer>
        </subjects>
        <schemaTypes>
            <customer>AVRO</customer>
        </schemaTypes>
        <metadata>
            <customer>
                <tags>
                    <ssn>PII</ssn>
                    <age>PHI</age>
                </tags>
                <properties>
                    <owner>Bob Jones</owner>
                    <email>bob@acme.com</email>
                </properties>
            </customer>
        </metadata>
        <ruleSet>
            <customer>
                <domainRules>
                    <rule>
                        <name>checkSsnLen</name>
                        <doc>Check the SSN length.</doc>
                        <kind>CONDITION</kind>
                        <mode>WRITE</mode>
                        <type>CEL</type>
                        <tags>
                            <tag>PII</tag>
                            <tag>PHI</tag>
                        </tags>
                        <params>
                            <key1>value1</key1>
                            <key2>value2</key2>
                        </params>
                        <expr>size(message.ssn) == 9</expr>
                        <onSuccess>NONE</onSuccess>
                        <onFailure>DLQ</onFailure>
                        <disabled>false</disabled>
                    </rule>
                </domainRules>
            </customer>
        </ruleSet>
    </configuration>
    <goals>
        <goal>register</goal>
    </goals>
</plugin>

Workflows and examples

You can integrate Maven Plugin goals with GitHub Actions into a continuous integration/continuous deployment (CI/CD) pipeline to manage schemas on Schema Registry. A general example for developing and validating an Apache Kafka® client application with a Python producer and consumer is provided in the kafka-github-actions demo repo.

The following sample pom.xml has project configurations for more detailed validate and register steps.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.confluent</groupId>
    <artifactId>GitHub-Actions-Demo</artifactId>
    <version>1.0</version>

<pluginRepositories>
    <pluginRepository>
        <id>confluent</id>
        <url>https://packages.confluent.io/maven/</url>
    </pluginRepository>
</pluginRepositories>

    <properties>
        <schemaRegistryUrl><$CONFLUENT_SCHEMA_REGISTRY_URL></schemaRegistryUrl>
        <schemaRegistryBasicAuthUserInfo>
            <$CONFLUENT_BASIC_AUTH_USER_INFO>
        </schemaRegistryBasicAuthUserInfo>
        <confluent.version>7.6.15</confluent.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>io.confluent</groupId>
                <artifactId>kafka-schema-registry-maven-plugin</artifactId>
                <version>${confluent.version}</version>
                <configuration>
                    <schemaRegistryUrls>
                        <param>${schemaRegistryUrl}</param>
                    </schemaRegistryUrls>
                    <userInfoConfig>${schemaRegistryBasicAuthUserInfo}</userInfoConfig>
                </configuration>
                <executions>

                    <execution>
                        <id>validate</id>
                        <phase>validate</phase>
                        <goals>
                            <goal>validate</goal>
                        </goals>
                        <configuration>
                            <subjects>
                                <Orders-value>src/main/resources/order.avsc</Orders-value>
                                <Flights-value>src/main/resources/flight.proto</Flights-value>
                            </subjects>
                            <schemaTypes>
                                <Flights-value>PROTOBUF</Flights-value>
                            </schemaTypes>

                        </configuration>
                    </execution>

                    <execution>
                        <id>set-compatibility</id>
                        <phase>validate</phase>
                        <goals>
                            <goal>set-compatibility</goal>
                        </goals>
                        <configuration>
                            <compatibilityLevels>
                                <Orders-value>FORWARD_TRANSITIVE</Orders-value>
                                <Flights-value>FORWARD_TRANSITIVE</Flights-value>
                            </compatibilityLevels>
                        </configuration>
                    </execution>


                    <execution>
                        <id>test-local</id>
                        <phase>validate</phase>
                        <goals>
                            <goal>test-local-compatibility</goal>
                        </goals>
                        <configuration>
                            <schemas>
                                <order>src/main/resources/order.avsc</order>
                                <flight>src/main/resources/flight.proto</flight>
                            </schemas>
                            <schemaTypes>
                                <flight>ProtoBuf</flight>
                            </schemaTypes>
                            <previousSchemaPaths>
                                <flight>src/main/resources/flightSchemas</flight>
                                <order>src/main/resources/orderSchemas</order>
                            </previousSchemaPaths>
                            <compatibilityLevels>
                                <order>FORWARD_TRANSITIVE</order>
                                <flight>FORWARD_TRANSITIVE</flight>
                            </compatibilityLevels>
                        </configuration>
                    </execution>

                    <execution>
                        <id>test-compatibility</id>
                        <phase>validate</phase>
                        <goals>
                            <goal>test-compatibility</goal>
                        </goals>
                        <configuration>
                            <subjects>
                                <Orders-value>src/main/resources/order.avsc</Orders-value>
                                <Flights-value>src/main/resources/flight.proto</Flights-value>
                            </subjects>
                            <schemaTypes>
                                <Flights-value>PROTOBUF</Flights-value>
                            </schemaTypes>
                        </configuration>
                    </execution>

                    <execution>
                        <id>register</id>
                        <goals>
                            <goal>register</goal>
                        </goals>
                        <configuration>
                            <subjects>
                                <Orders-value>src/main/resources/order.avsc</Orders-value>
                                <Flights-value>src/main/resources/flight.proto</Flights-value>
                            </subjects>
                            <schemaTypes>
                                <Flights-value>PROTOBUF</Flights-value>
                            </schemaTypes>
                        </configuration>
                    </execution>

                </executions>
            </plugin>

        </plugins>
    </build>


</project>

You can code the following workflows as GitHub Actions to set up CI/CD for schema management.

Check a new schema when a pull request proposes merging it into master. The checks validate the schema, test local compatibility, set the subject compatibility, and test compatibility with the subject.

run: mvn validate

The validate step runs these goals:

mvn schema-registry:validate@validate
mvn schema-registry:test-local-compatibility@test-local
mvn schema-registry:set-compatibility@set-compatibility
mvn schema-registry:test-compatibility@test-compatibility

The pull-request.yaml for this step might look like this:

name: Testing branch for compatibility before merging
on:
  pull_request:
    branches: [ master ]
    paths: [src/main/resources/*]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: maven
      - name: Validate if schema is valid
        run: mvn schema-registry:validate@validate

  test-local-compatibility:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: maven
      - name: Test schema with locally present schema
        run: mvn schema-registry:test-local-compatibility@test-local

  set-compatibility:
    needs: test-local-compatibility
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: maven
      - name: Set compatibility of subject
        run: mvn schema-registry:set-compatibility@set-compatibility

  test-compatibility:
    needs: set-compatibility
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: maven
      - name: Test schema with subject
        run: mvn schema-registry:test-compatibility@test-compatibility

If the compatibility checks pass, the pull request can go to approval.

Register schema when a pull request is approved and merged to master.

Run this action to register the new schema:

run: mvn schema-registry:register@register

The push.yaml for this step looks like this:

name: Registering Schema on merge of pull request
on:
  push:
    branches: [ master ]
    paths: [src/main/resources/*]
jobs:
  register-schema:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'temurin'
          cache: maven
      - name: Register Schema
        run: mvn io.confluent:kafka-schema-registry-maven-plugin:register@register

Schema Registry Maven Plugin for Confluent Platform

configs for all goals

Authentication

API key authentication

OAuth/OIDC

schema-registry:download

Example 1: Download the latest version of a schema

Example 2: Specify a schema version to download

schema-registry:set-compatibility

Example usage

schema-registry:test-local-compatibility

schema-registry:test-compatibility

Example usage

schema-registry:derive-schema

Avro rules and examples

JSON Schema rules and examples

Protobuf rules and examples

Primitive data types mapping

Workflow and pom.xml example

schema-registry:validate

schema-registry:register

Workflows and examples

Related content