Schema Validation

Schema Validation gives operators a centralized location to enforce data format correctness at the topic level.

Enabling Schema Validation

To enable Schema Validation on the brokers, specify confluent.schema.registry.url in the Kafka server.properties file (<path-to-confluent>bin/etc/kafka/server.properties).

For example:

confluent.schema.registry.url=http://schema-registry:8081

This configuration accepts a comma-separated list of URLs for Schema Registry instances. This setting is required to make Schema Validation available both from the Confluent CLI and on the Confluent Control Center.

By default, Confluent Server uses the TopicNameStrategy to map topics with schemas in Schema Registry. This can be changed for both the key and value with confluent.key.subject.name.strategy and confluent.value.subject.name.strategy.

Setting Schema Validation on a topic from the Confluent CLI

You can enable Schema Validation on a topic when you create a topic or modify an existing topic.

Creating a Topic with Schema Validation

To set Schema Validation on a topic when you create the topic, set confluent.value.schema.validation=true and confluent.key.schema.validation=true.

For example:

kafka-topics --create --bootstrap-server localhost:9092 --replication-factor 1 \
--partitions 1 --topic test-validation-true \
--config confluent.value.schema.validation=true

If a message is produced and does not have a valid schema for the value of the message, an error is returned to the producer, and the message is discarded.

If a batch of messages is sent and at least one is invalid, then the entire batch is discarded.

Adding Schema Validation to an Existing Topic

To change topic validation configuration on an existing topic (in this case from false to true), specify validation as using the alter and --add-config flags as shown in this example:

kafka-configs --zookeeper localhost:2181 --alter --entity-type topics --entity-name test-validator --add-config confluent.value.schema.validation=true

Enabling Schema Validation on a topic in Confluent Control Center

To enable Schema Validation on a topic from the Control Center:

  1. Click the Configuration tab on an existing topic.

  2. Click Switch to expert mode.

    ../_images/sv-c3-topic-expert-settings.png
  3. In Expert mode, change the settings for confluent.value.schema.validation and confluent.key.schema.validation from false to true.

    ../_images/sv-c3-topic-enable.png
  4. Click Save changes.