# Secure Schema Registry for Confluent Platform Schema Registry on Confluent Platform implements the same security features as Kafka, including encryption, authentication, and authorization. The following sections cover the considerations, configuration, and management for each. ## Features Confluent Schema Registry currently supports all Kafka security features, including: * Encryption * [TLS/SSL encryption](../../security/protect-data/encrypt-tls.md#encryption-ssl-schema-registry) with a secure Kafka cluster * [End-user REST API calls over HTTPS](#schema-registry-http-https) * Authentication * [Open Authentication (OAuth)](oauth-schema-registry.md#schemaregistry-oauth) for Schema Registry server * [TLS/SSL authentication](../../security/authentication/mutual-tls/overview.md#authentication-ssl-schema-registry) with a secure Kafka Cluster * [SASL authentication](../../security/authentication/overview.md#kafka-sasl-auth) with a secure Kafka Cluster * Jetty authentication as described in [Role-Based Access Control](rbac-schema-registry.md#schemaregistry-rbac) steps * Authorization (provided through the [Schema Registry Security Plugin for Confluent Platform](../../confluent-security-plugins/schema-registry/introduction.md#confluentsecurityplugins-schema-registry-security-plugin)) * [Role-Based Access Control](rbac-schema-registry.md#schemaregistry-rbac) * [Schema Registry ACL Authorizer for Confluent Platform](../../confluent-security-plugins/schema-registry/authorization/sracl_authorizer.md#confluentsecurityplugins-sracl-authorizer) * [Schema Registry Topic ACL Authorizer for Confluent Platform](../../confluent-security-plugins/schema-registry/authorization/topicacl_authorizer.md#confluentsecurityplugins-topicacl-authorizer) * [Schema Registry Authorization (reference of supported operations and resource URIs)](../../confluent-security-plugins/schema-registry/authorization/index.md#confluentsecurityplugins-schema-registry-authorization) For configuration details, check the [configuration options](../installation/config.md#schemaregistry-config). #### SEE ALSO For a configuration example that uses Schema Registry configured with security to a secure Kafka cluster, see the [Confluent Platform demo](../../tutorials/cp-demo/index.md#cp-demo). ## Schema Registry to Kafka Cluster ### Kafka Store Kafka is used as Schema Registry storage backend. The special Kafka topic `` (default `_schemas`), with a single partition, is used as a highly available write ahead log. All schemas, subject/version and ID metadata, and compatibility settings are appended as messages to this log. A Schema Registry instance therefore both produces and consumes messages under the `_schemas` topic. It produces messages to the log when, for example, new schemas are registered under a subject, or when updates to compatibility settings are registered. Schema Registry consumes from the `_schemas` log in a background thread, and updates its local caches on consumption of each new `_schemas` message to reflect the newly added schema or compatibility setting. Updating local state from the Kafka log in this manner ensures durability, ordering, and easy recoverability. The Schema Registry topic is compacted and therefore the latest value of every key is retained forever, regardless of the Kafka retention policy. You can validate this with `kafka-configs`: ```bash kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name _schemas --describe ``` Your output should resemble: ```none Configs for topic '_schemas' are cleanup.policy=compact ``` All Kafka security features are supported by Schema Registry. Relatively few services need access to Schema Registry, and they are likely internal, so you can restrict access to the Schema Registry itself via firewall rules and/or network segmentation. ## Clients to Schema Registry ### Configuring OAuth Starting with Confluent Platform 7.7, the Schema Registry server supports Open Authentication (OAuth). To learn how to configure clients to a Schema Registry with OAuth enabled, see [Configure OAuth on the Schema Registry client](oauth-schema-registry.md#config-oauth-on-sr-client). ### Configuring the REST API for HTTP or HTTPS By default Schema Registry allows clients to make REST API calls over HTTP. You may configure Schema Registry to allow either HTTP or HTTPS or both at the same time. The following configuration determines the protocol used by Schema Registry: `listeners` : Comma-separated list of listeners that listen for API requests over HTTP or HTTPS or both. If a listener uses HTTPS, the appropriate TLS/SSL configuration parameters need to be set as well.
* Type: list * Default: “[http://0.0.0.0:8081](http://0.0.0.0:8081)” * Importance: high On the clients, configure `schema.registry.url` to match the configured Schema Registry listener. ### Additional configurations for HTTPS If you configure an HTTPS listener, there are several additional configurations for Schema Registry. First, configure the appropriate TLS/SSL configurations for the keystore and optionally truststore for the Schema Registry cluster (for example, in `schema-registry.properties`). The truststore is required only when `ssl.client.auth` is set to true. ```bash ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks ssl.truststore.password= ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks ssl.keystore.password= ssl.key.password= ``` You may specify which protocol to use while making calls between the instances of Schema Registry. The secondary to primary node calls for writes and deletes will use the specified protocol. `inter.instance.protocol` : The protocol used while making calls between the instances of Schema Registry. The secondary to primary node calls for writes and deletes will use the specified protocol. The default value is `http`. When `https` is set, `ssl.keystore.` and `ssl.truststore.` configs are used while making the call. The `schema.registry.inter.instance.protocol` name is deprecated; use `inter.instance.protocol` instead.
* Type: string * Default: “http” * Importance: low Starting with 5.4, Confluent Platform provides the Schema Registry dedicated client configuration properties, as shown in the [example](https://github.com/confluentinc/cp-demo/tree/latest/docker-compose.yml). To configure clients to use HTTPS to Schema Registry, set the following properties or environment variables: 1. On the client, configure the `schema.registry.url` to match the configured listener for HTTPS. ```bash .schema.registry.url: ":" ``` 2. On the client, configure the environment variables to set the TLS/SSL keystore and truststore in one of two ways: - (Recommended) Use the Schema Registry dedicated properties to configure the client: ```bash .schema.registry.ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks .schema.registry.ssl.truststore.password= .schema.registry.ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks .schema.registry.ssl.keystore.password= .schema.registry.ssl.key.password= ``` The naming conventions for Confluent Control Center configuration differ slightly from the other clients. To configure Control Center as an HTTPS client to Schema Registry, specify these dedicated properties in the Control Center config file: ```bash confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.location=/etc/kafka/secrets/kafka.client.truststore.jks confluent.controlcenter.schema.registry.schema.registry.ssl.truststore.password= confluent.controlcenter.schema.registry.schema.registry.ssl.keystore.location=/etc/kafka/secrets/kafka.client.keystore.jks confluent.controlcenter.schema.registry.schema.registry.ssl.keystore.password= confluent.controlcenter.schema.registry.schema.registry.ssl.key.password= ``` #### SEE ALSO [Configure TLS proxy server access to Schema Registry](/control-center/current/security/ssl.html#configure-tls-proxy-server-access-to-sr) under [Configuring TLS/SSL for Control Center](/control-center/current/security/ssl.html) provides a detailed explanation of the naming conventions used in this configuration. - (Legacy, on client) Set environment variables depending on the client (one of `KAFKA_OPTS`, `SCHEMA_REGISTRY_OPTS`, `KSQL_OPTS`): ```bash export JAVA_OPTS: "-Djavax.net.ssl.trustStore=/etc/kafka/secrets/kafka.client.truststore.jks \ -Djavax.net.ssl.trustStorePassword= \ -Djavax.net.ssl.keyStore=/etc/kafka/secrets/kafka.client.keystore.jks \ -Djavax.net.ssl.keyStorePassword=" ``` #### IMPORTANT - If you use the legacy method of defining TLS/SSL values in system environment variables, TLS/SSL settings will apply to every Java component running on this JVM. For example on Connect, every [connector](/kafka-connectors/self-managed/overview.html) will use the given truststore. Consider a scenario where you are using an Amazon Web Services (AWS) connector such as S3 or Kinesis, and do not have the AWS certificate chain in the given truststore. The connector will fail with the following error: ```bash com.amazonaws.SdkClientException: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed ``` This does not apply if you use the dedicated Schema Registry client configurations. - For the `kafka-avro-console-producer` and `kafka-avro-console-consumer`, you must pass the Schema Registry properties on the command line. Here is an example for the producer: ```bash ./kafka-avro-console-producer --broker-list localhost:9093 --topic myTopic \ --producer.config ~/ect/kafka/producer.properties --property value.schema=‘{“type”:“record”,“name”:“myrecord”,“fields”:[{“name”:“f1”,“type”:“string”}]}’ \ --property schema.registry.url=https://localhost:8081 --property schema.registry.ssl.truststore.location=/etc/kafka/security/schema.registry.client.truststore.jks --property schema.registry.ssl.truststore.password=myTrustStorePassword ``` For more examples of using the producer and consumer command line utilities, see [Test drive Avro schema](../fundamentals/serdes-develop/serdes-avro.md#sr-test-drive-avro), [Test drive JSON Schema](../fundamentals/serdes-develop/serdes-json.md#sr-test-drive-json-schema), [Test drive Protobuf schema](../fundamentals/serdes-develop/serdes-protobuf.md#sr-test-drive-protobuf), and the demo in [Validate Broker-side Schemas IDs in Confluent Platform](../schema-validation.md#schema-validation). #### SEE ALSO To learn more, see these demos and examples: [Scripted Confluent Platform Demo](../../tutorials/cp-demo/index.md#cp-demo), [Kafka Client Application Examples](https://github.com/confluentinc/examples/tree/latest/clients), and [Using Schema Registry over HTTPS](../develop/using.md#sr-over-https-api-examples) in the API Usage Examples. ### Migrating from HTTP to HTTPS To upgrade Schema Registry to allow REST API calls over HTTPS in an existing cluster: - Add/Modify the `listeners` config to include HTTPS. For example: [http://0.0.0.0:8081,https://0.0.0.0:8082](http://0.0.0.0:8081,https://0.0.0.0:8082) - Configure Schema Registry with appropriate TLS/SSL configurations to setup the keystore and optionally truststore - Do a rolling bounce of the cluster This process enables HTTPS, but still defaults to HTTP so Schema Registry instances can still communicate before all nodes have been restarted. They will continue to use HTTP as the default until configured not to. To switch to HTTPS as the default and disable HTTP support, perform the following steps: - Enable HTTPS as mentioned in first section of upgrade (both HTTP & HTTPS will be enabled) - Configure `inter.instance.protocol` to https in all the nodes - Do a rolling bounce of the cluster - Remove http listener from the `listeners` in all the nodes - Do a rolling bounce of the cluster ### Configuring the REST API for Basic HTTP Authentication Schema Registry can be configured to require users to authenticate using a username and password via the Basic HTTP authentication mechanism. #### NOTE If you’re using Basic authentication, we recommended that you [configure Schema Registry to use HTTPS for secure communication](#schema-registry-http-https), because the Basic protocol passes credentials in plain text. Use the following settings to configure Schema Registry to require authentication: ```none authentication.method=BASIC authentication.roles=,,... authentication.realm= ``` The `authentication.roles` configuration defines a comma-separated list of user roles. To be authorized to access Schema Registry, an authenticated user must belong to at least one of these roles. For example, if you define `admin`, `developer`, `user`, and `sr-user` roles, the following configuration assigns them for authentication: ```none authentication.roles=admin,developer,user,sr-user ``` The `authentication.realm` configuration must match a section within `jaas_config.conf`, which defines how the server authenticates users and should be passed as a JVM option during server start: ```bash export SCHEMA_REGISTRY_OPTS=-Djava.security.auth.login.config=/path/to/the/jaas_config.conf schema-registry-start ${CONFLUENT_HOME}/etc/schema-registry/schema-registry.properties ``` An example `jaas_config.conf` is: ```none SchemaRegistry-Props { org.eclipse.jetty.security.jaas.spi.PropertyFileLoginModule required file="/path/to/password-file" debug="false"; }; ``` Assign the `SchemaRegistry-Props` section to the `authentication.realm` configuration setting: ```none authentication.realm=SchemaRegistry-Props ``` The example `jaas_config.conf` above uses the Jetty `PropertyFileLoginModule`, which authenticates users by checking for their credentials in a password file. You can also use other implementations of the standard Java `LoginModule` interface, such as the `LdapLoginModule`, or the `JDBCLoginModule` for reading credentials from a database. The file parameter is the location of the password file. The format is: ```none : ,[,,...] ``` Here’s an example: ```none fred: OBF:1w8t1tvf1w261w8v1w1c1tvn1w8x,user,admin barney: changeme,user,developer betty: MD5:164c88b302622e17050af52c89945d44,user wilma: CRYPT:adpexzg3FUZAk,admin,sr-user ``` Get the password hash for a user by using the `org.eclipse.jetty.util.security.Password` utility: ```bash schema-registry-run-class org.eclipse.jetty.util.security.Password fred letmein ``` Your output should resemble: ```none letmein OBF:1w8t1tvf1w261w8v1w1c1tvn1w8x MD5:0d107d09f5bbe40cade3de5c71e9e9b7 CRYPT:frd5btY/mvXo6 ``` Each line of the output is the password encrypted using different mechanisms, starting with plain text. Once Schema Registry is configured to use Basic authentication, clients must be configured with suitable valid credentials, for example: ```none basic.auth.credentials.source=USER_INFO basic.auth.user.info=fred:letmein ``` ## Governance To provide data governance with the Confluent Schema Registry : 1. disable auto schema registration 2. restrict access to the \_schemas topic 3. restrict access to Schema Registry operations ### Disabling Auto Schema Registration By default, client applications automatically register new schemas. If they produce new messages to a new topic, then they will automatically try to register new schemas. This is convenient in development environments, but in production environments it’s recommended that client applications do not automatically register new schemas. Best practice is to register schemas outside of the client application to control when schemas are registered with Schema Registry and how they evolve. Within the application, you can disable automatic schema registration by setting the configuration parameter `auto.register.schemas=false`, as shown in the following example. ```java props.put(AbstractKafkaAvroSerDeConfig.AUTO_REGISTER_SCHEMAS, false); ``` Once a client application disables automatic schema registration, it will no longer be able to dynamically register new schemas from within the application. However, it will still be able to retrieve existing schemas from the Schema Registry, assuming proper authorization. ### Authorizing Access to the Schemas Topic If you enable [Kafka authorization](../../security/authorization/acls/overview.md#kafka-authorization), you must grant the Schema Registry service principal the ability to perform the following [operations on the specified resources](../../security/authorization/acls/overview.md#acl-format-operations-resources): - `Read` and `Write` access to the internal **\_schemas** topic. This ensures that only authorized users can make changes to the topic. - `DescribeConfigs` on the schemas topic to verify that the topic exists - `describe topic` on the schemas topic, giving the Schema Registry service principal the ability to list the schemas topic - `DescribeConfigs` on the internal consumer offsets topic - Access to the Schema Registry cluster (`group`) - `Create` permissions on the Kafka cluster ```bash export KAFKA_OPTS="-Djava.security.auth.login.config=" bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --producer --consumer --topic _schemas --group schema-registry bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation DescribeConfigs --topic _schemas bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Describe --topic _schemas bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Read --topic _schemas bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Write --topic _schemas bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Describe --topic __consumer_offsets bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Create --cluster kafka-cluster ``` If you are using the [Schema Registry ACL Authorizer for Confluent Platform](../../confluent-security-plugins/schema-registry/authorization/sracl_authorizer.md#confluentsecurityplugins-sracl-authorizer), you also need permissions to `Read`, `Write`, and `DescribeConfigs` on the internal **\_schemas_acl** topic: ```bash bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --producer --consumer --topic _schemas_acl --group schema-registry bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Read --topic _schemas_acl bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation Write --topic _schemas_acl bin/kafka-acls --bootstrap-server localhost:9092 --command-config adminclient-configs.conf --add \ --allow-principal 'User:' --allow-host '*' \ --operation DescribeConfigs --topic _schemas_acl ``` #### NOTE - **Removing world-level permissions:** In previous versions of Schema Registry, we recommended making the **\_schemas** topic world readable and writable. Now that Schema Registry supports SASL, the world-level permissions can be dropped. ### Authorizing Schema Registry Operations with the Security Plugin The [Schema Registry security plugin](../../confluent-security-plugins/schema-registry/install.md#confluentsecurityplugins-schema-registry-security-quickstart) provides authorization for various Schema Registry operations. It authenticates the incoming requests and authorizes them via the configured authorizer. This allows schema evolution management to be restricted to administrative users, with application users provided with read-only access only. ### License A Confluent Enterprise license is required for the Schema Registry Security Plugins. For more information, see [License](../../installation/license.md#sr-license). ## Related Content - Blog post: [Ensure Data Quality and Data Evolvability with a Secured Schema Registry](https://www.confluent.io/blog/ensure-data-quality-and-evolvability-with-secured-schema-registry/)