You are viewing documentation for an older version of Confluent Platform. For the latest, click here.
Schema Registry Security Overview¶
Confluent Schema Registry currently supports all Kafka security features, including:
For configuration details, check the configuration options.
For a configuration example that uses Schema Registry configured with security to a secure Kafka cluster, see the Confluent Platform demo.
Schema Registry to Kafka Cluster¶
Kafka is used as Schema Registry storage backend.
The special Kafka topic
_schemas), with a single partition, is used as a highly available write ahead log.
All schemas, subject/version and ID metadata, and compatibility settings are appended as messages to this log.
A Schema Registry instance therefore both produces and consumes messages under the
It produces messages to the log when, for example, new schemas are registered under a subject, or when updates to compatibility settings are registered.
Schema Registry consumes from the
_schemas log in a background thread, and updates its local caches on consumption of each new
_schemas message to reflect the newly added schema or compatibility setting.
Updating local state from the Kafka log in this manner ensures durability, ordering, and easy recoverability.
All Kafka security features are supported by Schema Registry.
Relatively few services need access to Schema Registry, and they are likely internal, so you can restrict access to the Schema Registry itself via firewall rules and/or network segmentation.
Schema Registry supports both unauthenticated and SASL authentication to ZooKeeper.
Setting up ZooKeeper SASL authentication for Schema Registry is similar to Kafka’s setup. Namely, create a keytab for Schema Registry, create a JAAS configuration file, and set the appropriate JAAS Java properties.
In addition to the keytab and JAAS setup, be aware of the zookeeper.set.acl setting. This setting, when set to true, enables ZooKeeper ACLs, which limits access to znodes.
Important: if zookeeper.set.acl is set to true, Schema Registry’s service name must be the same as Kafka’s, which is kafka by default. Otherwise, Schema Registry will fail to create the _schemas topic, which will cause a leader not available error in the DEBUG log. Schema Registry log will show org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata when Kafka does not set ZooKeeper ACLs but Schema Registry does. Schema Registry’s service name can be set either with kafkastore.sasl.kerberos.service.name or in the JAAS file.
If Schema Registry has a different service name than Kafka, at this time zookeeper.set.acl must be set to false in both Schema Registry and Kafka.
Clients to Schema Registry¶
Configuring the REST API for HTTP or HTTPS¶
By default Schema Registry allows clients to make REST API calls over HTTP. You may configure Schema Registry to allow either HTTP or HTTPS or both at the same time.
The following configuration determines the protocol used by Schema Registry:
Comma-separated list of listeners that listen for API requests over HTTP or HTTPS or both. If a listener uses HTTPS, the appropriate SSL configuration parameters need to be set as well.
- Type: list
- Default: “http://0.0.0.0:8081”
- Importance: high
On the clients, configure
schema.registry.url to match the configured Schema Registry listener.
Additional configurations for HTTPS¶
If you configure an HTTPS listener, there are several additional configurations for Schema Registry.
First, configure the appropriate SSL configurations for the keystore and optionally truststore. The truststore is required only when
ssl.client.auth is set to true.
You may specify which protocol to use while making calls between the instances of Schema Registry. The slave to master node calls for writes and deletes will use the specified protocol.
The protocol used while making calls between the instances of Schema Registry. The slave to master node calls for writes and deletes will use the specified protocol. The default value would be http. When https is set, ssl.keystore. and ssl.truststore. configs are used while making the call. The schema.registry.inter.instance.protocol name is deprecated; prefer using inter.instance.protocol instead.
- Type: string
- Default: “http”
- Importance: low
To configure clients to use HTTPS to Schema Registry:
- On the client, configure the
schema.registry.urlto match the configured listener for HTTPS.
- On the client, configure the environment variables to set the SSL keystore and truststore. You will need to set the appropriate env variable depending on the client (one of
KSQL_OPTS). For example:
export KAFKA_OPTS="-Djavax.net.ssl.trustStore=/etc/kafka/secrets/kafka.client.truststore.jks \
Migrating from HTTP to HTTPS¶
To upgrade Schema Registry to allow REST API calls over HTTPS in an existing cluster:
- Add/Modify the
listenersconfig to include HTTPS. For example: http://0.0.0.0:8081,https://0.0.0.0:8082
- Configure Schema Registry with appropriate SSL configurations to setup the keystore and optionally truststore
- Do a rolling bounce of the cluster
This process enables HTTPS, but still defaults to HTTP so Schema Registry instances can still communicate before all nodes have been restarted. They will continue to use HTTP as the default until configured not to. To switch to HTTPS as the default and disable HTTP support, perform the following steps:
- Enable HTTPS as mentioned in first section of upgrade (both HTTP & HTTPS will be enabled)
inter.instance.protocolto https in all the nodes
- Do a rolling bounce of the cluster
- Remove http listener from the
listenersin all the nodes
- Do a rolling bounce of the cluster
Configuring the REST API for Basic HTTP Authentication¶
Schema Registry can be configured to require users to authenticate using a username and password via the Basic HTTP authentication mechanism.
If you’re using Basic authentication, we recommended that you configure Schema Registry to use HTTPS for secure communication, because the Basic protocol passes credentials in plain text.
Use the following settings to configure Schema Registry to require authentication:
authentication.roles config defines a comma-separated list of user roles. To be authorized
to access Schema Registry, an authenticated user must belong to at least one of these roles.
For example, if you define
the following configuration assigns them for authentication:
authentication.realm config must match a section within
defines how the server authenticates users and should be passed as a JVM option during server start:
SchemaRegistry-Props section to the
authentication.realm config setting:
jaas_config.file above uses the Jetty
authenticates users by checking for their credentials in a password file.
You can also use other implementations of the standard Java
LoginModule interface, such as
LdapLoginModule, or the
JDBCLoginModule for reading credentials from a database.
The file parameter is the location of the password file, The format is:
<username>: <password-hash>[,<rolename> ...]
Here’s an example:
Get the password hash for a user by using the
bin/schema-registry-run-class org.eclipse.jetty.util.security.Password fred letmein
Your output should resemble:
Each line of the output is the password encrypted using different mechanisms, starting with plain text.
Once Schema Registry is configured to use Basic authentication, clients must be configured with suitable valid credentials, for example:
To provide data governance with the Confluent Schema Registry :
- disable auto schema registration
- restrict access to the _schemas topic
- restrict access to Schema Registry operations
Disabling Auto Schema Registration¶
By default, client applications automatically register new schemas. If they produce new messages to a new topic, then they will automatically try to register new schemas. This is very convenient in development environments, but in production environments we recommend that client applications do not automatically register new schemas. Register schemas outside of the client application to control when schemas are registered with Confluent Schema Registry and how they evolve.
Within the application, disable automatic schema registration by setting the configuration parameter auto.register.schemas=false, as shown in the example below.
Once a client application disables automatic schema registration, it will no longer be able to dynamically register new schemas from within the application. However, it will still be able to retrieve existing schemas from the Schema Registry, assuming proper authorization.