.. _schemaregistry_rbac: Configure Role-Based Access Control for |sr| in |cp| ==================================================== |sr-long| supports :ref:`rbac-overview` (|rbac|). Users are granted access to manage, read, and write to particular topics and their associated schemas (contained in |sr| :ref:`subjects`) based on |rbac| :ref:`roles`. User access is scoped to specified resources and |sr| supported :ref:`operations`. .. include:: ../../includes/sr-security.rst :start-after: sr-rbac-intro-start :end-before: sr-rbac-intro-stop Overview -------- |rbac| makes it easier and more efficient to set up and manage user access to |sr| subjects and topics. ------------------------------- Schema Management before |rbac| ------------------------------- Without |rbac|, an administrator must specify every subject or use `*` (for all) and specify each operation (SUBJECT_READ, SUBJECT_COMPATIBILITY_READ, and so forth) that a user needs. If you have 100 developers who need to read schemas, you must set up access 100 times. .. figure:: ../../images/sr-pre-rbac.png :align: center |sr| before |rbac| ----------------------------- Schema Management with |rbac| ----------------------------- An |rbac|-enabled environment addresses the following use cases: - Jack can give limited access to Samantha by assigning her a ``DeveloperRead`` role and specify a set of subjects with a prefix. - Samantha can have multiple roles on different subjects. She can be a ``DeveloperRead`` for "transactions-value" and "orders-value", but assume a ``DeveloperWrite`` role for "customers-value". - If Jack needs to serve 100 developers in his organization, he can create a group for developers and grant the group read-only access to schemas in |sr|. .. figure:: ../../images/sr-rbac.png :align: center |sr| with |rbac| ------------- How it Works ------------- When a client communicates to the |sr| HTTPS endpoint, |sr| passes the client credentials to Metadata Service (MDS) for authentication. MDS is a REST layer on the |ak| broker within Confluent Server, and it integrates with LDAP to authenticate end users on behalf of |sr| and other |cp| services such as |kconnect|, |c3|, and ksqlDB. As shown in :ref:`cp-demo`, clients must have predefined LDAP entries. Once a client is authenticated, you must enforce that only authorized entities have access to the permitted resources. You can use you can use :ref:`ACLs `, |rbac|, or both to do so. While ACLs and |rbac| can be used together or independently, |rbac| is the preferred solution as it provides finer-grained authorization and a unified method for managing access across |cp|. The combined authentication and authorization workflow for a |ak| client connecting to |sr| is shown in the diagram below. .. figure:: ../../images/sr-rbac-rest-api-request.png :align: center --------------- Setup Overview --------------- To enable |rbac-long| on schemas, you must configure the ``schema-registry.properties`` file with connection information to a :ref:`metadata service (MDS)` running |rbac|, and use the :confluent-cli:`Confluent CLI|index.html` to grant user and application access to subjects and other resources based on :ref:`roles`. Typically, you can request account access and the MDS details needed for |rbac| from your security administrator. If you are in a security admin role experimenting with a fully local setup, you would first set up |rbac| using :ref:`MDS`, then create a |rbac-sa| for |sr| using the |confluent-cli|. Following that, you can create principal user accounts with various roles such as ``ResourceOwner``, ``DeveloperRead``, ``DeveloperWrite``, and ``DeveloperManage`` bound to subjects (schemas associated with |ak| topics) and other resources. .. _sr-role-mappings: ----------------------------------------------------------- Role Mappings to Operations for Subject Based Authorization ----------------------------------------------------------- After |sr| is configured and running in an |rbac|-enabled environment, users can read and write schemas to subjects, based on their authorization for operation on a resource (:ref:`roles` and ``rolebinding``). |rbac| supports all |sr| operations as listed in :ref:`operations`. For more details on these operations, see the :ref:`Schema Registry API`. =============== ======================= ======================= ======================= ======================= ======================== Role Name Read Write Delete ReadCompatibility WriteCompatibility =============== ======================= ======================= ======================= ======================= ======================== SystemAdmin Yes Yes Yes Yes Yes UserAdmin No No No No No ClusterAdmin Yes Yes Yes Yes Yes Operator No No No No No SecurityAdmin No No No No No ResourceOwner Yes (based on scoping) Yes (based on scoping) Yes (based on scoping) Yes (based on scoping) Yes (based on scoping) DeveloperRead Yes (based on scoping) No No Yes (based on scoping) No DeveloperWrite Yes Yes (based on scoping) No No No DeveloperManage No No No Yes Yes =============== ======================= ======================= ======================= ======================= ======================== .. tip:: - |sr| has its own cluster. - You may have multiple schema registries. - A single |sr| cluster can be connected to multiple |ak| clusters. - There is no notion of "global compatibility" in terms of roles. To grant permission to a user to manage global compatibility, give them ``DeveloperManage`` role on a subject resource named ``__GLOBAL``. - Starting with |cp| 5.4.x, users with ``developerRead`` and ``developerWrite`` roles also need the ``developerManage`` role if they want to :ref:`view and work with schemas ` on |sr| through the :ref:`control_center`. (Previous to 5.4.x, ``developerRead`` and ``developerWrite`` roles were sufficient to interact with |sr| through the |c3-short|.) ----------------------------- Example ClusterAdmin Use Case ----------------------------- Jack as a ``ClusterAdmin`` wants to set up a |sr| cluster for his organization. **Step 1.** Jack as a cluster administrator contacts the |rbac| security administrator with the following information. - The |ak| cluster to use (if there is more than one to choose from) - |sr| cluster ID (e.g. schema-registry-a) - Jack's principal name - Resources required by |sr| (schemas topic, schemas topic group, and so on). **Step 2.** ``UserAdmin`` creates a |rbac-sa| to represent the |sr| cluster and does the following. - Grants that |rbac-sa| permissions to access the internal schemas topic and other required roles necessary for a |sr| cluster to operate - Provides Jack with the credentials for that |rbac-sa| - Grants Jack the role ``ClusterAdmin`` on the |sr| cluster - Provides Jack with a public key that can be used to authenticate requests. **Step 3.** Jack configures the |sr| cluster to use the provided public key, the specified group ID (for example, "schema-registry-a"), and the |rbac-sa| provided by the ``UserAdmin``, then spins up the cluster. ----------------------- Example User Experience ----------------------- Samantha as a developer needs READ access to two subjects, "transactions-value" and "orders-value", to understand schemas her application needs to interact with. **Step 1.** Samantha as a developer contacts the user administrator with the following information: - List of subjects (for example, "transactions-value", "orders-value") - |sr| cluster ID (for example, "schema-registry-a") **Step 2.** ``UserAdmin`` grants Samantha access to the subjects. **Step 3.** When Samantha runs ``GET /subjects``, she will only see "transactions-value" and "orders-value". **Step 4.** Accidental ``POST`` or ``DELETE`` operations on these subjects will be prevented. Quick Start ----------- This Quick Start describes how to configure |sr| for Role-Based Access Control to manage user and application authorization to topics and subjects (schemas), including how to: - Configure |sr| to start and connect to the |rbac|-enabled |ak-tm| cluster (edit ``schema-registry.properties`` :ref:`use the Confluent CLI to create roles`) - Use the |confluent-cli| to grant a SecurityAdmin role to the |sr| |rbac-sa|. - Use the |confluent-cli| to grant a ResourceOwner role to the |sr| |rbac-sa| on the internal topic and group (used to coordinate across the |sr| cluster). - Use the |confluent-cli| to grant users access to topics (and associated subjects in |sr|). The examples assume a local install of |sr| and shared |rbac| and MDS configuration. Your production environment may differ (for example, |ccloud| or remote |sr|). If you were to use a local |ak|, |zk|, and bootstrap server as might be the case for testing, these would also need authorization through |rbac|, requiring additional prerequisite setup and credentials. .. include:: ../../includes/rbac-demo.rst ---------------- Before You Begin ---------------- If you are new to |cp| or to |sr|, consider first reading or working through these tutorials to get a baseline understanding of the platform, |sr|, and Role-Based Access Control across |cp|. - :ref:`schema_registry_onprem_tutorial` - :ref:`rbac-mds-config` ----------------- Steps at a glance ----------------- .. removed image map because calling an image via raw HTML wasn't working in new docs; i.e., 6.0.0-post + The image map is still in "Docs best practices" .. figure:: ../../images/sr-rbac-setup.png :align: center .. centered:: `Schema Registry RBAC quick start at a glance` ------------- Prerequisites ------------- To run a resource like |sr| in an |rbac| environment you need a |sr| |rbac-sa| (user account for the resource), credentials, and location of the |mds-long| running |rbac|. This enables you to configure |sr| properties to talk to the |rbac|-enabled |ak| cluster, and grant various types of access to |sr| using the |confluent-cli|. Specifically, you need the following to get started. - An |rbac|-enabled |cp| environment and |sr|. - Location of the |mds-long| running |rbac|. (This will be a URL for the |mds-long|, or file path if you are testing a local MDS.) - Authorization to create and modify principals for the organization. - A public key file to use to verify requests with token-based authorization. - A service (user) account for |sr|. In most cases, you will get this information from your Security administrator. ------------------------------------ Install |cp| and the |confluent-cli| ------------------------------------ #. If you have not done so already, :ref:`download and install Confluent Platform` locally.) #. Install the :confluent-cli:`Confluent CLI|installing.html`. --------------------------------------------------- Configure |sr| to communicate with |rbac| services --------------------------------------------------- The next set of examples show how to connect a local |sr| to a remote |mds-long| running |rbac|. The ``schema.registry.properties`` file configurations reflect a remote |mds-long| URL, location, and |ak| cluster ID. Also, the examples assume you are using credentials you got from your Security administrator for a pre-configured schema registry principal user ("|rbac-sa|"), as mentioned in the prerequisites. .. tip:: - In the examples, backslashes (``\``) are used before carriage returns to show multi-line property values. These line breaks and backslashes may cause errors when used in the actual properties file. If so, remove the backslashes and join the lines so as not to break up a property value across multiple lines with returns. - If you have multiple servers to reference, use a semicolon separated list. (For example, as values for ``metadataServerUrls`` or ``confluent.metadata.bootstrap.server.urls``.) Define these settings in ``CONFLUENT_HOME/etc/schema-registry/schema-registry.properties``: #. Configure |sr| authorization for communicating with the |rbac| |ak| cluster. The ``username`` and ``password`` are |rbac| credentials for the |sr| |rbac-sa|, and metadataServerUrls is the location of your |rbac| |ak| cluster (for example, a URL to an `ec2` server). .. code:: bash # Authorize Schema Registry to talk to Kafka (security protocol may also be SASL_SSL if using TLS/SSL) kafkastore.security.protocol=SASL_PLAINTEXT kafkastore.sasl.mechanism=OAUTHBEARER kafkastore.sasl.login.callback.handler.class=io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler kafkastore.sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \ username="" \ password="" \ metadataServerUrls="://:"; #. Configure |rbac| authorization, and bearer/basic authentication, for the |sr| resource. These settings can be used as-is, JETTY_AUTH is the recommended authentication mechanism. .. code:: bash # These properties install the Schema Registry security plugin, and configure it to use RBAC for # authorization and OAuth for authentication resource.extension.class=io.confluent.kafka.schemaregistry.security.SchemaRegistrySecurityResourceExtension confluent.schema.registry.authorizer.class=io.confluent.kafka.schemaregistry.security.authorizer.rbac.RbacAuthorizer rest.servlet.initializor.classes=io.confluent.common.security.jetty.initializer.InstallBearerOrBasicSecurityHandler confluent.schema.registry.auth.mechanism=JETTY_AUTH .. tip:: - The above setting for ``resource.extension.class`` :ref:`activates the security plugin `. - The above setting for ``confluent.schema.registry.auth.mechanism`` :ref:`sets the authentication mechanism ` as Jetty, which is recommended for use with |rbac|. #. Tell |sr| how to communicate with the |ak| cluster running the |mds-long| and how to authenticate requests using a public key. - The value for ``confluent.metadata.bootstrap.server.urls`` can be the same as ``metadataServerUrls``, depending on your environment. - In this step, you need a public key file to use to verify requests with token-based authorization, as mentioned in the prerequisites. .. code:: bash # The location of the metadata service confluent.metadata.bootstrap.server.urls=://: # Credentials to use with the MDS, these should usually match those used for talking to Kafka confluent.metadata.basic.auth.user.info=: confluent.metadata.http.auth.credentials.provider=BASIC # The path to public keys that should be used to verify json web tokens during authentication public.key.path= For additional configurations available to any client communicating with MDS, see also :ref:`rest-client-mds-config` in the |cp| Security documentation. #. Specify the ``kafkastore.bootstrap.server`` you want to use. The default is a commented out line for a local server. If you do not change this or uncomment it, the default will be used. .. code:: bash #kafkastore.bootstrap.servers=PLAINTEXT://localhost:9092 Uncomment this line and set it to the address of your bootstrap server. This may be different from the MDS server URL. The standard port for the |ak| bootstrap server is ``9092``. .. code:: bash kafkastore.bootstrap.servers=:9092 #. (Optional, Legacy) Specify the :ref:`kafkastore-connection-url` you want to use to connect the :ref:`confluentsecurityplugins_schema_registry_security_plugin` to |zk|. The default is shown on the commented out line for a local |zk|. If you do not change this or uncomment it, the default will be used. .. code:: bash #kafkastore.connection.url=localhost:2181 Uncomment this line and set it to the address of your |zk| server. The standard port for the |zk| server is ``2181``. .. note:: :ref:`kafkastore-connection-url` is deprecated. It was previously needed in older versions (5.4.x and earlier) when the |sr| Security Plugin was installed and configured to use ACLs. Starting with |cp| 5.5.0, this is no longer the case, given :ref:`confluentsecurityplugins_sracl_authorizer`. If you do not have the ACL Authorizer, upgrade to a |cp| version that has it. To upgrade to |ak| leader election, see :ref:`schemaregistry_zk_migration`. #. (Optional) Specify a custom ``schema.registry.group.id`` (to serve as |sr| cluster ID) which is different from the default, **schema-registry**. In the example, ``schema.registry.group.id`` is set to "schema-registry-cool-cluster". .. code:: bash # Schema Registry group id, which is the cluster id # The default for |sr| cluster ID is **schema-registry** schema.registry.group.id=schema-registry-cool-cluster .. tip:: The |sr| cluster ID is the same as ``schema-registry-group-id``, which defaults to **schema-registry**. This is used to specify the target resource in ``rolebinding`` commands on the |confluent-cli|. You might need to specify a custom cluster ID to differentiate your |sr| from others in the organization so as to avoid overwriting roles and users in multiple registries. #. (Optional) Specify a custom name for the |sr| default topic. (The default is **_schemas**.) In the example, ``schema.registry.group.id`` is set to ``_jax-schemas-topic``. .. code:: bash # The name of the topic to store schemas in # The default schemas topic is **_schemas** kafkastore.topic=_jax-schemas-topic .. tip:: - The |sr| itself uses an internal topic to hold schemas. The default name for this topic is **_schemas**. You might need to specify a custom name for the schemas internal topic to differentiate it from others in the organization and avoid overwriting data. - An underscore is not required in the name; this is a convention used to indicate an internal topic. #. (Optional) Enable anonymous access to requests that occur without authentication. Any requests that occur without authentication are automatically granted the principal ``User:ANONYMOUS`` .. code:: bash # This enables anonymous access with a principal of User:ANONYMOUS confluent.schema.registry.anonymous.principal=true authentication.skip.paths=/* If you get the following error about not having authorization when you run the ``curl`` command to list subjects as described in :ref:`rbac-start-and-test-sr`, you can enable anonymous requests to bypass the authentication temporarily while you troubleshoot credentials. .. code:: bash curl localhost:8081/subjects Error 401 Unauthorized

HTTP ERROR 401

Problem accessing /subjects. Reason:

    Unauthorized


Powered by Jetty:// 9.4.18.v20190429
.. tip:: - For the above curl command to be successful, you can configure rolebindings or ACLs for ``User:ANONYMOUS``. - The command bypasses the requirement to present valid credentials with a REST request, but not the authorization that is then performed on that request to ensure that the user (or, if no credentials are provided, User:ANONYMOUS) has the proper roles or ACLs to perform that action. ---------------------------------------------------------- Get the |ak| cluster ID for the MDS server you plan to use ---------------------------------------------------------- You will need this in order to specify the |ak| cluster to use in ``rolebinding`` commands on the |confluent-cli|. - To get the |ak| cluster ID for a local host: ``bin/zookeeper-shell localhost:2181 get /cluster/id`` - To get the |ak| cluster ID on a remote host: ``zookeeper-shell : get /cluster/id`` For example, the output of this command currently shows the Kafka cluster ID: ``my-kafka-cluster-ID``: .. code:: bash zookeeper-shell :2181 get /cluster/id Your output should resemble: :: Connecting to :2181 WATCHER:: WatchedEvent state:SyncConnected type:None path:null {"version":"1","id":"my-kafka-cluster-ID"} ... .. _sr-grant-roles-rbac: ---------------------------------- Grant roles for the |sr| |rbac-sa| ---------------------------------- In these steps, you use the |confluent-cli| to log on to MDS and create the |sr| |rbac-sa| . After you have these roles set up, you can use the Confluent CLI to manage |sr| users. For this example, assume the commands use the MDS server credentials, URLs, and property values you set up on your local |sr| properties file. (Optionally, you can use a :ref:`registered cluster name ` in your role bindings.) #. Log on to MDS. .. code:: bash confluent login --url ://: #. Grant the user the role ``SecurityAdmin`` on the |sr| cluster. .. code:: bash confluent iam rbac role-binding create \ --role SecurityAdmin \ --principal User: \ --kafka-cluster-id \ --schema-registry-cluster-id #. Use the command ``confluent iam rbac role-binding list `` to view the role you just created. .. code:: bash confluent iam rbac role-binding list \ --principal User: \ --kafka-cluster-id \ --schema-registry-cluster-id For example, here is a listing for a user "jack-sr" granted ``SecurityAdmin`` role on "schema-registry-cool-cluster", connecting to MDS through a |ak| cluster ``my-kafka-cluster-ID``: .. code:: bash confluent iam rbac role-binding list \ --principal User:jack-sr \ --kafka-cluster-id my-kafka-cluster-ID \ --schema-registry-cluster-id schema-registry-cool-cluster Role | ResourceType | Name | PatternType +---------------+--------------+------+-------------+ SecurityAdmin | Cluster | | #. Grant the user the role ``ResourceOwner`` on the group that |sr| nodes use to coordinate across the cluster. .. code:: bash confluent iam rbac role-binding create \ --principal User: \ --role ResourceOwner \ --resource Group: \ --kafka-cluster-id For example: .. code:: bash confluent iam rbac role-binding create \ --principal User:jack-sr \ --role ResourceOwner \ --resource Group:schema-registry-cool-cluster \ --kafka-cluster-id my-kafka-cluster-ID #. Grant the user the role ``ResourceOwner`` |ak| topic that |sr| uses to store its schemas. .. code:: bash confluent iam rbac role-binding create \ --principal User: \ --role ResourceOwner \ --resource Topic: \ --kafka-cluster-id For example: .. code:: bash confluent iam rbac role-binding create \ --principal User:jack-sr \ --role ResourceOwner \ --resource Topic:_jax-schemas-topic \ --kafka-cluster-id my-kafka-cluster-ID #. Use the command ``confluent iam rbac role-binding list `` to view the role you just created. .. code:: bash confluent iam rbac role-binding list \ --principal User:jack-sr \ --role ResourceOwner \ --kafka-cluster-id my-kafka-cluster-ID For example: .. code:: bash confluent iam rbac role-binding list \ --principal User:jack-sr \ --role ResourceOwner \ --kafka-cluster-id my-kafka-cluster-ID Role | ResourceType | Name | PatternType +-------------+--------------+----------------------------------+-------------+ ResourceOwner | Topic | _jax-schemas-topic | LITERAL ResourceOwner | Group | schema-registry-cool-cluster | LITERAL ResourceOwner | Topic | _schemas | LITERAL ResourceOwner | Group | schema-registry | LITERAL .. _sr-client-authn-authz: --------------------------------------- Client authentication and authorization --------------------------------------- .. include:: ../../includes/client-license.rst .. _sr-use-registred-cluster-name: ---------------------------------------- (Optional) Use a registered cluster name ---------------------------------------- Starting in |cp| 6.0, you can register your |sr| |ak| cluster in the :ref:`cluster registry ` and specify a user-friendly cluster name, which makes it easier to create role bindings. In all of the example commands to :ref:`sr-grant-roles-rbac`, you can use the registered cluster name instead of ```` and ````. For example, the role binding command for a non-registered cluster must include both the |sr| group ID and cluster ID: .. sourcecode:: bash :emphasize-lines: 4,5 confluent iam rbac role-binding create \ --principal User: \ --role ResourceOwner \ --resource Group: \ --kafka-cluster-id Assuming your |sr| cluster has been registered in the |cp| |cr|, you can replace the ```` and ```` with the user-friendly name of the registered cluster: .. sourcecode:: bash :emphasize-lines: 4 confluent iam rbac role-binding create \ --principal User: \ --role ResourceOwner \ --cluster-name .. _rbac-start-and-test-sr: ---------------------- Start |sr| and test it ---------------------- Make sure your local |zk| and |ak| servers are shut down just to keep a clean slate. Remember, for this example you are running against a remote cluster, so now you only have to start |sr|. #. Open a command window, change directories into your local install of |cp|, and run the command to start |sr|. .. code:: bash ./bin/schema-registry-start ./etc/schema-registry/schema-registry.properties #. Run the following command to view subjects. .. code:: bash curl localhost:8081/subjects If you get an empty ``[ ]`` as shown above (or a bracket with topics), that's success! (The empty brackets indicate that no topics are created yet.) Successfully starting and running the local |sr| and the `curl` on ``subjects`` indicates that you have access to |sr| through |rbac|. .. tip:: - If the ``curl`` command shown above does not work, try sending |sr| |rbac-sa| credentials along with it: .. code:: bash curl -u : localhost:8081/subjects - You can create and manage topics, subjects, and schemas through an |rbac|-enabled |c3| or with ``curl`` commands. See :ref:`schema_registry_onprem_tutorial` and :ref:`schemaregistry_using`. The available resources and actions will be scoped in |rbac| based on login credentials. At this point you have successfully configured |sr| to run with |rbac|. Now, you can use the |confluent-cli| to grant role-based access to |sr| users scoped to subjects. -------------------------------------------------------- Log on to |confluent-cli| and grant access to |sr| users -------------------------------------------------------- #. Log on to MDS again. .. code:: bash confluent login --url ://: #. To grant a user named "sam" the role ``ResourceOwner`` on a Subject named "transactions" for a |ak| cluster named ``my-kafka-cluster-ID`` and a |sr| cluster named "schema-registry-cool-cluster": .. code:: bash confluent iam rbac role-binding create \ --principal User:sam \ --resource Subject:transactions \ --role ResourceOwner \ --kafka-cluster-id my-kafka-cluster-ID \ --schema-registry-cluster-id schema-registry-cool-cluster Related content --------------- - Blog post: `Ensure Data Quality and Data Evolvability with a Secured Schema Registry `_ - :ref:`cp-schema-linking-rbac` - :ref:`rbac-mds-config` - :ref:`rbac-overview` - :ref:`rbac-mds-config` - :confluent-cli:`Confluent CLI|installing.html` - :ref:`rbac-cli-quickstart` - :ref:`rbac-predefined-roles` - :ref:`confluentsecurityplugins_schema_registry_security_plugin` - :ref:`confluentsecurityplugins_schema_registry_authorization`