.. title:: Authorize using centrailized access control lists (ACLs) .. meta:: :description: Learn how to use centrailized access control lists (ACLs) to control access to Confluent Cloud resources and data. .. _authorization-acl-with-mds: Authorization Using Centralized ACLs ------------------------------------ If you are using |cp| RBAC-enabled |ak| clusters, then you can manage ACLs for multiple |ak| clusters using the shared, centralized MDS. This enables you to configure and store ACLs similar to the way you manage and store RBAC role bindings. If you are planning to use centralized (MDS-based) ACLs, but will continue using |zk|-based ACLs until you migrate, you can continue to use the :platform:`AdminClient|clients/javadocs/javadoc/org/apache/kafka/clients/admin/AdminClient.html` API to manage ACLs up until you migrate. .. note:: |ak-tm| ships with a pluggable, out-of-box Authorizer implementation that uses |zk-full| to store all the ACLs. If you are not using |rbac-long| on MDS, then refer to :ref:`kafka_authorization` for details about authorization using ACLs (also known as |zk|-based ACLs). .. _acl-migration-prereqs: Prerequisites ~~~~~~~~~~~~~ Before creating and using ACLs in the centralized MDS, you must satisfy the following requirements: - You must configure RBAC on a :ref:`new cluster ` (using ``confluent.authorizer.access.rule.providers=CONFLUENT``) that does not have any |zk|-based ACLs. You must configure MDS broker principals as ``super.users`` or ``broker.users``. .. note:: The ``broker.users`` option is a semicolon-separated list of principals of users who are allowed access to all resources on interbroker listeners. Unlike ``super.users``, ``broker.users`` only allows requests originating from interbroker listeners. The primary purpose of this option is to bootstrap MDS clusters when configuring authorization using centralized ACLs. For details about this configuration, see :ref:`confluent_server_authorizer`. - If you have already configured RBAC on an :ref:`existing cluster ` that is using |kraft|-based or |zk|-based ACLs, then you must migrate these ACLs to centralized storage in MDS. .. _acl-migration: Migrating ACLs to MDS ~~~~~~~~~~~~~~~~~~~~~ The |cp| :ref:`Metadata Service (MDS) ` is a centralized service that stores ACLs and RBAC role bindings for multiple |ak| clusters. You can use the |confluent-cli| to manage ACLs and RBAC role bindings for multiple |ak| clusters using the centralized MDS. You can migrate existing |kraft|-based or |zk|-based ACLs to MDS by following the steps in this section. After migration, you can add additional ACLs to MDS, which are then applied to all brokers and stored in MDS. .. tabs:: .. tab:: |kraft| ACLs to MDS To migrate |kraft|-based ACLs to MDS, you must first enable RBAC on running clusters and configure the MDS broker principals as ``super.users`` or ``broker.users``. Note that RBAC in |kraft| mode is not supported for CFK. After enabling RBAC, brokers in your |ak| clusters can read ACLs from both |zk| and MDS. To avoid inconsistent ACLs, do not add MDS-based ACLs until after this migration is complete. To migrate the existing |kraft|-based ACLs to MDS: #. Confirm that ``confluent.authorizer.access.rule.providers=CONFLUENT, KRAFT_ACL`` is configured for all broker nodes and controller nodes in the ``broker.properties`` and ``controller.properties`` files, as mentioned in :ref:`Prerequisites `. #. Specify ``confluent.authorizer.migrate.acls.from.cluster=true`` on a single broker of choice (If the MDS cluster and Kafka cluster for which you are migrating ACLs are same, then select a broker that is not a MDS writer. You can identify the MDS writer broker in the ``writerMemberId``, which is in the latest ``metadata-service.log`` log file). The non-MDS broker is the broker performing the |kraft| ACL migration. Make note of this broker’s ``broker.id``, as it is required in the next step. #. Restart the broker which has set ``confluent.authorizer.migrate.acls.from.cluster=true`` as mentioned in Step 2. When restarted, this broker copies the |kraft|-based ACLs to MDS and also listens for any ACL updates during migration. After completing this step, you can add additional |ak|-based ACLs, which are applied to all broker nodes and stored in the MDS. #. After the migration completes (you can verify completion in the ``kafka.authorizer.log``), verify that the centralized ACLs are in MDS using the following |confluent-cli| command: .. code-block:: shell confluent iam acl list --kafka-cluster-id #. Remove migration related flags: - Specify ``confluent.authorizer.access.rule.providers=CONFLUENT`` for all broker nodes and controller nodes. - Specify ``confluent.authorizer.migrate.acls.from.cluster=false`` for the broker from Step 2 in above section. - Perform a rolling restart of all the broker nodes in the cluster, including the broker from Step 2, which should be the last broker to be restarted. This ensures that you do not miss any ACL updates that may have occurred during migration. - Perform a rolling restart of all the controller nodes in the cluster. After completing these steps, any ACLs added directly to the |ak| cluster are ignored. Note: MDS cluster should only have ``confluent.authorizer.access.rule.providers=CONFLUENT``. If MDS is currenly configured with ``confluent.authorizer.access.rule.providers=CONFLUENT, KRAFT_ACL``, update the MDS configuration and roll the cluster. .. tab:: |zk| ACLs to MDS You must first enable RBAC on running clusters and configure the MDS broker principals as ``super.users`` or ``broker.users``. After enabling RBAC, brokers in your |ak| clusters can read ACLs from both |zk| and MDS. To avoid inconsistent ACLs, do not add MDS-based ACLs until after this migration is complete. To migrate the existing |zk|-based ACLs to MDS: #. Confirm that ``confluent.authorizer.access.rule.providers=CONFLUENT, ZK_ACL`` is configured for all brokers in the ``server.properties`` file (as mentioned in :ref:`acl-migration-prereqs`). #. Specify ``confluent.authorizer.migrate.acls.from.cluster=true`` on a single broker of choice (If the MDS cluster and |ak| cluster for which you are migrating ACLs are same, then select a broker that is not a MDS writer. You can identify the MDS writer broker in the ``writerMemberId``, which is in the latest ``metadata-service.log`` log file). The non-MDS broker is the broker performing the |zk| ACL migration. Make note of this broker's ``broker.id``, as it is required in the next step. #. Perform a :ref:`rolling restart ` of the broker mentioned in Step 2. When restarted, this broker copies the |zk|-based ACLs to MDS and also listens for any ACL updates during migration. After completing this step, you can add additional |ak|-based ACLs, which are applied to all brokers and stored in the MDS. #. After the migration completes (you can verify completion in the ``kafka.authorizer.log``), verify that the centralized ACLs are in MDS by running the following |confluent-cli| command: .. code-block:: text confluent iam acl list --kafka-cluster-id #. Specify ``confluent.authorizer.access.rule.providers=CONFLUENT`` for all brokers. #. Specify ``confluent.authorizer.migrate.acls.from.cluster=false`` for the broker from Step 2. #. Perform a rolling restart of all the brokers in the cluster, including the broker from Step 2, *which should be the last broker to be restarted.* This ensures that you do not miss any ACL updates that may have occurred during migration. From this point forward, you cannot update |zk| using AdminClient, and any ACLs added directly to |zk| are ignored. .. _using-acls-mds: Using centralized ACLs ~~~~~~~~~~~~~~~~~~~~~~ Examples in this section use the CLI :confluent-cli:`confluent iam rbac role-binding create|command-reference/iam/rbac/role-binding/confluent_iam_rbac_role-binding_create.html` command to create, delete or list ACLs. For detailed information about the supported options, run the CLI ``confluent iam acl --help`` command. For details about ACL format and customizing user names, see :ref:`kafka_authorization`. .. _centralized-acl-limitations: Limitations ^^^^^^^^^^^ Centralized ACLs supports up to 1000 ACLs per cluster. .. _creating_acls_mds: Creating centralized ACLs ^^^^^^^^^^^^^^^^^^^^^^^^^ In this example you are creating an ACL where Principal ``User:Bob`` is allowed to perform a Read operation from IP 198.51.100.0 on Topic ``test-topic`` from a specified |ak| cluster. You can do this by running the following |confluent-cli| command: .. codewithvars:: bash confluent iam acl create --allow --principal User:Bob --operation READ --host 198.51.100.0 --topic test-topic --kafka-cluster-id Note that ``confluent iam acl`` supports IPv6 addresses, but does not support IP ranges and subnets. By default all principals that don't have an explicit ACL allowing an operation to access a resource are denied. In rare cases where an ACL that allows access to all but some principal is desired, you can use the ``--deny`` option. For example, use the following commands to allow all users to Read from ``test-topic`` in a specified |ak| cluster, but deny only ``User:BadBob`` from the specified |ak| cluster: .. codewithvars:: bash confluent iam acl create --allow --principal User:'*' --operation READ --topic test-topic --kafka-cluster-id confluent iam acl create --deny --principal User:BadBob --operation READ --topic test-topic --kafka-cluster-id Note that when multiple ACLs apply to a user, as shown above, the ``--deny`` option always overrides the ``--allow`` option. The preceding examples create ACLs for a topic by specifying ``--topic [topic-name]`` as the resource pattern option. Similarly, you can create ACLs for a cluster by specifying ``--cluster-scope`` and to a group by specifying ``--group:[group-name]``. In the event that you want to grant permission to all groups, you can do so by specifying ``--group='*'`` as shown in the following |confluent-cli| command: .. codewithvars:: bash confluent iam acl create --allow --principal User:'*' --operation READ --topic test --group='*' --kafka-cluster-id You can add ACLs on prefixed resource patterns too. For example, if you include ``--topic abc-`` and ``--prefix`` in the command, it will impact permissions on all resources whose name starts with ``abc-``. The next example shows how to add an ACL for user Jane to access any topic whose name starts with ``test-`` in a specified |ak| cluster. You can do this by running the following |confluent-cli| command, with following options: .. codewithvars:: bash confluent iam acl create --allow --principal User:Jane --prefix --topic test- --kafka-cluster-id .. _delete-acls-mds: Deleting centralized ACLs ^^^^^^^^^^^^^^^^^^^^^^^^^ To delete the ACL added in the first example above, run the following |confluent-cli| command: .. codewithvars:: bash confluent iam acl delete --allow --principal User:Bob --operation READ --host 198.51.100.0 --topic test-topic --kafka-cluster-id If you want to remove the ACL that you created using a prefixed pattern, run the following |confluent-cli| command: .. codewithvars:: bash confluent iam acl create --allow --principal User:Jane --prefix --topic test-topic --kafka-cluster-id .. _list-acls-mds: Listing centralized ACLs ^^^^^^^^^^^^^^^^^^^^^^^^ You can list the ACLs for a given |ak| cluster using the |confluent-cli| ``confluent iam acl list`` command and the cluster ID. For example, to list all ACLs for ``test-topic`` that use the prefix ``test-`` in a specified |ak| cluster, execute the following: .. codewithvars:: bash confluent iam acl list --topic test-topic --prefix --kafka-cluster-id Note that this will only return the ACLs that have been added to this exact prefix pattern.