Apache Kafka® ships with a pluggable, out-of-box Authorizer implementation that uses
Apache ZooKeeper™ to store all the ACLs. If you are not using role-based access control (RBAC)
on MDS, then refer to Authorization using ACLs for details about
authorization using ACLs (also known as ZooKeeper-based ACLs).
If you are using Confluent Platform RBAC-enabled Kafka clusters, then you can manage ACLs for
multiple Kafka clusters using the shared, centralized MDS. This enables you to
configure and store ACLs similar to the way you manage and store RBAC role
If you are planning to use centralized (MDS-based) ACLs, but will continue using
ZooKeeper-based ACLs until you migrate, you can continue to use the
API to manage ACLs up until you migrate.
Before creating and using ACLs in the centralized MDS, you must satisfy the following requirements:
You must configure RBAC on a new cluster
(using confluent.authorizer.access.rule.providers=CONFLUENT) that does not have
any ZooKeeper-based ACLs. You must configure MDS broker principals as super.users
The broker.users option is a semicolon-separated list of principals of
users who are allowed access to all resources on inter-broker listeners.
Unlike super.users, broker.users only allows requests originating
from inter-broker listeners. The primary purpose of this option is to
bootstrap MDS clusters when configuring authorization using centralized ACLs.
For details about this configuration, see Configuring the Confluent Server Authorizer.
If you have already configured RBAC on an existing cluster
that is using ZooKeeper-based ACLs, then you must migrate the ZooKeeper-based ACLs to
centralized storage in MDS.
You must first enable RBAC on running clusters and configure the MDS broker
principals as super.users or broker.users.
After enabling RBAC, brokers in your Kafka clusters can read ACLs from both ZooKeeper
and MDS. To avoid inconsistent ACLs, do not add MDS-based ACLs until after this
migration is complete.
To migrate the existing ZooKeeper-based ACLs to MDS:
Confirm that confluent.authorizer.access.rule.providers=CONFLUENT, ZK_ACL
is configured for all brokers in the server.properties file (as mentioned
Specify confluent.authorizer.migrate.acls.from.zk=true on a single broker
of choice. This will be the broker performing the
ZooKeeper ACL migration. Make note of this broker’s broker.id, as it is
required in the next step.
Perform a rolling restart of the broker mentioned in
Step 2. When restarted, this broker copies the ZooKeeper-based ACLs to MDS and also
listens for any ACL updates during migration. After completing this step, you
can add additional Kafka-based ACLs, which are applied to all brokers and
stored in the MDS.
After the migration completes (you can verify completion in the
kafka.authorizer.log), verify that the centralized ACLs are in MDS:
confluent iam acl list --kafka-cluster-id <kafka-cluster-id>
Specify confluent.authorizer.access.rule.providers=CONFLUENT for all
Specify confluent.authorizer.migrate.acls.from.zk=false for the broker from
Perform a rolling restart of all the brokers in the cluster, including
the broker from Step 2, which should be the last broker to be restarted. This
ensures that you do not miss any ACL updates that may have occurred during
From this point forward, you cannot update ZooKeeper using AdminClient, and any
ACLs added directly to ZooKeeper are ignored.
Examples in this section use
confluent iam rolebinding create
to create, delete or list ACLs. For detailed information about the supported
options, run confluent iam acl --help.
confluent iam acl --help
For details about ACL format and customizing user names refer to Authorization using ACLs.
Centralized ACLs supports up to 1000 ACLs per cluster.
In this example you are creating an ACL where Principal User:Bob is allowed
to perform a Read operation from IP 198.51.100.0 on Topic test-topic from a
specified Kafka cluster. You can do this by executing the following:
confluent iam acl create --allow --principal User:Bob --operation READ --host 198.51.100.0 --topic test-topic --kafka-cluster-id <kafka-cluster-id>
Note that confluent iam acl supports IPv6 addresses, but does not support
IP ranges and subnets.
confluent iam acl
By default all principals that don’t have an explicit ACL allowing an operation
to access a resource are denied. In rare cases where an ACL that allows access
to all but some principal is desired, you can use the --deny option.
For example, use the following commands to allow all users to Read from test-topic
in a specified Kafka cluster, but deny only User:BadBob from the specified Kafka cluster:
confluent iam acl create --allow --principal User:'*' --operation READ --topic test-topic --kafka-cluster-id <kafka-cluster-id>
confluent iam acl create --deny --principal User:BadBob --operation READ --topic test-topic --kafka-cluster-id <kafka-cluster-id>
Note that when multiple ACLs apply to a user, as shown above, the --deny option
always overrides the --allow option.
The preceding examples create ACLs for a topic by specifying --topic [topic-name]
as the resource pattern option. Similarly, you can create ACLs for a cluster by
specifying --cluster-scope and to a group by specifying --group:[group-name].
In the event that you want to grant permission to all groups, you can do so by
specifying --group='*' as shown in the following command:
confluent iam acl create --allow --principal User:'*' --operation READ --topic test --group='*' --kafka-cluster-id <kafka-cluster-id>
You can add ACLs on prefixed resource patterns too. For example, if you include
--topic abc- and --prefix in the command, it will impact permissions on
all resources whose name starts with abc-. The next example shows how to add
an ACL for user Jane to access any topic whose name starts with test- in a specified
Kafka cluster. You can do this using the CLI with following options:
confluent iam acl create --allow --principal User:Jane --prefix --topic test- --kafka-cluster-id <kafka-cluster-id>
To delete the ACL added in the first example above, execute the following:
confluent iam acl delete --allow --principal User:Bob --operation READ --host 198.51.100.0 --topic test-topic --kafka-cluster-id <kafka-cluster-id>
If you want to remove the ACL that you created using a prefixed pattern, execute
confluent iam acl create --allow --principal User:Jane --prefix --topic test-topic --kafka-cluster-id <kafka-cluster-id>
You can list the ACLs for a given Kafka cluster using the confluent iam acl list
command and the cluster ID. For example, to list all ACLs for test-topic that use the
prefix test- in a specified Kafka cluster, execute the following:
confluent iam acl list
confluent iam acl list --topic test-topic --prefix --kafka-cluster-id <kafka-cluster-id>
Note that this will only return the ACLs that have been added to this exact prefix