# Use Access Control Lists (ACLs) for Authorization in Confluent Platform Access control lists (ACLs) in Confluent Platform control access to resources in Apache Kafka® clusters, such as topics, consumer groups, and clusters. An ACL grants or denies a principal (a user or service) permission to perform a specific operation on a resource. Kafka enforces ACLs through a pluggable Authorizer, configured using the `authorizer.class.name` property in the Confluent Server broker configuration file. The StandardAuthorizer is available for KRaft-based clusters. For KRaft-based clusters, ACLs are stored in the KRaft-based Kafka cluster metadata. Confluent Server brokers use the authorizer to determine whether to authorize an operation based on the principal and the resource being accessed. Setting ACLs is important – if a resource does not have associated ACLs, only super users can access the resource. To learn more about authorization using ACLs, see [Authorization module](https://developer.confluent.io/learn-kafka/security/authorization/) of the free Confluent Developer course, [Apache Kafka Security](https://developer.confluent.io/learn-kafka/security/). ## ACL concepts Access control lists (ACLs) provide important authorization controls for your organization’s Apache Kafka® cluster data. The following sections describe the key concepts for creating and using ACLs to manage access to components and cluster data. ### Authorizer An authorizer is a server plugin used by Apache Kafka® to authorize operations. More specifically, an authorizer controls whether to authorize an operation based on the principal and the resource being accessed. For KRaft-based Kafka clusters, the authorizer is StandardAuthorizer (`org.apache.kafka.metadata.authorizer.StandardAuthorizer`). While this page discusses Apache Kafka® authorizers only, Confluent also provides the [Configure Confluent Server Authorizer in Confluent Platform](../../csa-introduction.md#confluent-server-authorizer) to allow for proprietary LDAP group-based and role-based access control ([RBAC](../rbac/overview.md#rbac-overview)), and [centralized ACLs](../rbac/authorization-acl-with-mds.md#authorization-acl-with-mds). To enable and use the StandardAuthorizer on a KRaft-based Kafka cluster, set the full class name for your configuration on all nodes (brokers, controllers, or combined brokers and controllers) in their configuration file to: ```text authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer ``` Custom authorizers should be tested and validated to ensure compatibility. #### How KRaft forwards principals to the controller In KRaft clusters, administrator requests, such as `CreateTopics` and `DeleteTopics`, are sent to the broker listeners by the client. The broker then forwards the request to the active controller through the first listener configured in `controller.listener.names`. Authorization of these requests is done on the controller node. This is achieved through an `Envelope` request which packages both the underlying request from the client and the client principal. When the controller receives the forwarded Envelope request from the broker, it first authorizes the `Envelope` request using the authenticated broker principal. Then it authorizes the underlying request using the forwarded principal. All of this implies that Kafka must understand how to serialize and deserialize the client principal. The authentication framework allows for customized principals by overriding the `principal.builder.class` configuration. For customized principals to work with KRaft, the configured class must implement `org.apache.kafka.common.security.auth.KafkaPrincipalSerde` so that Kafka knows how to serialize and deserialize the principals. The default implementation `org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder` uses the Kafka RPC format defined in the source code: `clients/src/main/resources/common/message/DefaultPrincipalData.json`. For details about request forwarding in `KRaft`, see [KIP-590](https://cwiki.apache.org/confluence/display/KAFKA/KIP-590%3A+Redirect+Zookeeper+Mutation+Protocols+to+The+Controller). ### Principal A principal is an entity that can be authenticated by the authorizer. Clients of a Confluent Server broker identify themselves as a particular principal using various security protocols. The way a principal is identified depends upon which security protocol it uses to connect to the Confluent Server broker (for example: [mTLS](../../../kafka/configure-mds/mutual-tls-auth-rbac.md#mutual-tls-auth-rbac), [SASL/GSSAPI](../../authentication/sasl/gssapi/overview.md#kafka-sasl-auth-gssapi), or [SASL/PLAIN](../../authentication/sasl/plain/overview.md#kafka-sasl-auth-plain)). Authentication depends on the security protocol in place (such as SASL or TLS) to recognize a principal within a Confluent Server broker. The following examples show the principal name format based on the security protocol being used: - When a client connects to a Confluent Server broker using the TLS security protocol, the principal name is in the form of the TLS certificate subject name: `CN=quickstart.confluent.io,OU=TEST,O=Sales,L=PaloAlto,ST=Ca,C=US`. Note that there are no spaces after the comma between subject parts. - When a client connects to a Confluent Server broker using the SASL security protocol with GSSAPI (Kerberos) mechanism, the principal is in the Kerberos principal format: `kafka-client@hostname.com`. For more detail, refer to [Kerberos Principal Names](https://docs.oracle.com/cd/E19253-01/816-4557/refer-31/index.html). - When a client connects to a Confluent Server broker using the SASL security protocol with a PLAIN or SCRAM mechanism, the principal is a simple text string, such as `alice`, `admin`, or `billing_etl_job_03`. In the following ACL, the plain text principals (`User:alice`, `User:fred`) are identified as Kafka users who are allowed to run specific operations (read and write) from either of the specified hosts (host-1, host-2) on a specific resource (topic): ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:alice \ --allow-principal User:fred \ --allow-host host-1 \ --allow-host host-2 \ --operation read \ --operation write \ --topic finance-topic ``` To follow best practices, create one principal per application and give each principal only the ACLs required and no more. For example, if Alice is writing three programs that access different topics to automate a billing workflow, she could create three principals: `billing_etl_job_01`, `billing_etl_job_02`, and `billing_etl_job_03`. She would then grant each principal permissions on only the required topics and run each program with its specific principal. Alternatively, she could take a middle-ground approach and create a single `billing_etl_jobs` principal with access to all topics that the billing programs require and run all three with that principal. Alice should not run these programs as her own principal because she would presumably have broader permissions than the jobs actually need. Running with one principal per application also helps significantly with debugging and auditing because it’s clearer which application is performing each operation. #### Wildcard principals You can create ACLs for all principals by using a wildcard in the principal `User:*`. ACLs that include a wildcard for the user principal apply to all users. For example, the following command grants everyone access to the topic `testTopic`: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:* \ --operation All \ --topic testTopic ``` If you use an authorizer that supports group principals, such as Confluent Server Authorizer, you can also create ACLs for all group principals using the principal `Group:*`. ACLs that include the wildcard for the principal apply to all users belonging to at least one group. Wildcards are not supported for super users. For example, specifying the wildcard `User:*` in the `super.users` property does not make every user a super user because no wildcard match is performed. ```text super.users=User:* ``` When using a wildcard with the `kafka-acls` command’s `--topic` or `--group` flag, make sure you encapsulate the wildcard with quotes. Quoting the wildcard ensures that Kafka, not the shell, interprets it. If you forget to encapsulate, the command can return unexpected results. The following shows the format you would use to create a wildcard for resources when creating a role binding: ```text confluent iam rbac role-binding create --role ResourceOwner --resource 'Topic:*' --principal User: --kafka-cluster ``` #### NOTE If you use Confluent Server Authorizer, role bindings do not support wildcard matching. Assigning a role to `User:*` does not grant the role to every user. For details about RBAC principals, see [Use Role-Based Access Control (RBAC) for Authorization in Confluent Platform](../rbac/overview.md#rbac-overview). #### SASL/Kerberos principals If you use Kerberos, your Kafka principal is based on your Kerberos principal (for example, `kafka/kafka1.hostname.com@EXAMPLE.COM`). By default, Kafka only uses the primary name of the Kerberos principal, which is the name that appears before the slash (`/`). If the broker Kerberos principal is `kafka/broker1.example.com@EXAMPLE`, then the principal used by the Kafka authorizer is `kafka`. The hostname is different for every broker. This parsing is automatically implemented by the default value of `sasl.kerberos.principal.to.local.rules`. For details about Kerberos principal names and configurations, see [Kerberos Principals](https://docs.oracle.com/cd/E21455_01/common/tutorials/kerberos_principal.html). #### NOTE If your organization uses a Kerberos or Active Directory server, ask your Kerberos administrator for a principal for each Confluent Server broker in your Kafka cluster and for every operating system user that accesses the cluster with Kerberos authentication (using clients and tools). Server principals are of type NT_HOSTBASED_SERVICE. Each Confluent Server broker must be able to communicate with all of the other brokers for replication and when it acts as the controller. You must add the broker principal as a super user, otherwise Kafka does not work. ##### Configuration options for customizing SASL/Kerberos user name By default, the Kafka principal is the primary part of the Kerberos principal. You can change this behavior by specifying a customized rule for `sasl.kerberos.principal.to.local.rules` in `server.properties`. The format of `sasl.kerberos.principal.to.local.rules` takes the form of a list where each rule works in the same way it does in `auth_to_local` in the [Kerberos configuration file (krb5.conf)](https://web.mit.edu/Kerberos/krb5-latest/doc/admin/conf_files/krb5_conf.html). These rules support the use of lowercase or uppercase to force the translated result to be all lowercase (`/L`) or all uppercase (`/U`). This enforcement is achieved by adding a `/L` or `/U` to the end of the rule. Each rule starts with `RULE:` and contains an expression. The following examples show the format and syntax: ```text RULE:[n:string](regexp)s/pattern/replacement/ RULE:[n:string](regexp)s/pattern/replacement/g RULE:[n:string](regexp)s/pattern/replacement//L RULE:[n:string](regexp)s/pattern/replacement/g/L RULE:[n:string](regexp)s/pattern/replacement//U RULE:[n:string](regexp)s/pattern/replacement/g/U ``` This rule translates `user@MYDOMAIN.COM` to `user` while keeping the default rule in place: ```text sasl.kerberos.principal.to.local.rules=RULE:[1:$1@$0](.*@MYDOMAIN.COM)s/@.*//,DEFAULT ``` #### TLS principal user name When TLS client authentication is enabled, the Confluent Server broker identifies each client by its TLS principal, derived from the DN (Distinguished Name) of the client’s certificate. To use TLS principals in ACLs, you must understand how this name is formed. By default, the name of the principal identified by a TLS certificate is the DN (X.500 Distinguished Name) of that certificate (also known as the Subject), which uses the form `CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown`. You can use `ssl.principal.mapping.rules` to translate the DN to a more manageable principal name. If TLS is enabled, but client authentication is not configured, clients connect anonymously using the TLS port and appear to the server with the user name ANONYMOUS. Such a configuration provides encryption and server authentication, but clients connect anonymously. The server also identifies the ANONYMOUS user when the PLAINTEXT security protocol is used. If you grant read and write permission to the ANONYMOUS user, anyone can access the brokers without authentication. ##### Configuration options for customizing TLS user name Use `ssl.principal.mapping.rules` to map the full TLS DN to a shorter, readable principal name for use in ACLs. For details about creating user-friendly principal names for TLS authentication and authorization, see [Use Principal Mapping in Confluent Platform](../../authentication/mutual-tls/tls-principal-mapping.md#tls-principal-mapping). ### ACL operations and allowed APIs An operation is an action performed on a [resource](#acl-resources). In addition to identifying the resources to which users or groups have access, ACLs identify the operations those users or groups are authorized to perform. For each resource, an operation is mapped to one or more Kafka APIs or request types for that resource. For example, a READ operation for the Topic resource is mapped to Fetch, OffsetCommit, and TxnOffsetCommit. Or, a WRITE operation for the Topic resource is mapped to Produce and AddPartitionsToTxn. The following tables list the operations available for each resource type in Confluent Platform and describe the relationship between operations, resources, and APIs. This list is not comprehensive. To learn more about additional cluster resource operations, see [Authorization (ACLs)](../../../multi-dc-deployments/cluster-linking/security.md#cluster-link-acls) in the Cluster Linking documentation. #### Cluster resource operations | Operation | Resource | APIs allowed | |-----------------|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------| | Alter | Cluster | AlterReplicaLogDirs,
CreateAcls,
DeleteAcls | | AlterConfigs | Cluster | AlterConfigs | | ClusterAction | Cluster | Fetch (for replication only),
LeaderAndIsr,
OffsetForLeaderEpoch,
StopReplica,
UpdateMetadata,
ControlledShutdown,
WriteTxnMarkers | | Create | Cluster | CreateTopics, Metadata | | Describe | Cluster | DescribeAcls,
DescribeLogDirs,
ListGroups | | DescribeConfigs | Cluster | DescribeConfigs | #### Topic resource type operations | Operation | Resource | APIs allowed | |-----------------|------------|----------------------------------------------------------------------| | Alter | Topic | CreatePartitions | | AlterConfigs | Topic | AlterConfigs | | Create | Topic | CreateTopics, Metadata | | Delete | Topic | DeleteRecords,
DeleteTopics | | Describe | Topic | ListOffsets,
Metadata,
OffsetFetch,
OffsetForLeaderEpoch | | DescribeConfigs | Topic | DescribeConfigs | | Read | Topic | Fetch,
OffsetCommit,
TxnOffsetCommit | | Write | Topic | Produce,
AddPartitionsToTxn | #### Group resource type operations | Operation | Resource | APIs allowed | |-------------|------------|--------------------------------------------------------------------------------------------------------------------------------------| | Delete | Group | DeleteGroups | | Describe | Group | DescribeGroup,
FindCoordinator,
ListGroups | | Read | Group | AddOffsetsToTxn,
Heartbeat,
JoinGroup,
LeaveGroup,
OffsetCommit,
OffsetFetch,
SyncGroup,
TxnOffsetCommit | #### Token resource type operations | Operation | Resource | API allowed | |-------------|-----------------|----------------| | Describe | DelegationToken | DescribeTokens | #### Transactional ID resource type operations | Operation | Resource | APIs allowed | |-------------|-----------------|-----------------------------------------------------------------------------------------------------------| | Describe | TransactionalId | FindCoordinator | | Write | TransactionalId | Produce,
AddPartitionsToTxn,
AddOffsetsToTxn,
EndTxn,
InitProducerId,
TxnOffsetCommit | The operations in the preceding tables are for clients (producers, consumers, admin) and interbroker operations of a cluster. In a secure cluster, client requests and interbroker operations require authorization. The interbroker operations are split into two classes: cluster and topic. Cluster operations refer to operations necessary for the management of the cluster, like updating broker and partition metadata, changing the leader and the set of in-sync replicas of a partition, and triggering a controlled shutdown. Because of how replication of topic partitions works internally, the broker principal must be a super user so that the broker can replicate topics properly from leader to follower. Producers and consumers must be authorized to perform operations on topics, and they should use different principals from the brokers. The main operations that producers require authorization to run are `WRITE` and `READ`. Admin users can run command line tools and require authorization. Operations that an admin user might need authorization for are `DELETE`, `CREATE`, and `ALTER`. You can use wildcards (`*`) for producers and consumers so that you set the operations only once. #### Implicitly-derived operations In Kafka, certain operations implicitly grant additional permissions: - When granted READ, WRITE, or DELETE, users implicitly derive the DESCRIBE operation. - When granted ALTER_CONFIGS, users implicitly derive the DESCRIBE_CONFIGS operation. #### ACL order of precedence In contexts where you have both allow and deny ACLs, deny ACLs take precedence over allow ACLs. ### ACL resource types ACL resources are the Kafka objects on which access is granted or denied. Resource types include cluster, group, topic, transactional ID, and delegation token. ACLs specify which users can access a specified resource and the operations they can perform on that resource. Within Kafka, resources include: Cluster : The Kafka cluster. To run operations that impact the entire Kafka cluster, such as a controlled shutdown or creating new topics, must be assigned privileges on the cluster resource. Delegation Token : Delegation tokens are shared secrets between Apache Kafka® brokers and clients. Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing SASL/SSL methods. Refer to [Use Delegation Tokens for Authentication in Confluent Platform](../../authentication/delegation-tokens/overview.md#kafka-sasl-delegate-auth) for more details. Group : Groups in the brokers. All protocol calls that work with groups, such as joining a group, must have corresponding privileges with the group in the subject. Group (`group.id`) includes Consumer Group, Stream Group (`application.id`), Connect Worker Group, or any other group that uses the Consumer Group protocol, like Schema Registry cluster.
When using the `kafka-acls` command’s `--group` flag with a wildcard, you must encapsulate the wildcard with quotes. Failure to do this can cause unexpected results. Topic : All Kafka messages are organized into topics (and partitions). To access a topic, you must have a corresponding operation (such as READ or WRITE) defined in an ACL.
When using the `kafka-acls` command’s `--topic` flag with a wildcard, you must encapsulate the wildcard with quotes. Failure to do this can cause unexpected results. Transactional ID : A transactional ID (`transactional.id`) identifies a single producer instance across application restarts and provides a way to ensure a single writer; this is necessary for exactly-once semantics (EOS). Only one producer can be active for each `transactional.id`. When a producer starts, it first checks whether there is a pending transaction by a producer with its own `transactional.id`. If there is, then it waits until the transaction has finished (abort or commit). This guarantees that the producer always starts from a consistent state.
When used, a producer must be able to manipulate transactional IDs and have all the permissions set. For example, the following ACL allows all users in the system access to an EOS producer:
```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --transactional-id * \ --allow-principal User:* \ --operation write ```
In cases where you need to create ACLs for a Kafka cluster to allow Streams exactly once (EOS) processing:
```shell # Allow Streams EOS: kafka-acls ... --add \ --allow-principal User:team1 \ --operation WRITE \ --operation DESCRIBE \ --transactional-id team1-streams-app1 \ --resource-pattern-type prefixed ```
For additional information about the role of transactional IDs, refer to [Transactions in Apache Kafka](https://www.confluent.io/blog/transactions-apache-kafka). The [ACL operations and allowed APIs](#acl-operations) available to a user depend on the resources to which the user has been granted access. All resources have a unique resource identifier. For example, for the topic resource type, the resource identity is the topic name, and for the group resource type, the resource identity is the group name. You can view the ACLs for a specific resource using the `--list` option. For example, to view all ACLs for the topic `test-topic` run the following command: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --list \ --topic test-topic ``` #### Use prefixed ACLs You can specify ACL resources using either a LITERAL value (default), a PREFIXED pattern type, or a wildcard (`*`), which allows both. If you identify the resource as LITERAL, Kafka attempts to match the full resource name (for example, topic or consumer group) with the resource specified in the ACL. In some cases, you might want to use an asterisk (`*`) to specify all resources. If you identify the resource as PREFIXED, Kafka attempts to match the prefix of the resource name with the resource specified in ACL. For example, you can add an ACL for user `User:kafka/kafka1.host-1.com@bigdata.com` to produce to any topic with a name that uses the prefix `Test-`. You can do this by running the following command: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:kafka/kafka1.host-1.com@bigdata.com \ --producer \ --topic Test- \ --resource-pattern-type prefixed ``` In the following example, a program called “BillingPublisher”, which was built using the Kafka Java SDK, requires an ACL that allows it to write only to topics that use the prefix `billing-`: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:BillingPublisher \ --allow-host 198.51.100.0 \ --producer \ --topic billing- \ --resource-pattern-type prefixed ``` Be aware that you cannot use the PREFIXED resource pattern type for a topic while granting access to all groups `*` (wildcard) within a single command. Instead, split permissions across different commands. For example, grant READ and DESCRIBE access to the user for the prefixed topics: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:username \ --operation Read \ --operation Describe \ --topic topicprefix \ --resource-pattern-type prefixed ``` Then grant user READ access to all groups: ```shell kafka-acls --bootstrap-server localhost:9092 \ --command-config adminclient-configs.conf \ --add \ --allow-principal User:username \ --operation Read \ --group '*' ``` When using a wildcard for the `group` (or `topic` flag), make sure you encapsulate the wildcard with quotes as shown in this example. Quoting the wildcard ensures that Kafka, not the shell, interprets it. If you forget to encapsulate, the command can return unexpected results. #### Super users A super user in Kafka can access any resource regardless of its ACLs. By default, if a resource has no associated ACLs, then only super users can access that resource. If you want to change that behavior, you can include the following in `server.properties`: `allow.everyone.if.no.acl.found=true`. #### NOTE Use of the `allow.everyone.if.no.acl.found` configuration option in production environments is strongly discouraged. - If you specify this option based on the assumption that you have ACLs, but then your last ACL is deleted, you essentially open up your Kafka clusters to all users. - If you’re using this option to disable ACLs, exercise caution: if someone adds an ACL, all the users who previously had access will lose that access. You can add super users in `server.properties` (note that the delimiter is a semicolon because TLS/SSL user names might contain a comma) as shown here: ```text super.users=User:Bob;User:Alice ``` ## Related content - [Use Centralized ACLs with MDS for Authorization in Confluent Platform](../rbac/authorization-acl-with-mds.md#authorization-acl-with-mds) - [Schema Registry ACL Authorizer for Confluent Platform](../../../confluent-security-plugins/schema-registry/authorization/sracl_authorizer.md#confluentsecurityplugins-sracl-authorizer) - [Confluent Replicator to Confluent Cloud ACL Configurations](/cloud/current/get-started/examples/ccloud/docs/replicator-to-cloud-configuration-types.html) - [Configure Authorization of ksqlDB with Kafka ACLs](../../../ksqldb/operate-and-deploy/installation/security.md#ksqldb-installation-security-auth-with-acls) - [Required ACL setting for secure Kafka clusters](../../../streams/developer-guide/security.md#streams-developer-guide-security-acls) - [Cluster Linking Authorization (ACLs)](../../../multi-dc-deployments/cluster-linking/security.md#cluster-link-acls) - [Configure Control Center to work with Kafka ACLs on Confluent Platform](/control-center/current/security/config-c3-for-kafka-acls.html) - [Confluent CLI confluent iam acl](https://docs.confluent.io/confluent-cli/current/command-reference/iam/acl/index.html#confluent-iam-acl) - [Access Control Lists (ACLs) for Confluent Cloud](/cloud/current/access-management/acl.html)