Use Role-Based Access Control (RBAC) for Authorization in Confluent Platform

Role-based access control (RBAC) is a method for controlling system access based on roles assigned to users within an organization. RBAC is defined around predefined roles and the privileges associated with those roles (also known as role bindings). Roles are a collection of permissions that you can bind to a resource; this binding allows the privileges associated with that role to be performed on that resource. You must grant the role to a principal at the time you bind a resource to the role.

Using RBAC, you can manage who has access to specific Confluent Platform resources, and the actions a user can perform within that resource. RBAC leverages the Confluent Platform Metadata Service to configure and manage your RBAC implementation from a centralized configuration context, thereby simplifying access management across Confluent Platform resources.

Before implementing RBAC you should evaluate the security needs of the users in your organization and, based on the resources they require to perform their duties, group users into roles that satisfy those requirements. See RBAC Role Use Cases for an example of this planning task. It is a best practice to limit users to the minimum required role necessary for them to complete their assigned tasks.

Like ACLs, RBAC uses principals, so you can associate whatever principal a client is using with an RBAC role, which is authorized by the Confluent Server Authorizer to communicate with both RBAC and ACLs. For details about the Confluent Server Authorizer, see Configure Confluent Server Authorizer in Confluent Platform. RBAC roles do not support DENY rules, and there is no difference in the way you create and use legacy ZooKeeper-based ACLs while also using RBAC. However, if you intend to continue using ACLs, we recommend that you migrate to centralized ACLs, which store ACL information in MDS, just like role bindings. Legacy ZooKeeper-based ACLs will continue to work in the meantime.

For details about enabling and configuring RBAC, see Metadata Service Configuration Settings and Configure Metadata Service (MDS) in Confluent Platform.

RBAC benefits

RBAC helps you:

• Manage security access across the Confluent Platform (Kafka, ksqlDB, Connect, Schema Registry, Confluent Control Center) using granular permissions to control user and group access. For example, with RBAC you can specify permissions for each connector in a cluster, making it easier and quicker to get multiple connectors up and running.
• Manage authorization at scale. Administrators can centrally manage the assignment of predefined roles, and also delegate the responsibility of managing access and permissions to the different departments or business units who are the true owners and most familiar with those resources.
• Centrally manage authentication and authorization for multiple clusters, which includes: the MDS, a Kafka cluster, Connect, ksqlDB, Schema Registry clusters, and a single instance of Confluent Control Center.

How RBAC works

Predefined role assignments determine who can access specific Confluent Platform resources, and what actions an individual user can perform within that resource. An administrator assigns predefined roles to users and groups on various resources; each user can be assigned multiple roles on each resource. Certain privileged users (such as the UserAdmin or SystemAdmin) assign roles to users and groups, and then map specific resources to those user roles. For example, a ResourceOwner in the finance department can grant department members access to all topics that use the prefix finance_, which makes it easier for them to manage the resources with which they are most familiar.

User administrators can add LDAP users and groups, making it quicker and easier to centrally configure authentication and authorization for the various Confluent Platform resources used in an organization.

RBAC Overview

With RBAC, the user administrator can map roles to LDAP users and groups that are scoped to specific resources (role binding). After a role binding is set using the Confluent CLI confluent iam rbac role-binding create command, users can’t go to an API or Confluent Control Center to bypass and get access to resources. Binding roles to groups enables client administrators to avoid having to grant explicit access to each user across every component. For details about viewing role bindings, refer to Confluent CLI confluent iam rbac role-binding list command. Note that role binding does not support wildcard matching in principal names. For details, refer to Wildcard principals.

Note

When setting up role bindings (confluent iam rbac role-binding create), if you need to troubleshoot, it may be helpful to view audit logs on the fly to identify authorization events for specific principals, resources, or operations. For details, refer to View audit logs on the fly.

Confluent Platform cluster registry provides a way for cluster administrators to centrally register Kafka clusters in the metadata service (MDS) to enable a more user-friendly RBAC role binding experience. For details, refer to Cluster Registry in Confluent Platform.

Confluent Platform Metadata Service

MDS is the primary mechanism by which RBAC is implemented, and offers a single, centralized configuration context that once set up for a cluster, saves administrators from the complex and time-consuming task of defining and assigning roles for each resource on an individual basis. The Confluent Platform MDS binds and enforces a Kafka cluster configuration across different resources (such as topics, connectors, and schemas). The Metadata Service (MDS) acts as the central authority for all authorization and authentication data. You must configure each Kafka broker in the MDS cluster with MDS.

MDS provides ease of use and convenience in the implementation of role-based access control (RBAC); it can also scale so that you can use this same non-binding permissions model to offer other types of security.

Running on a Confluent Server broker, MDS is also integrated with LDAP to provide authentication and refreshable bearer tokens for impersonation. The MDS is also the master of record for role binding. For details about configuring LDAP integration with RBAC, see Configure LDAP Group-Based Authorization in Confluent Platform and Configure LDAP Authentication. For details about configuring MDS, see Configure Metadata Service (MDS) in Confluent Platform.

RBAC and ACLs

RBAC serves as an additional authorization enforcement layer on top of ACLs, and does not change the way ACLs are created or managed. When considering whether to use RBAC or ACLs for access control, it is suggested you use RBAC as the default because of its ease of use and manageability at scale, but for edge cases where you need to have more granular access control, or wish to explicitly deny access, ACLs may make more sense. For example, you could use RBAC to allow access for a group of users, but an ACL to deny access for a particular member of that group.

RBAC adds an additional authorization mechanism that addresses the following authorization challenges when using ACLs:

• Without RBAC, you cannot use ACLs to grant access to connectors. With RBAC, each connector has its own principal that identifies access to resources. Users have access only to connectors on which they have explicitly been granted permission. If you require connector access control, RBAC is essential.

• RBAC provides the ability to offer Confluent Control Center users granular access to resources; prior to RBAC, any user with access to Confluent Control Center had full or read-only access to topics and resources. If granular access control in Confluent Control Center is a requirement, RBAC is recommended.

• RBAC provides a consistent authentication and authorization mechanism for users access across the entire Confluent Platform, which is not possible if solely using ACLs.

• Prior to RBAC, the creation and management of ACLs could be difficult to manage and maintain, and in organizations with thousands of resources and users, ACL setup could take a long time. With RBAC, the delegation of responsibility to various resources is managed using the ResourceOwner role.

  For example, say you are responsible for managing user access to 1000 topics. Using RBAC, you could grant ResourceOwner to other users to manage the topics owned by specific business units, and in turn, those users could manage access for others within their own teams. Using ACLs in this scenario, you would need to centrally manage access to all the topics, which would be a time and resource-intensive task.

RBAC Authentication Options

The authentication options in use prior to implementing RBAC may require additional configuration, or you may need to use a different authentication method altogether.

Important

Confluent Platform components that have a REST endpoint (such as Schema Registry and Confluent Control Center), don’t support using a principal derived from mTLS authentication when using RBAC. So if you relied on TLS/SSL certificate authentication across Confluent Platform before configuring RBAC, when using RBAC you must also provide HTTP Basic Auth credentials (such as LDAP user) to authenticate against other components or REST API endpoints.

HTTP Basic Auth presents login credentials to other Confluent Platform components and the component uses those credentials to get an OAuth token for the user with MDS (which validates the credentials against LDAP) and then the component uses the OAuth token to make authorization requests to the MDS. You must specify the bearer token for Use HTTP Basic Authentication in Confluent Platform and more specifically, must specify basic.auth.user.info and basic.auth.credentials.source.

When configuring Confluent Platform components (for example, Confluent Control Center, ksqlDB, and REST Proxy) for RBAC, use OAuth for authentication with MDS and Kafka clusters. For authentication with other Confluent Platform components, use Use HTTP Basic Authentication in Confluent Platform.

For Confluent Platform components with REST endpoints (such as Schema Registry and Confluent Control Center), you must use HTTP Basic Authentication to authenticate with MDS. For details, refer to Configure RBAC using the REST API in Confluent Platform. You cannot use principal propagation with Confluent Platform components (for example, REST Proxy) that have a REST endpoint that requires RBAC.

When using RBAC with Schema Registry and Connect you can use any of the authentication methods supported by Confluent Platform to communicate with Kafka clusters and MDS. For authentication with other Confluent Platform components, use Use HTTP Basic Authentication in Confluent Platform.

When using RBAC with Kafka clients, you can use any of the authentication methods supported by Confluent Platform except OAUTHBEARER. For details, refer to Configure Clients for SASL/OAUTHBEARER authentication in Confluent Platform.

RBAC Authentication Options

Terminology

The following terms are used in RBAC:

Access control

Access is the ability of an individual user or application to perform a specific task, such as view, create or modify a resource (e.g. topics). Access control enables secure access to Confluent Platform services and resources.

Principal

The identity of a user or software requesting permission to perform a specific action on a specific resource. Principals can be authenticated or non-authenticated (ANONYMOUS).

User principal

A single identity tied to a specific user or piece of software.

Group principal

A shared identity that groups together a list of user principals and/or other group principals.

Role

A Confluent-defined job function that is assigned a set of permissions required to perform specific actions or operations on Confluent resources. Each role is bound to a principal and Confluent resources.

Resource

A resource can be an Apache Kafka® topic, consumer group, transactionalID, cluster, Schema Registry, ksqlDB, and any other Confluent Platform component.

Role binding

A principal-role-resource combination that allows a principal to perform operations on a resource or set of resources as defined by the role.

Role-based access control (RBAC)

With RBAC, permissions are associated with roles, and users or groups are assigned to appropriate roles. Roles are defined according to job competency, authority, and responsibility within the enterprise. Users and groups are easily reassigned from one role to another. Permissions assigned to roles tend to change relatively slowly compared with changes in user membership of roles.

RBAC limitations

For optimal performance of your RBAC configuration, we recommend that you adhere to these limits.

Resource	Limit
Role bindings	1000
Requests per second (RPS) API calls to add, remove, or look up role bindings	15
Centralized ACLs	Up to 1000 per cluster

Important

The user ID specified in group role bindings is case-specific, and must match the case specified in the AD record. Also note that when logging in as a super user, the login ID is also case-specific and must match the case specified for the user ID in role bindings.

RBAC error codes

The following user access HTTP error codes are in use for RBAC:

Note

In some cases when a user’s credentials are correct, but the user does not have the correct permissions, you would expect a 404 error when querying a nonexistent resource. In such cases, error code 403 is returned to avoid exposing details about specific resources.

401

User login failed due to missing or insufficient credentials (user lacks sufficient permissions).

403

User credentials may be correct, but login failed because user does not have required permissions for a specific resource (such as Schema Registry or Connect).

404

Not Found: user has correct credentials and access to a resource (for example, the user has the ResourceOwner role), but the resource (such as Connect) doesn’t exist.

502

MDS is unreachable. Contact your security administrator.

Schema Registry has many granular error codes that extend beyond the context of RBAC. See Schema Registry API Reference for descriptions of Schema Registry HTTP errors (for example, 40401) not covered here.

RBAC implementation checklist

This section provides a checklist to help you plan your RBAC implementation.

Note

Ansible offers a simpler way to configure and deploy RBAC and MDS. Refer to Ansible RBAC settings for details.

To set up RBAC:

☐ Install Confluent Platform, including the confluent-server commercial component. For more information, see Migrate Confluent Platform to Confluent Server.

☐ Work with your security team to evaluate the needs of the users in your organization and and, based on the resources they require to perform their duties, identify which roles should be assigned to users and groups.

For a description of some typical use cases and required roles for each, refer to RBAC role use cases.

To bootstrap RBAC, you must identify an ACL-level super.user in the Confluent Server broker’s server.properties file on the cluster that hosts MDS. This super.user can then assign the SystemAdmin role to another user who can create the required clusters and scope the required role bindings for users and groups. Be sure to identify which user will serve as a bootstrap super.user. For details, refer to Use Predefined RBAC Roles in Confluent Platform.

☐ Configure the Metadata Service (MDS).

The MDS implements the core RBAC functionality and communicates with LDAP to get user and group information and authenticate users. After configuring MDS, you can proceed with role bindings and configuration of other Confluent Platform components.

Refer to Configure the LDAP identity provider to view an LDAP configuration for MDS.

☐ After you have determined which roles must be assigned to users and groups, create the appropriate role bindings for users to access the resources (for example, Schema Registry, ksqlDB, Connect, and Confluent Control Center) they require to perform their duties.

☐ Confirm the user and group roles you defined using the confluent iam rbac role-binding list command.

☐ Configure Confluent Platform components to communicate with MDS for authentication and authorization. For details, see:

• Configure RBAC for Control Center on Confluent Platform
• Kafka Connect and RBAC
• Deploy Secure ksqlDB with RBAC in Confluent Platform
• Configure Role-Based Access Control for Schema Registry in Confluent Platform
• Configure RBAC for REST Proxy
• Configure RBAC using the REST API in Confluent Platform

Demo

To see a working example of role-based access control (RBAC), check out Confluent Platform demo. This demo and accompanying tutorial show users how to deploy an Apache Kafka® event streaming application. All the components in the demo have security enabled end-to-end, including RBAC.