Use SASL/OAUTHBEARER Authentication between Confluent Server Brokers and Kafka Clients in Confluent Platform

Overview

Confluent Platform supports OAuth/OIDC for authentication across all its services and interfaces. For Confluent Server brokers, you can use the SASL/OAUTHBEARER mechanism to authenticate Kafka clients, other Confluent Platform services, like Schema Registry, Connect, or for Cluster Linking with Confluent Cloud clusters.

Confluent Server brokers in Confluent Platform clusters support SASL/OAUTHBEARER authentication, a SASL-based mechanism for authenticating clients using OAuth tokens based on the Java Web Token (JWT) standard, specified in RFC 7519. The SASL/OAuthBearer mechanism provided through Confluent Server brokers is built on KIP-768, and the Kafka client configuration experience is the same as the OAuth-based authentication used in Apache Kafka®, which is also based on KIP-768. Additionally, Confluent Server brokers provide enhanced enterprise capabilities for OAuth/OIDC authorization. For example, group-based authorization can be used when OAuth/OIDC authentication is deployed with RBAC.

SASL/OAUTHBEARER authentication flow

A trust relationship must be established between the Confluent Platform cluster and an OIDC-compliant identity provider (IdP). This includes configuring Confluent Server brokers and all Confluent Platform services to accept IdP-issued OAuth tokens for authentication. Confluent for Kubernetes or Ansible Playbooks for Confluent Platform can be used for automating the setup of trust relationship.

After you configure the Confluent Server brokers and Kafka clients to use SASL/OAUTHBEARER authentication, the following sequence diagram and steps describe a successful authentication flow between a Kafka client and Confluent Server brokers.

SASL/OAUTHBEARER authentication flow between a Kafka client and Confluent Server brokers

The SASL/OAuthBearer mechanism in Confluent Platform extends from the OAuth client credential grant flow. As discussed earlier, this particular flow is the same as OAuth authentication in Apache Kafka® and Confluent Cloud OAuth for its clients.

In the sequence diagram above, there are three main entities:

  • Confluent Server broker, which is integrated with an identity provider for enabling OAuth token-based authentication.
  • Identity provider (OIDC-compliant)
  • Client application (for example, a producer or consumer application)

The following steps describe the flow for a Kafka client:

  1. The Kafka client requests an OAuth token from the identity provider by submitting its client credentials.
  2. The identity provider validates the client credentials and, if successful, returns OAuth access to the client application. In the OAuth token is client identity details, including principal and scope of the client application.
  3. The Kafka client uses the OAuth token to request resource access (like topic A) to the Confluent Server broker over SASL/OAuthBearer.
  4. Confluent Server brokers use the JWKS keys (made available during setup) and validates the incoming OAuth token for client authenticity. If the validation is successful, brokers authenticate the incoming client.
  5. Because the client authentication is successful, the client principal is extracted from the token and used for authorization using RBAC or ACLs specified on the Confluent Platform cluster.
  6. If the client principal extracted from the OAuth token has access policies specified for requested topic A, then the client request for access to topic A is successfully authorized.
  7. After successful authentication and authorization, the Kafka client can proceed with Kafka operations, such as producing or consuming messages.

The same flow as described above (named the client credentials grant flow in the OpenID Connect specification and RFC 6749) is also triggered when OAuth/OIDC is enabled for authentication between clients (producers or consumers) and other Confluent Platform services, such as Schema Registry or REST Proxy. Also, when OAuth/OIDC is used for securing service to service communication between all Confluent Platform services, for example, between Schema Registry and Confluent Server brokers or between Confluent Server brokers to Schema Registry.

The SASL/OAUTHBEARER authentication flow provides the following benefits:

  • The client identities are hosted on an OIDC-compliant identity provider, enabling centralized identity management and streamlined authentication.
  • The use of short-lived tokens enhances security.
  • Fine-grained access control, using the tokens to carry specific permissions.

Related content