OAuth/OIDC Service-to-Service Authentication in Confluent Platform¶
Confluent Platform services communicate with each other in multiple different scenarios. You must configure authentication among these services for secure communication. This page describes how to configure OAuth-based service-to-service authentication on your Confluent Platform clusters. For example, OAuth tokens are used to authenticate Schema Registry to Confluent Server brokers, Connect to Schema Registry, Connect to Confluent Server brokers, and between other Confluent Platform components.
The client credentials grant type is used for authentication among these services. The grant type used for internal service-to-service authentication is the same grant type used for client-to-Kafka authentication, using the SASL/OAuthBearer mechanism, as specified in KIP-768.
OAuthBearer authentication for Confluent Platform services¶
A trust relationship must be established between your Confluent Platform cluster and an OIDC-compliant identity provider (IdP). This means configuring all Confluent Platform services (Confluent Server, Schema Registry, Confluent Control Center, and Connect) to fetch, accept IdP-issued OAuth tokens for authentication. To automate the setup of these trust relationships, you can use Confluent for Kubernetes or Ansible Playbooks for Confluent Platform.
After you configure all the Confluent Platform services to authenticate to each other using OAuth, the following sequence diagram and steps describe a successful authentication flow between two different Confluent Platform services.
For the example above, consider Connect as the Confluent Platform Service 1 and Schema Registry as the Confluent Platform Service 2. And, these services are configured to both fetch and accept OAuth tokens from an OIDC-compliant identity provider already.
Connect wants to register, retrieve schemas to/from Schema Registry. So, Connect here needs to authenticate and get authorized for the schema request to be successful.
Connect fetches an OAuth token to initiate the above. It gives its client credentials with the identity provider and gets authenticated to the identity provider. Post successful validation, Identity provider now provisions an OAuth token embedding the Connect identity into it.
Now, Connect reaches out to Schema Registry and provides its OAuth token. AuthenticationHandler at Schema Registry receives this OAuth token with Connect principal, scope, claims embedded in it.
Schema Registry now validates the Connect OAuth token using the earlier configured JWKS keys. If successful, it then extracts the principal, scope, claims of Connect. Now, Schema Registry looks for access policy rules defined for Connect to access the requested schema and accordingly grants access by establishing a session.
The flow is exactly the same when you configure OAuthBearer authentication between any platform services like Confluent Server to Schema Registry, Schema Registry to Confluent Server, Connect to Confluent Server, Connect to Schema Registry, Confluent Control Center to Confluent Server, Confluent Control Center to Schema Registry, and so on.
It’s called the client credential grant type of OAuth.
AuthenticationHandler¶
All Confluent Platform services use the AuthenticationHandler to accept OAuth tokens. This handler is designed to support multiple authentication requests and facilitate seamless migration from one authentication mechanism to another (for example, from mTLS to OAuth or from LDAP to OAuth) without causing downtime for clients during the transition. For more information, see Use Multi-Protocol Authentication in Confluent Platform and Use the AuthenticationHandler Class for Multi-Protocol Authentication in Confluent Platform.
JWKS keys¶
To validate incoming OAuth token requests, Confluent Platform services must be configured to cache JWKS keys from the corresponding identity provider. These keys are set up during service startup. If the JWKS keys expire due to a rotation policy, the services will immediately fetch updated keys from the identity provider to prevent authentication failures.
OAuth/OIDC platform integration using two identity providers¶
Confluent Platform can be configured to accept OAuth/OIDC tokens simultaneously from two different identity providers. One identity provider can host all user identities for SSO authentication across Confluent Control Center and Confluent CLI. And the other identity provider can host all the service accounts representing the identities of client applications and Confluent Platform services.
Both Confluent for Kubernetes and Ansible Playbooks for Confluent Platform support deployment of OAuth using two different identity providers.