Use OAuth/OIDC to Authenticate to Confluent Cloud

OAuth 2.0 is an authorization framework that authenticates applications to Confluent Cloud using short-lived JSON Web Tokens (JWTs) issued by your identity provider instead of long-lived API keys. Configure your identity provider (for example, Okta, Microsoft Entra ID, Auth0, or Google) to issue tokens, create identity pools in Confluent Cloud to map tokens to permissions, and use SASL/OAUTHBEARER in your Kafka clients.

Confluent supports OAuth 2.0 for the authorization framework and OpenID Connect (OIDC) as the identity layer built on top of OAuth 2.0.

Use OAuth/OIDC for these circumstances:

  • You want to manage application identities through your own identity provider.

  • You need short-lived, secure credentials for application authentication.

  • You want to integrate with existing enterprise identity systems.

  • You need fine-grained access control based on user attributes and groups.

For information about other authentication methods, see authentication overview.

Summary of key features provided by OAuth 2.0 support in Confluent Cloud:

  • Manage application identities and credentials through your own identity provider.

  • Authenticate with Confluent Cloud resources using short-lived credentials (JSON Web Tokens).

  • Confluent Cloud’s OAuth 2.0 service provides OIDC-based tokens for authentication and authorization that are based on the OAuth 2.0 Authorization Framework [RFC 6749] and is compliant with OpenID Connect (OIDC).

  • Use identity pools to map group and other attributes to policies, such as role-based access control (RBAC) or access control lists (ACLs). For details, see Use OAuth Identity Pools with Your OAuth/OIDC Identity Provider on Confluent Cloud.

  • You can configure OAuth using the Confluent Cloud Console, Confluent CLI, and REST API, and automate end-to-end using the OAuth REST API.

  • Support for OAuth auto pool mapping allows automatic mapping of clients to multiple identity pools based on matching filters and removes the need to explicitly specify identity pool IDs in client configurations. For details, see Use auto pool mapping with OAuth identity pools.

Supported identity providers:

  • Microsoft Entra ID (Azure AD), including Azure User-Assigned Managed Identity (UAMI)

  • Okta

  • Auth0

  • Google Identity Platform

  • AWS IAM

  • Other OAuth/OIDC-compliant providers

For step-by-step instructions to add an identity provider, see Add an identity provider using Confluent Cloud Console.

Core OAuth concepts

OAuth in Confluent Cloud uses these core concepts: JWT claims, access token format, identity pools, pool filters, and the security model.

JWT claims

JWT claims are key-value pairs in JWTs that provide identity and authorization information. Common claims include:

  • sub (subject): The unique identifier for the user or application

  • aud (audience): The intended recipient of the token

  • iss (issuer): The identity provider that issued the token

  • scp (scope): The permissions granted to the token

  • groups: User group memberships for authorization

Note

The scp claim is specific to identity providers such as Okta and Microsoft Entra ID. The standard IANA-registered claim for scopes is scope.

Access token format

A JSON Web Token (JWT) is a signed or encrypted string that represents a set of claims as a JSON object in a JSON Web Signature (JWS) or JSON Web Encryption (JWE) structure. Confluent Cloud only accepts JWT access tokens, which follow an open, industry standard for transferring claims securely between two parties.

Each JWT includes a header, body, and signature that are formatted like this:

header.body.signature

For details about JWT credentials, see the following resources:

Identity pools

Identity pools are groups of external identities that map to specific Confluent Cloud permissions. They act as a bridge between your identity provider and Confluent Cloud resources, using claims-based policies to assign access.

For detailed information about creating and managing identity pools, see Use OAuth Identity Pools with Your OAuth/OIDC Identity Provider on Confluent Cloud.

Pool filters

Pool filters are Common Expression Language (CEL) expressions that evaluate JWT claims to determine which identity pool to use. They enable dynamic, claims-based access control policies that automatically map users based on their token claims.

For examples and configuration details, see Set OAuth identity pool filters.

Security model

The OAuth security model in Confluent Cloud validates tokens through this flow:

  1. Your identity provider issues JWTs with claims.

  2. Confluent Cloud validates tokens using trusted JSON Web Key Sets (JWKS).

  3. Pool filters evaluate claims to determine which identity pool to use.

  4. Identity pools provide access based on configured policies (RBAC or ACLs).

For information about managing JWKS URIs, see Manage the JWKS URI on Confluent Cloud.

Tip

If you’re curious to learn about OAuth for Kafka and the centralized identity management system, check out the following podcast:

Rethinking Apache Kafka Security and Account Management.

OAuth 2.0 authentication flow

At a high level, the following diagram shows a sample OAuth flow for an organization.

OAuth and OIDC authentication flow between Confluent Cloud and an identity provider

Here is a summary of the steps in the OAuth 2.0 flow:

  1. Establish trust between Confluent Cloud and your identity provider.

    To establish trust, you need to add the identity provider. This step:

    • Defines the identity provider type.

    • Creates a trust relationship between Confluent Cloud and your identity provider.

    • Adds the claims for authentication and authorization.

  2. Configure your identity pool and access policy.

    An identity pool is a group of external identities that are assigned a certain level of access based on policy.

    For details, see Use OAuth Identity Pools with Your OAuth/OIDC Identity Provider on Confluent Cloud.

  3. Configure clients.

    To configure your clients:

    1. Configure the client ID and client secret in the Kafka client.

      The identity provider generates a client ID and client secret and gives them to the client to use for all future OAuth exchanges.

    2. The client requests a JSON Web Token (JWT) from the identity provider using the client credentials grant.

      The client credentials grant is an OAuth 2.0 flow where the client authenticates directly with the identity provider using its client credentials to obtain an access token.

    3. Use the access token.

      The Kafka client (SASL/OAUTHBEARER) sends the token to Confluent Cloud. If you are using auto pool mapping, the Kafka client automatically matches the token to the appropriate identity pool based on the token claims. For details, see Use auto pool mapping with OAuth identity pools.

      For detailed client configuration instructions, see Configure Kafka Clients for OAuth 2.0 Authentication in Confluent Cloud.

      Producer and consumer configuration with explicit identity pool ID

      Replace the placeholder values with your actual values.

      bootstrap.servers=<bootstrap_url>
security.protocol=SASL_SSL
sasl.oauthbearer.token.endpoint.url=https://myidp.example.com/oauth2/default/v1/token
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler
sasl.mechanism=OAUTHBEARER
sasl.jaas.config= \
  org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
    clientId='<client_id>'
    scope='<requested_scope>'
    clientSecret='<client_secret>'
    extension_logicalCluster='<cluster_id>'
    extension_identityPoolId='<pool_id>';

      Here is an example of the Kafka client configuration:

      bootstrap.servers=pkc-e8mp9.us-east-1.aws.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.oauthbearer.token.endpoint.url=https://auth.example.com/oauth2/v1/token
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler
sasl.mechanism=OAUTHBEARER
sasl.jaas.config= \
  org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
    clientId='kafka-client-123'
    scope='kafka'
    clientSecret='client-secret-abc123'
    extension_logicalCluster='lkc-ab123'
    extension_identityPoolId='pool-1234abc';

      Producer and consumer configuration with auto pool mapping

      Replace the placeholder values with your actual values.

      bootstrap.servers=<bootstrap_url>
security.protocol=SASL_SSL
sasl.oauthbearer.token.endpoint.url=https://myidp.example.com/oauth2/default/v1/token
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler
sasl.mechanism=OAUTHBEARER
sasl.jaas.config= \
  org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
    clientId='<client_id>'
    scope='<requested_scope>'
    clientSecret='<client_secret>'
    extension_logicalCluster='<cluster_id>';

      Note the absence of the extension_identityPoolId parameter in the auto pool mapping configuration. When omitted, the auto pool mapping feature automatically matches the client to the appropriate identity pool based on the token claims. For details, see Use auto pool mapping with OAuth identity pools.

      Here is an example of the Kafka client configuration, with the extension_identityPoolId parameter omitted:

      bootstrap.servers=pkc-e8mp9.us-east-1.aws.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.oauthbearer.token.endpoint.url=https://auth.example.com/oauth2/v1/token
sasl.login.callback.handler.class=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginCallbackHandler
sasl.mechanism=OAUTHBEARER
sasl.jaas.config= \
  org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
    clientId='kafka-client-123'
    scope='kafka'
    clientSecret='client-secret-abc123'
    extension_logicalCluster='lkc-ab123';

    4. Confluent Cloud validates the token received based on the trusted JSON Web Key Set (JWKS), extracts the authenticated ID (sub) or other configured claim, extracts the authorization ID (pool ID), and maps to the authorization policy.

      JSON Web Token (JWT) example:

      {
  "ver": 1,
  "jti": "AT.-u7tKPqYmJm2t2wZgHnzKVOCY6Hy51y2ohXdRX0Z1gQ",
  "iss": "https://mycompany/oauth2/default",
  "aud": "mycompany-okta",
  "iat": 1617050423,
  "exp": 1617054023,
  "sub": "0oa1xn4ddcJb2GyFN4x7",
  "groups": [
    "Marketing",
    "ProjectA"
  ]
}

For information about accessing Kafka REST APIs with OAuth, see Access Kafka REST APIs with an OAuth-OIDC identity provider on Confluent Cloud.

Token exchange flows

Token exchange flows are OAuth 2.0 methods for obtaining access tokens. Confluent Cloud supports multiple token exchange flows for different authentication scenarios. For Kafka clients, the most relevant flows are client_credentials and jwt_bearer, which are machine-to-machine authentication flows that don’t require human interaction.

Client credentials flow

The client_credentials flow is an OAuth 2.0 token exchange method in which the client authenticates with the identity provider using its client ID and secret to get an access token. This is the most common flow for Kafka clients, follows RFC 6749 Section 4.4, and is the currently supported exchange flow for Java and non-Java clients in Confluent Cloud.

Flow overview

The client credentials flow follows these steps:

  1. Client authentication: The client authenticates with the identity provider using its client ID and client secret with HTTP Basic authentication.

  2. Token request: The client sends a POST request to the token endpoint with the grant type set to client_credentials.

  3. Token validation: The identity provider validates the client credentials and issues an access token.

  4. Token response: The identity provider returns the access token to the client.

  5. Resource access: The client uses the access token to access protected resources (Kafka brokers).

Request format

The client sends a request to the identity provider with HTTP Basic authentication:

POST /token HTTP/1.1
Host: server.example.com
Authorization: Basic czZCaGRSa3F0Mzo3RmpmcDBaQnIxS3REUmJuZlZkbUl3
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials

The Authorization header contains the base64-encoded client_id:client_secret pair, and the request body specifies the grant type as client_credentials.

Implementation details

The Kafka Java Client uses the HttpAccessTokenRetriever class to handle client credentials requests. The implementation:

  • Formats the authorization header using the client ID and secret

  • URL-encodes the credentials according to RFC 6749

  • Constructs the request body with grant_type=client_credentials

  • Sends the request to the configured token endpoint

  • Processes the response to extract the access token

JWT bearer flow

The jwt_bearer flow is an OAuth 2.0 token exchange method in which the client uses a signed JWT assertion instead of client credentials to get an access token. Defined in RFC 7523 Section 8.1, this flow is useful for integrations with providers like Google OIDC that don’t support the client_credentials grant.

Flow overview

The JWT bearer flow follows these steps:

  1. JWT creation: The client creates a signed JWT assertion containing claims (issuer, subject, audience, expiration).

  2. Token request: The client sends a POST request to the token endpoint with the grant type set to urn:ietf:params:oauth:grant-type:jwt-bearer and the signed JWT assertion included.

  3. JWT validation: The identity provider validates the JWT signature and claims.

  4. Token response: The identity provider returns an access token to the client.

  5. Resource access: The client uses the access token to access protected resources (Kafka brokers).

Request format

The client sends a request with a signed JWT assertion in the request body:

POST /token.oauth2 HTTP/1.1
Host: authz.example.net
Content-Type: application/x-www-form-urlencoded

grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer
&assertion=eyJhbGciOiJFUzI1NiIsImtpZCI6IjE2In0.
eyJpc3Mi[...omitted for brevity...].
J9l-ZhwP[...omitted for brevity...]

The JWT assertion contains:

  • A kid (key ID) header identifying the private key

  • A signed payload with claims like iss, sub, aud, exp

  • The signature created using the corresponding private key

Implementation details

The JWT bearer flow requires:

  • A private key for signing the assertion

  • Configuration of JWT claims (issuer, subject, audience, expiration)

  • Support for different signing algorithms (RS256, ES256)

  • Proper JWT construction and signing

Note

Client assertion support added in KIP-1258

OAuth 2.0 client assertions are now supported for the client_credentials grant type (per RFC 7523). You no longer need to write custom token providers or custom login callback handlers to pass short-lived JSON Web Tokens (JWTs) to your identity provider (IdP). For the Java Client, the assertion flow is selected automatically when the sasl.oauthbearer.assertion.* properties are set. For librdkafka-based clients (Python Client, Go Client, .NET Client, JavaScript Client), set sasl.oauthbearer.method=oidc and the grant type to urn:ietf:params:oauth:grant-type:jwt-bearer.

Depending on your scenario, you may want to implement a custom provider. For details, see Custom OAuth implementations for the Java client.

Token request implementation

The Kafka client’s token request implementation follows this high-level flow:

  1. Authentication trigger: The Kafka client initiates authentication when connecting to brokers.

  2. Callback handler: The OAuthBearerLoginCallbackHandler processes the authentication request.

  3. Token retrieval: The HttpAccessTokenRetriever sends HTTP requests to the configured token endpoint.

  4. Token validation: The AccessTokenValidator validates the received token.

  5. Retry logic: If the request fails, the retry mechanism implements exponential backoff.

  6. Token caching: Successful tokens are cached and reused for subsequent connections.

Key components

  • OAuthBearerLoginCallbackHandler: Main callback handler that processes authentication requests.

  • HttpAccessTokenRetriever: Handles HTTP requests to the token endpoint.

  • AccessTokenValidator: Validates received tokens.

  • Retry mechanism: Implements exponential backoff for failed requests.

Retry logic

The client implements an exponential backoff retry mechanism:

  1. Immediate attempt to connect to the HTTP endpoint.

  2. If the first attempt fails, a second attempt after sasl.login.retry.backoff.ms.

  3. If the second attempt fails, the duration is doubled before a third attempt.

  4. This pattern repeats until sasl.login.retry.backoff.max.ms is reached.

Token caching

After successful authentication, the returned access token can be reused by other connections from the same client. While additional connections don’t issue new token retrieval HTTP calls, the broker validates the token each time it’s sent by a client connection.

Based on KIP-368, the OAuth token reauthentication logic is automatically inherited by this implementation, so no additional work is needed to support that feature.

Limitations

OAuth 2.0 for Confluent Cloud includes the following limitations:

  • Authentication is supported for Standard, Enterprise, Dedicated, and Freight Kafka clusters only.

  • ACLs for identity pools can be managed only by using Confluent CLI and the REST API.

  • Supported clients include:

    • Apache Kafka client: 3.2.1 or later

    • Confluent Platform: 7.2.1 or later; 7.1.3 or later

    • librdkafka: 1.9.2 or later

For default OAuth service limits, see:

What’s next

Now that you understand the core OAuth concepts and flow, you can:

  1. Review identity pool filters examples to see how claims are evaluated.

  2. Study best practices for secure implementation.

  3. Practice with your identity provider’s test environment.

  4. Implement an OAuth integration following the step-by-step guides.

For additional learning resources, see:

For information about managing OAuth configurations, see Manage OAuth-OIDC identity provider configurations on Confluent Cloud.