Migrate from mTLS to OAuth Authentication in Confluent Platform

This page provides guidance for migrating from mTLS (mutual TLS) to OAuth authentication in your Confluent Platform clusters. Migrating can help improve your clusters’ security and simplify authentication management in the Confluent Platform environment.

During the migration phase your clients have little downtime. In this phase Confluent Platform services can simultaneously accept both mTLS and OAuth tokens for authentication through the AuthenticationHandler. Also, the handler enables backward compatibility for clients that haven’t converted to OAuth yet.

Understand principal compatibility

Recall that a principal is an identity that represents a user or service in your cluster. Principal compatibility is the process of standardizing identities (principals) across different authentication mechanisms, such as mTLS and OAuth, into a unified format. Extraction refers to the process of obtaining the principal (identity) from the raw authentication credentials.

For mTLS, principals are extracted from the Distinguished Name (DN) field of the TLS certificate. For example:

DN: CN=client1,OU=engineering,O=example.com,L=SF,ST=CA,C=US

For OAuth, principals are extracted from the JWT token issued by the identity provider. For example:

JWT claim: sub: client1

The Confluent Platform implementation of principal compatibility integrates seamlessly with Kafka. It also provides tools to map principals from mTLS and OAuth into a standard format, streamlining RBAC and ACL management. For both mTLS and OAuth principals, Confluent Platform maps the principals into this format:

Principal: User:client1

If you decide to migrate authentication from mTLS to OAuth, ensure your principal definitions between the two are compatible. With compatible principals you can continue to use your existing configuration mechanisms such as access control lists (ACLs) or role-based access control (RBAC).

If you used a principal mapper to formulate your identities, you can use it to confirm that switching from mTLS to OAuth does not require changes to ACLs or RBAC in your cluster. For details about using the principal mapper, see Use Principal Mapping in Confluent Platform.

Cluster architecture before and after

The diagrams bellow depict the impact of moving from a pure mTLS environment to one that makes use of OAuth. In the first diagram, mTLS is configured the same throughout for clients, brokers, and Confluent Platform services:

Before migrate to all authentication provided using mTLS

The Confluent Platform cluster configuration in the next diagram supports simultaneous mTLS and OAuth authentication. The addition of the AuthenticationHandler to the Schema Registry and Connect makes this simultaneous authentication support possible. In addition to handling mTLS requests, the OAuth listener is also enabled for the broker to handle OAuth requests.

After migrate mTLS for server-to-server authentication and OAuth for clients to cluster services

In this second topology, the Client App1, ACLs which were previously evaluated based on mTLS are instead evaluated based on the OAuth principal. This is possible without any ACL change because the Client App1 OAuth principal and mTLS principal are identical.

Client App2 continues to use mTLS to authenticate to the Confluent Platform cluster, and the ACLs are evaluated based on the mTLS principal.

How to Migrate

This section explains the processes you need to follow to migrate from mTLS based to OAuth authentication in your Confluent Platform clusters.

  1. List the existing ACLs, using the following Confluent CLI command:

    confluent kafka acl list

    This command lists the ACLs currently in place. Save this list for comparison after you complete the migration.

  2. Make sure you are running Confluent Platform version 7.7 or later.

  3. Configure the AuthenticationHandler according to Use the AuthenticationHandler Class for Multi-Protocol Authentication in Confluent Platform.

  4. Enable OAuth/OIDC for Confluent Server and other Confluent Platform services.

  5. Change one or more client authentication from mTLS to OAuth.

  6. Bring your cluster up.

  7. Restart your client applications.

  8. List the ACLs again to verify the principals remain compatible.

    confluent kafka acl list

    Compare the list to the one you created in step 1. If the principals remain the same, no changes to the ACLs are necessary. The ACLs continue to work as before and are evaluated based on the OAuth principal instead of the mTLS principal.

Alternatively, you can Deploy with Ansible Playbooks and Confluent for Kubernetes to upgrade to 7.7 and migrate from mTLS to OAuth.

Troubleshooting

If you encounter issues during or after the migration, consider the following:

Verify OAuth configuration
Ensure that your OAuth settings are correct and that your identity provider is properly configured.
Check ACL mappings
Confirm that the OAuth principals correctly map to the intended mTLS principals.
Review logs
Check Confluent Platform logs for any authentication or authorization errors.
Test with both authentication methods
If possible, test access using both mTLS and OAuth to isolate any issues.

For further help, contact Confluent support.

Related content