Use Delegation Tokens for Authentication in Confluent Platform¶

Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing SASL/SSL methods. Delegation tokens are shared secrets between Confluent Server brokers and Kafka clients. Delegation tokens can help frameworks to distribute the workload to available workers in a secure environment without the added cost of distributing Kerberos TGT/keytabs or keystores when mutual TLS (mTLS) is used.

The typical steps for delegation token usage are:

You authenticate with the Confluent Platform cluster via SASL or SSL, and obtain a delegation token. You can do this using the AdminClient APIs or the kafka-delegation-tokens script.
You provide the delegation token to Kafka clients securely for authenticating with your Confluent Platform cluster.
The token owner or renewer can renew or expire the delegation tokens.

Token management¶

A primary secret key is used to generate and verify delegation tokens. This is supplied using the configuration option delegation.token.master.key. The same secret key must be configured across all Confluent Server brokers. If the secret is not set or is set to an empty string, the brokers will disable the delegation token authentication.

In the current implementation, token details are stored in ZooKeeper and they can be used in Confluent Platform clusters where ZooKeeper is on a private network. The primary secret key is stored as plain text in the server.properties configuration file.

A token has a current life, and a maximum renewable life. By default, tokens must be renewed once every 24 hours for up to 7 days. You can configure the token life by using the delegation.token.expiry.time.ms and delegation.token.max.lifetime.ms configuration properties.

You can also explicitly cancel tokens. If a token is not renewed by the token’s expiration time or if token is beyond the maximum life time, it will be deleted from all broker caches and from ZooKeeper.

Create delegation tokens¶

You can create tokens by using AdminClient APIs or by using kafka-delegation-tokens script. Delegation token requests (create, renew, expire, and describe) must be issued on SASL or SSL authenticated channels. Tokens cannot be requested if the initial authentication is done by using a delegation token. kafka-delegation-tokens script examples are given below.

Create a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --create --max-life-time-period -1 \
  --command-config client.properties --renewer-principal User:user1

Renew a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --renew --renew-time-period -1 \
  --command-config client.properties --hmac ABCDEFGHIJK

Expire a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --expire --expiry-time-period -1 \
  --command-config client.properties  --hmac ABCDEFGHIJK

Existing tokens can be described using the --describe option:

kafka-delegation-tokens --bootstrap-server localhost:9092 --describe \
  --command-config client.properties  --owner-principal User:user1

Token authentication¶

Delegation token authentication piggybacks on the current SASL/SCRAM authentication mechanism. You must enable SASL/SCRAM mechanism on your Confluent Platform cluster, as described in Use SASL/SCRAM authentication.

Configure Kafka clients¶

You can configure the JAAS configuration property for each client in producer.properties or consumer.properties files. The login module describes how the clients like producer and consumer can connect to the Confluent Server broker. The following is an example configuration for a Kafka client to use token authentication:

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="tokenID123" \
  password="lAYYSFmLs4bTjf+lTZ1LCHR/ZZFNA==" \
  tokenauth="true";

The options username and password are used by clients to configure the token ID and token HMAC. And the option tokenauth is used to indicate the server about token authentication. In this example, clients connect to the Confluent Server broker using token ID: tokenID123. Different clients within a JVM may connect using different tokens by specifying different token details in sasl.jaas.config.

Rotate the secret¶

You must redeploy your Confluent Platform cluster when the secret is rotated. During this process, currently connected Kafka clients continue to work, but any new connection requests and renew and expire requests with old tokens can fail. Here are the steps:

Expire all existing tokens.
Rotate the secret by rolling upgrade.
Generate new tokens.

Notes on delegation tokens¶

Currently, you can only create a delegation token for yourself. Owner and renewers can renew or expire tokens. Owner and renewers can describe their own tokens. To describe others tokens, you must add DESCRIBE permission on Token Resource.