Authentication using Delegation Tokens¶

Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing SASL/SSL methods. Delegation tokens are shared secrets between Apache Kafka® brokers and clients. Delegation tokens can help frameworks to distribute the workload to available workers in a secure environment without the added cost of distributing Kerberos TGT/keytabs or keystores when bidirectiona TLS is used.

The typical steps for delegation token usage are:

You authenticate with the Kafka cluster via SASL or SSL, and obtain a delegation token. You can do this using the AdminClient APIs or the kafka-delegation-tokens script.
You provide the delegation token to Kafka clients securely for authenticating with the Kafka cluster.
The token owner or renewer can renew or expire the delegation tokens.

Token Management¶

A primary secret key is used to generate and verify delegation tokens. This is supplied using config option delegation.token.master.key. The same secret key must be configured across all the brokers. If the secret is not set or is set to an empty string, the brokers will disable the delegation token authentication.

In the current implementation, token details are stored in ZooKeeper and they can be used in Kafka installations where ZooKeeper is on a private network. The primary secret key is stored as plain text in the server.properties configuration file.

A token has a current life, and a maximum renewable life. By default, tokens must be renewed once every 24 hours for up to 7 days. You can configure the token life by using the delegation.token.expiry.time.ms and delegation.token.max.lifetime.ms config options.

You can also explicitly cancel tokens. If a token is not renewed by the token’s expiration time or if token is beyond the maximum life time, it will be deleted from all broker caches and from ZooKeeper.

Creating Delegation Tokens¶

You can create tokens by using AdminClient APIs or by using kafka-delegation-tokens script. Delegation token requests (create/renew/expire/describe) must be issued on SASL or SSL authenticated channels. Tokens cannot be requested if the initial authentication is done by using a delegation token. kafka-delegation-tokens script examples are given below.

Create a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --create --max-life-time-period -1 \
  --command-config client.properties --renewer-principal User:user1

Renew a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --renew --renew-time-period -1 \
  --command-config client.properties --hmac ABCDEFGHIJK

Expire a delegation token:

kafka-delegation-tokens --bootstrap-server localhost:9092 --expire --expiry-time-period -1 \
  --command-config client.properties  --hmac ABCDEFGHIJK

Existing tokens can be described using the --describe option:

kafka-delegation-tokens --bootstrap-server localhost:9092 --describe \
  --command-config client.properties  --owner-principal User:user1

Token Authentication¶

Delegation token authentication piggybacks on the current SASL/SCRAM authentication mechanism. You must enable SASL/SCRAM mechanism on Kafka cluster as described here.

Configuring Kafka Clients¶

You can configure the JAAS configuration property for each client in producer.properties or consumer.properties files. The login module describes how the clients like producer and consumer can connect to the Kafka broker. The following is an example configuration for a client for the token authentication:

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="tokenID123" \
  password="lAYYSFmLs4bTjf+lTZ1LCHR/ZZFNA==" \
  tokenauth="true";

The options username and password are used by clients to configure the token ID and token HMAC. And the option tokenauth is used to indicate the server about token authentication. In this example, clients connect to the broker using token ID: tokenID123. Different clients within a JVM may connect using different tokens by specifying different token details in sasl.jaas.config.

Procedure to Manually Rotate the Secret¶

You must redeploy your Kafka cluster when the secret is rotated. During this process, currently connected clients will continue to work, but any new connection requests and renew and expire requests with old tokens can fail. Here are the steps:

Expire all existing tokens.
Rotate the secret by rolling upgrade.
Generate new tokens.

Notes on Delegation Tokens¶

Currently, you can only create a delegation token for yourself. Owner and renewers can renew or expire tokens. Owner and renewers can describe their own tokens. To describe others tokens, you must add DESCRIBE permission on Token Resource.