Authentication using Delegation Tokens¶
Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing SASL/SSL methods. Delegation tokens are shared secrets between Apache Kafka® brokers and clients. Delegation tokens can help frameworks to distribute the workload to available workers in a secure environment without the added cost of distributing Kerberos TGT/keytabs or keystores when 2-way SSL is used.
The typical steps for delegation token usage are:
- You authenticate with the Kafka cluster via SASL or SSL, and obtain a delegation token. You can do this using the AdminClient APIs or the
- You provide the delegation token to Kafka clients securely for authenticating with the Kafka cluster.
- The token owner or renewer can renew or expire the delegation tokens.
A primary secret key is used to generate and verify delegation tokens. This is supplied using config
delegation.token.master.key. The same secret key must be configured across all the brokers.
If the secret is not set or is set to an empty string, the brokers will disable the delegation token authentication.
In the current implementation, token details are stored in ZooKeeper and they can be used in Kafka
installations where ZooKeeper is on a private network. The primary secret key is stored as
plain text in the
server.properties config file.
A token has a current life, and a maximum renewable life. By default, tokens must be renewed once every 24 hours
for up to 7 days. You can configure the token life by using the
delegation.token.max.lifetime.ms config options.
You can also explicitly cancel tokens. If a token is not renewed by the token’s expiration time or if token is beyond the maximum life time, it will be deleted from all broker caches and from ZooKeeper.
Creating Delegation Tokens¶
You can create tokens by using AdminClient APIs or by using
Delegation token requests (create/renew/expire/describe) must be issued on SASL or SSL authenticated channels.
Tokens cannot be requested if the initial authentication is done by using a delegation token.
kafka-delegation-tokens script examples are given below.
Create a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --create --max-life-time-period -1 \ --command-config client.properties --renewer-principal User:user1
Renew a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --renew --renew-time-period -1 \ --command-config client.properties --hmac ABCDEFGHIJK
Expire a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --expire --expiry-time-period -1 \ --command-config client.properties --hmac ABCDEFGHIJK
Existing tokens can be described using the
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --describe \ --command-config client.properties --owner-principal User:user1
Delegation token authentication piggybacks on the current SASL/SCRAM authentication mechanism. You must enable SASL/SCRAM mechanism on Kafka cluster as described here.
Configuring Kafka Clients¶
You can configure the JAAS configuration property for each client in
consumer.properties files. The login module describes how the clients like producer and consumer can
connect to the Kafka broker. The following is an example configuration for a client for the token authentication:
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \ username="tokenID123" \ password="lAYYSFmLs4bTjf+lTZ1LCHR/ZZFNA==" \ tokenauth="true";
password are used by clients to configure the token ID and
token HMAC. And the option
tokenauth is used to indicate the server about token authentication.
In this example, clients connect to the broker using token ID:
tokenID123. Different clients within a
JVM may connect using different tokens by specifying different token details in
Procedure to Manually Rotate the Secret¶
You must redeploy your Kafka cluster when the secret is rotated. During this process, currently connected clients will continue to work, but any new connection requests and renew and expire requests with old tokens can fail. Here are the steps:
- Expire all existing tokens.
- Rotate the secret by rolling upgrade.
- Generate new tokens.
Notes on Delegation Tokens¶
Currently, you can only create a delegation token for yourself. Owner and renewers can renew or expire tokens.
Owner and renewers can describe their own tokens. To describe others tokens, you must add
DESCRIBE permission on Token Resource.