Authentication using Delegation Tokens¶
Authentication based on delegation tokens is a lightweight authentication mechanism that you can use to complement existing SASL/SSL methods. Delegation tokens are shared secrets between Apache Kafka® brokers and clients. Delegation tokens can help frameworks to distribute the workload to available workers in a secure environment without the added cost of distributing Kerberos TGT/keytabs or keystores when 2-way SSL is used.
The typical steps for delegation token usage are:
- You authenticate with the Kafka cluster via SASL or SSL, and obtain a delegation token. You can do this using the
AdminClient APIs or the
kafka-delegation-tokens
script. - You provide the delegation token to Kafka clients securely for authenticating with the Kafka cluster.
- The token owner or renewer can renew or expire the delegation tokens.
Token Management¶
A primary secret key is used to generate and verify delegation tokens. This is supplied using config
option delegation.token.master.key
. The same secret key must be configured across all the brokers.
If the secret is not set or is set to an empty string, the brokers will disable the delegation token authentication.
In the current implementation, token details are stored in ZooKeeper and they can be used in Kafka
installations where ZooKeeper is on a private network. The primary secret key is stored as
plain text in the server.properties
config file.
A token has a current life, and a maximum renewable life. By default, tokens must be renewed once every 24 hours
for up to 7 days. You can configure the token life by using the delegation.token.expiry.time.ms
and
delegation.token.max.lifetime.ms
config options.
You can also explicitly cancel tokens. If a token is not renewed by the token’s expiration time or if token is beyond the maximum life time, it will be deleted from all broker caches and from ZooKeeper.
Creating Delegation Tokens¶
You can create tokens by using AdminClient APIs or by using kafka-delegation-tokens
script.
Delegation token requests (create/renew/expire/describe) must be issued on SASL or SSL authenticated channels.
Tokens cannot be requested if the initial authentication is done by using a delegation token.
kafka-delegation-tokens
script examples are given below.
Create a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --create --max-life-time-period -1 \
--command-config client.properties --renewer-principal User:user1
Renew a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --renew --renew-time-period -1 \
--command-config client.properties --hmac ABCDEFGHIJK
Expire a delegation token:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --expire --expiry-time-period -1 \
--command-config client.properties --hmac ABCDEFGHIJK
Existing tokens can be described using the --describe
option:
bin/kafka-delegation-tokens --bootstrap-server localhost:9092 --describe \
--command-config client.properties --owner-principal User:user1
Token Authentication¶
Delegation token authentication piggybacks on the current SASL/SCRAM authentication mechanism. You must enable SASL/SCRAM mechanism on Kafka cluster as described here.
Configuring Kafka Clients¶
You can configure the JAAS configuration property for each client in producer.properties
or consumer.properties
files. The login module describes how the clients like producer and consumer can
connect to the Kafka broker. The following is an example configuration for a client for the token authentication:
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
username="tokenID123" \
password="lAYYSFmLs4bTjf+lTZ1LCHR/ZZFNA==" \
tokenauth="true";
The options username
and password
are used by clients to configure the token ID and
token HMAC. And the option tokenauth
is used to indicate the server about token authentication.
In this example, clients connect to the broker using token ID: tokenID123
. Different clients within a
JVM may connect using different tokens by specifying different token details in sasl.jaas.config
.
Procedure to Manually Rotate the Secret¶
You must redeploy your Kafka cluster when the secret is rotated. During this process, currently connected clients will continue to work, but any new connection requests and renew and expire requests with old tokens can fail. Here are the steps:
- Expire all existing tokens.
- Rotate the secret by rolling upgrade.
- Generate new tokens.
Notes on Delegation Tokens¶
Currently, you can only create a delegation token for yourself. Owner and renewers can renew or expire tokens.
Owner and renewers can describe their own tokens. To describe others tokens, you must add DESCRIBE
permission on Token Resource.