Important
You are viewing documentation for an older version of Confluent Platform. For the latest, click here.
Authentication using SASL¶
Prerequisites¶
Kerberos¶
If your organization is already using a Kerberos server (for example, by using Active Directory), there is no need to install a new server just for Kafka. Otherwise you will need to install one, your Linux vendor likely has packages for Kerberos and a short guide on how to install and configure it (Ubuntu, Redhat). Note that if you are using Oracle Java, you will need to download JCE policy files for your Java version and copy them to $JAVA_HOME/jre/lib/security
.
Kerberos Principals¶
If you are using the organization’s Kerberos or Active Directory server, ask your Kerberos administrator for a principal for each Kafka broker in your cluster and for every operating system user that will access Kafka with Kerberos authentication (via clients and tools).
If you have installed your own Kerberos, you will need to create these principals yourself using the following commands:
sudo /usr/sbin/kadmin.local -q 'addprinc -randkey kafka/{hostname}@{REALM}'
sudo /usr/sbin/kadmin.local -q "ktadd -k /etc/security/keytabs/{keytabname}.keytab kafka/{hostname}@{REALM}"
All hosts must be reachable using hostnames¶
It is a Kerberos requirement that all your hosts can be resolved with their FQDNs.
Configuring Kafka Brokers¶
- Add a suitably modified JAAS file similar to the one below to each Kafka broker’s config directory, let’s call it
kafka_server_jaas.conf
for this example (note that each broker should have its own keytab):
KafkaServer {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/etc/security/keytabs/kafka_server.keytab"
principal="kafka/kafka1.hostname.com@EXAMPLE.COM";
};
# Zookeeper client authentication
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/etc/security/keytabs/kafka_server.keytab"
principal="kafka/kafka1.hostname.com@EXAMPLE.COM";
};
- Pass the name of the JAAS file as a JVM parameter to each Kafka broker:
-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
You may also wish to specify the path to the krb5.conf
file (see JDK’s Kerberos Requirements for more details):
-Djava.security.krb5.conf=/etc/kafka/krb5.conf
- Make sure the keytabs configured in the JAAS file are readable by the operating system user who is starting the Kafka broker.
- Configure a SASL port in
server.properties
, by adding at least one ofSASL_PLAINTEXT
orSASL_SSL
to thelisteners
parameter, which contains one or more comma-separated values:
listeners=SASL_PLAINTEXT://host.name:port
If SASL_SSL
is used, then SSL must also be configured. Note that a PLAINTEXT
port is needed if you intend to use any client that does not support SASL.
If you are only configuring a SASL port (or if you want the Kafka brokers to authenticate each other using SASL) then make sure you set the same SASL protocol for inter-broker communication:
security.inter.broker.protocol=SASL_PLAINTEXT (or SASL_SSL)
We must also configure the service name in server.properties
, which should match the principal name of the Kafka brokers. In the above example, the principal is kafka/kafka1.hostname.com@EXAMPLE.com
, so:
sasl.kerberos.service.name="kafka"
Important notes:
KafkaServer
is a section name in the JAAS file used by each broker. This section tells the broker which principal to use and the location of the keytab where this principal is stored. It allows the broker to login using the keytab specified in this section.- The
Client
section is used to authenticate a SASL connection with zookeeper. It also allows the brokers to set SASL ACL on ZooKeeper nodes which locks these nodes down so that only the brokers can modify it. It is necessary to have the same principal name across all brokers. If you want to use a section name other thanClient
, set the system propertyzookeeper.sasl.client
to the appropriate name (e.g.-Dzookeeper.sasl.client=ZkClient
). - ZooKeeper uses
zookeeper
as the service name by default. If you want to change this, set the system propertyzookeeper.sasl.client.username
to the appropriate name (e.g.-Dzookeeper.sasl.client.username=zk
).
Configuring Kafka Clients¶
SASL authentication is only supported for the new Kafka producer and consumer, the older API is not supported. To configure SASL authentication on the clients:
1. Clients (producers, consumers, connect workers, etc) will authenticate to the cluster with their own principal (usually with the same name as the user running the client), so obtain or create these principals as needed. Then create a JAAS file for each principal.
The KafkaClient
section describes how the clients like producer and consumer can connect to the Kafka Broker. The following is an example configuration for a client using a keytab (recommended for long-running processes):
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/etc/security/keytabs/kafka_client.keytab"
principal="kafka-client-1@EXAMPLE.COM";
};
For command-line utilities like kafka-console-consumer
or kafka-console-producer
, kinit
can be used along with useTicketCache=true
as in:
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true;
};
- Pass the name of the JAAS file as a JVM parameter to the client JVM:
-Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf
You may also wish to specify the path to the krb5.conf
file (see JDK’s Kerberos Requirements for more details).
-Djava.security.krb5.conf=/etc/kafka/krb5.conf
- Make sure the keytabs configured in the
kafka_client_jaas.conf
are readable by the operating system user who is starting kafka client. - Configure the following properties in
producer.properties
and/orconsumer.properties
:
security.protocol=SASL_PLAINTEXT (or SASL_SSL)
sasl.kerberos.service.name="kafka"