Authenticate with SASL using JAAS in Confluent Platform

Apache Kafka® brokers support client authentication using SASL. SASL authentication can be enabled concurrently with TLS encryption (TLS client authentication will be disabled).

You can learn more about authentication with SASL in Kafka Authentication with TLS/SSL and SASL_SSL (module of Apache Kafka Security) free course.

Tip

These instructions are based on the assumption that you are installing Confluent Platform by using ZIP or TAR archives. For more information, see Install Confluent Platform On-Premises.

The supported SASL mechanisms are:

• GSSAPI (Kerberos authentication)
• OAUTHBEARER
• SCRAM
• PLAIN
• Delegation Tokens
• LDAP

See also

For an example that shows how to set Docker environment variables for Confluent Platform running in ZooKeeper mode, see the Confluent Platform demo. Refer to the demo’s docker-compose.yml file for a configuration reference.

JAAS Configurations

Kafka uses the Java Authentication and Authorization Service (JAAS) for SASL configuration. You must provide JAAS configurations for all SASL authentication mechanisms. There are two ways to configure Kafka clients to provide the necessary information for JAAS:

• Specify the JAAS configuration using the sasl.jaas.config configuration property (recommended)
• Pass a static JAAS configuration file into the JVM using the java.security.auth.login.config property at runtime

Recommended Broker JAAS Configuration

Brokers can configure JAAS using the broker configuration property sasl.jaas.config. You must prefix the property name with the listener prefix, including the SASL mechanism, as shown below:

listener.name.<listenerName>.<saslMechanism>.sasl.jaas.config

You can only specify one login module in the configuration value. To configure multiple mechanisms on a listener, you must provide a separate configuration for each mechanism using the listener and mechanism prefix. For example:

listener.name.sasl_ssl.scram-sha-256.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="admin" \
  password="admin-secret";
listener.name.sasl_ssl.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="admin" \
  password="admin-secret" \
  user_admin="admin-secret" \
  user_alice="alice-secret";

Note that in cases where you have more listeners (for example, one for interbroker communication, one for clients, and another for RBAC), you must identify which mechanism each listener should use.

When Kafka attempts to create a listener.name in a listener-scoped JAAS configuration, one of the following occurs:

• If you define listener.name.internal.sasl.enabled.mechanisms Kafka loads the property and replaces the global sasl.enabled.mechanisms with the current internal listener SASL mechanisms.
• If you do not use the method in the preceding bullet, then Kafka will attempt to load a JAAS configuration for each listener and SASL mechanism (as defined by the global sasl.enabled.mechanism).

Hence, if you have two listeners (internal, external) and two SASL mechanisms (SCRAM-SHA-512, OAUTHBEARER) Kafka will attempt to load a JAAS configuration for internal-scram-sha-512 (listener.name.internal.scram-sha-512.sasl.jaas.config), internal-oauthbearer (listener.name.internal.oauthbearer.jaas.config), external-scram-sha-512, external-oauthbearer, and so on.

If one combination does not provide a JAAS configuration, then the broker will fall back to the default java.security.auth.login.config value.

In the following example, SCRAM-SHA-512 is defined as the fallback mechanism, but it is explicitly overidden for the internal and external listeners:

sasl.enabled.mechanisms=SCRAM-SHA-512

listener.name.internal.sasl.enabled.mechanisms=SCRAM-SHA-512
listener.name.internal.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
     username="kafka" \
     password="kafka";

listener.name.external.sasl.enabled.mechanisms=OAUTHBEARER
listener.name.external.oauthbearer.sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \
     publicKeyPath="/tmp/conf/public.pem";

• For additional details about configuring multiple SASL mechanisms, refer to Enabling Multiple SASL Mechanisms.
• To view the SASL mechanism in an MDS configuration, refer to Full primary Kafka cluster MDS configuration.

Alternative JAAS Configuration

Alternatively, brokers can configure JAAS by passing a static JAAS configuration file into the JVM using the java.security.auth.login.config property at runtime. For example:

export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf"
kafka-server-start etc/kafka/server.properties

If JAAS configuration is defined at different levels, the order of precedence used is:

1. Broker configuration property listener.name.<listenerName>.<saslMechanism>.sasl.jaas.config
2. <listenerName>.KafkaServer section of static JAAS configuration
3. KafkaServer section of static JAAS configuration

KafkaServer is the section name in the JAAS file used by each broker. This section provides SASL configuration options for the broker, including any SASL client connections made by the broker for interbroker communication. If multiple listeners are configured to use SASL, you can prefix the section name with the lower-case listener name followed by a period, for example: sasl_ssl.KafkaServer.

Client JAAS Configurations

Clients also have two ways to configure JAAS:

1. (Preferred Method) Embed the JAAS configuration itself in the configuration property sasl.jaas.config.
2. Pass a static JAAS configuration file into the JVM using the java.security.auth.login.config property at runtime, as done with brokers.

Embed the JAAS configuration in sasl.jaas.config. For example:

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
   username="confluent" \
   password="confluent-secret";

If a client specifies both the client property sasl.jaas.config and the static JAAS configuration system property java.security.auth.login.config, then the client property sasl.jaas.config will be used.

JAAS and Confluent Control Center

There is one scenario when you should explicitly use the client property sasl.jaas.config instead of passing in the JVM argument -Djava.security.auth.login.config=/path/to/jaas.config, and that is when you are using Confluent Control Center that has its own Kafka cluster separate from the production cluster being monitored, and the security profile differs between the two clusters.

If you were to use java.security.auth.login.config, it would set a single configuration for the whole JVM for communicating to the production cluster being monitored as well as the cluster backing Control Center. This may be undesirable because this wouldn’t distinguish security profiles to each cluster.

Instead, use the client property to differentiate the security profiles (for example, different JAAS configurations), depending on the target cluster:

• Production cluster: for example, sasl.jaas.config
• Monitoring cluster: for Confluent Metrics Reporter use confluent.metrics.reporter.sasl.jaas.config, and for Confluent Monitoring Interceptors use confluent.monitoring.interceptor.sasl.jaas.config

Enabling Multiple SASL Mechanisms

You can enable multiple SASL mechanisms on the broker simultaneously while each client has to choose one mechanism.

Using JAAS Configuration Property (sasl.jaas.config)

The recommended method for specifying multiple SASL mechanisms on a broker is to use the broker configuration property sasl.jaas.config. You must prefix the property name with the listener prefix, including the SASL mechanism. For example:

listener.name.sasl_ssl.scram-sha-256.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="admin" \
  password="admin-secret";
listener.name.sasl_ssl.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="admin" \
  password="admin-secret" \
  user_admin="admin-secret" \
  user_alice="alice-secret";

To configure GSSAPI (Kerberos) and SCRAM SASL mechanisms for a sasl_plaintext listener:

sasl.enabled.mechanisms=SCRAM-SHA-512,GSSAPI
listener.name.sasl_plaintext.gssapi.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
  useKeyTab=true \
  storeKey=true \
  keyTab="/var/lib/secret/kafka.key" \
  principal="kafka/kafka.kerberos-demo.local@TEST.CONFLUENT.IO";
listener.name.sasl_plaintext.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="kafka" \
  password="kafka";

Using JAAS Configuration File

Optionally, you can specify multiple SASL mechanisms on a broker by specifying the configuration for the login modules of all enabled mechanisms in the KafkaServer section of the broker JAAS configuration file. For example:

   KafkaServer {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   storeKey=true
   keyTab="/etc/security/keytabs/kafka_server.keytab"
   principal="kafka/kafka1.hostname.com@EXAMPLE.COM";

   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_alice="alice-secret";
};

Enable the SASL mechanisms in server.properties:

# List of enabled mechanisms, can be more than one
sasl.enabled.mechanisms=GSSAPI,PLAIN

Specify the SASL security protocol and mechanism for interbroker communication in server.properties if required:

# Configure SASL_SSL if TLS/SSL encryption is enabled, otherwise configure SASL_PLAINTEXT
security.inter.broker.protocol=SASL_SSL

# Configure the appropriate interbroker protocol
sasl.mechanism.inter.broker.protocol=GSSAPI

Then follow the mechanism-specific steps in GSSAPI, SCRAM, and PLAIN to configure SASL for the enabled mechanisms.

Suggested Reading: Kafka Listeners - Explained

Modifying SASL mechanisms in a Running Cluster

To modify SASL mechanisms in a running cluster:

1. Enable new SASL mechanism by adding the mechanism to sasl.enabled.mechanisms in server.properties for each broker. Update the JAAS configuration file to include both mechanisms as described here. Incrementally restart the cluster nodes, taking into consideration the recommendations for doing rolling restarts to avoid downtime for end users..
2. Restart clients using the new mechanism (if required).
3. To change the interbroker communication mechanism (if required), set sasl.mechanism.inter.broker.protocol in server.properties to the new mechanism and incrementally restart the cluster again.
4. To remove the old mechanism (if required), remove the old mechanism from sasl.enabled.mechanisms in server.properties and remove the entries for the old mechanism from JAAS configuration file. Incrementally restart the cluster again.

Note that the sequence above is somewhat complex to cater for all possible mechanism changes. For example, to add a new mechanism in the brokers and swap the clients to use it, you would have to do steps 1 and 2.