MDS File-Based Authentication for Confluent Platform

This page explains how to configure the Metadata Service (MDS) in Confluent Platform to use file-based authentication.

Configuration Properties

To configure MDS for file-based authentication:

Set the following properties:
confluent.metadata.server.user.store
Set this property to FILE to enable file-based user storage.
confluent.metadata.server.user.store.file.path
Specify the absolute path to the file that will store the user credentials.
For example:
```
confluent.metadata.server.user.store=FILE
confluent.metadata.server.user.store.file.path=<FILE-path>
```
Confirm that the MDS process has read permissions for the FILE-path.
Note
In Confluent Platform 8.0.0 and later, additional security checks are enforced on the FILE and directory permissions. For details, see Security checks on FILE and directory permissions.
In Confluent Platform versions earlier than 8.0.0, you must restart MDS for changes to the FILE user store to take effect.
In Confluent Platform 8.0.0 and later, you can optionally configure MDS to refresh the FILE user store automatically, as described below.

Automatic FILE user store refresh

When confluent.metadata.server.user.store=FILE is configured, you can enable MDS to reload the FILE user store without restarting. To do this, set the following property:

# Interval in seconds to refresh the FILE user store.
# Default is 0, which disables automatic refresh.
# Values less than 0 also disable automatic refresh.
confluent.metadata.server.user.store.file.refresh.interval=5

The value is interpreted in seconds:

A positive value enables automatic refresh of the FILE user store. When a change is made to the FILE, MDS detects it, validates the file, and reloads users after the configured interval without requiring an MDS or broker restart.
A value of 0 (the default) disables automatic refresh. In this case, changes to the FILE user store take effect only after an MDS restart.
Negative values also disable automatic refresh.

Configure user credentials

The user FILE contains usernames and passwords. You can configure the file with basic plaintext passwords or with hashed passwords for stronger security. For production environments, you should use hashed passwords (preferably salted hashes) instead of plaintext passwords for improved security.

Basic plaintext configuration

For basic configurations, provide usernames and passwords in plaintext within the file. Add each entry on a new line in the format username:password, for example:

admin:admin-secret
alice:alice-secret
bob:bob-secret

Hash configuration

Important

This configuration applies to Confluent Platform versions 7.7.4, 7.8.3, 7.9.2, 8.0.0, and later.

You can maintain password hashes in the FILE using one of these MDS-supported hash algorithms:

CRYPT
MD5
SHA256
SHA384
SHA512

The format for hashed passwords is username:ALGORITHM:hash_value.

# SHA256 hashed passwords
admin:SHA256:d104909435cef2bd7567b281a7e95fd716cca56462208923a3020e2df7166c8a
user1:SHA256:39ea7b02d4cc77ca8f956e4650a3e116ab5200209d2cba813e1c14b32416dab9

# SHA512 hashed passwords
poweruser:SHA512:392241b7cd760920cc0cd81410cb818c18a174b8f76e07df10373f5a338f4035fc951e0269494e722438f040d79ef9dba8397bf88b1f80999f92792e2015b67

# Mixed format file (plaintext + hashed)
testuser:plaintext_password
hashuser:SHA256:6fa2288c361becce3e30ba4c41be7d8ba01e3580566f7acc76a7f99994474c46

Salted hashes ensure unique hash values, even for identical passwords. This makes brute-force attacks more difficult. The format for salted hashes is username:ALGORITHM:$salt$hash_value.

user1:SHA256:$salt-1$8f6ed82844bdc9b1d16907cf109affeb089e81440a771ac0622b963cf0b82a10
user2:SHA256:$salt-2$7136d0f6d3c229ab0a89185a9d1ca3bfa1b7b8538c9fdfb8cdfa4a587d7eee7e

Generating hashed passwords

This section outlines the process for generating basic and salted hashes.

Simple hash generation

To generate a basic hash for a password:

Choose your hashing algorithm (SHA256, SHA384, or SHA512).
Hash your password using the selected algorithm.
Format the entry as:
```
username:ALGORITHM:hash_value
```

Salted hash generation

For enhanced security with salted hash you would do the following:

Choose a unique salt value for each user.
Original password: mypassword
User Salt: user-specific-salt
Hash your original password with the chosen algorithm, for example:
```
hash1 = SHA256("mypassword")
```
Concatenate the hash result from the last step with the salt value:
```
combined = hash1 + "user-specific-salt"
```
Hash the concatenated string again with the same algorithm:
```
final_hash = SHA256(combined)
```
Format as:
```
username:ALGORITHM:$salt$final_hash
```

Adding custom metadata (optional)

You can append additional, colon-separated metadata to your credentials after the hash values.

username:ALGORITHM:hash_value:metadata1:metadata2

This additional metadata is intended for your organization to use and is completely ignored by Confluent Platform.

User store refresh and validation

When confluent.metadata.server.user.store=FILE and a positive confluent.metadata.server.user.store.file.refresh.interval is configured, Metadata Service (MDS) validates and reloads the FILE user store when changes are detected.

Validation behavior

On each refresh, MDS validates the FILE contents before loading them into the active user store. Validation includes:

Ensuring each entry matches one of the supported formats:
- username:password (plaintext).
- username:ALGORITHM:hash_value for hashed passwords.
- username:ALGORITHM:$salt$hash_value for salted hashed passwords.
Verifying that all entries use supported hash algorithms when hashes are configured (for example, SHA256, SHA384, or SHA512), as described in the hash configuration examples.

If validation fails:

MDS logs detailed error messages describing the invalid entries (for example, line numbers and invalid formats).
All authentication that depends on the FILE user store fails (fail-closed) until a valid FILE is provided and passes validation on a subsequent refresh.

Security checks on FILE and directory permissions

Each time MDS validates the FILE user store, it also enforces basic filesystem security checks on the credentials file and its parent directory:

The password file must not be world-readable.
The directory containing the password file must not be world-writable, unless the sticky bit is set.

If any of these checks fail, MDS treats the FILE user store as invalid and fails authentication until the file and directory permissions are corrected and the next validation succeeds.

Operational considerations

Important

File-based MDS: Required action before upgrading

Upgrading from 7.x to 8.0.4 or later
Upgrading from 8.0.0 - 8.0.3 to 8.0.4, 8.1.2, 8.2.0 or later
Upgrading from 8.1.0 - 8.1.1 to 8.1.2, 8.2.0 or later

If you are upgrading with FILE-based user store in a non-Ansible/CFK deployment, you must ensure that the password file and directory permissions meet the security requirements mentioned above before upgrading.

If these permissions are not corrected before upgrade, MDS will fail to start. For details, see Security checks on FILE and directory permissions.

With a non-zero refresh interval, changes to the FILE user store (such as adding, removing, or updating users) typically take effect within one refresh interval after the file is updated and passes validation.

If the FILE contents or permissions are invalid, all logins that depend on the FILE user store (for example, HTTP Basic authentication to MDS from Confluent Control Center or the Confluent CLI) fail until the file is fixed and a subsequent refresh completes successfully.