Metadata Service Configuration Settings

To enable the Metadata Service (also known as the Confluent Server Authorizer), the broker configuration in the server.properties file must set authorizer.class.name to io.confluent.kafka.security.authorizer.ConfluentServerAuthorizer.

To retain ACLs (that have already been enabled) and enable RBAC, set confluent.authorizer.access.rule.providers=ZK_ACL,CONFLUENT.

For more details about how to configure RBAC, refer to Enable RBAC in a Running Cluster.

RBAC supports the following Kafka configurations of the Metadata Service (MDS) back end, which you can override by using the prefixes specified below:

Configuration

confluent.metadata.server.listeners

Binds Metadata Service HTTP or HTTPS service to port.

  • Type: int

  • Default: 8090

confluent.metadata.server.advertised.listeners

Configures HTTP or HTTPS service advertised hostname.

  • Type: string

  • Default: null

confluent.metadata.server.token.max.lifetime.ms

Specifies a token’s maximum configured lifetime, in milliseconds.

  • Type: int

  • Default: 3600000

confluent.metadata.server.token.key.path

Location of the PEM-encoded public/private key pair to be used for signing and verifying tokens. Because the token service only supports RS256 signatures, key pairs must be generated using the RSA algorithm.

  • Type: string

  • Default: null

confluent.metadata.server.token.signature.algorithm

Signature scheme to be used when signing and verifying tokens as defined in (Algorithm) Header Parameter Values for JWS. Only RS256 is currently supported.

  • Type: string

  • Default: RS256

HTTPS configurations

Use the following attributes to configure MDS to allow connections over HTTPS.

confluent.metadata.server.ssl.keystore.location

The key store file location. This is optional for client and can be used for two-way authentication for client.

  • Type: string

  • Default: null

  • Importance: high

confluent.metadata.server.ssl.keystore.password

The key store file password. Optional for client, and only required if confluent.metadata.server.ssl.keystore.location is configured.

  • Type: password

  • Default: null

  • Importance: high

confluent.metadata.server.ssl.key.password

The private key password in the key store file. Optional for client.

  • Type: password

  • Default: null

  • Importance: high

confluent.metadata.server.ssl.truststore.location

The trust store file location.

  • Type: string

  • Default: null

  • Importance: high

confluent.metadata.server.ssl.truststore.password

The trust store file password. If a password is not set, access to the truststore is still available, but integrity checking is disabled.

  • Type: password

  • Default: null

  • Importance: high

The following is an example configuration for setting up HTTPS for MDS. Any content in brackets (<>) must be customized for your environment.

authorizer.class.name=io.confluent.kafka.security.authorizer.ConfluentServerAuthorizer
confluent.authorizer.access.rule.providers=ZK_ACL,CONFLUENT
confluent.metadata.server.listeners=https://localhost:8090
confluent.metadata.server.advertised.listeners=https://localhost:8090

confluent.metadata.server.ssl.keystore.location=<path-to-keystore>
confluent.metadata.server.ssl.keystore.password=<host-keystore-password>
confluent.metadata.server.ssl.key.password=<host-cert-password>
confluent.metadata.server.ssl.truststore.location=<path-to-truststore>
confluent.metadata.server.ssl.truststore.password=<host-truststore-password>

When logging in to an MDS that is configured for HTTPS, most clients, including the Confluent CLI, automatically pick up trusted certificates from the system truststore. To use self-signed or custom certificates that are not part of the system truststore when using the Confluent CLI, see the --ca-cert-path option in Confluent CLI confluent login.

REST client configurations

Components such as Schema Registry, Connect, and ksqlDB can interact as clients of MDS by means of REST Proxy. Configurations for these clients are defined in component-specific properties files; for example, ${CONFLUENT_HOME}/etc/ksqldb/ksql-server.properties or ${CONFLUENT_HOME}/etc/schema-registry/schema-registry.properties. The following properties can be configured in the component properties files to specify details related to MDS.

confluent.metadata.bootstrap.server.urls

A comma-separated list of valid URLs that specify where the metadata server or servers can be reached. For example: http://localhost:8080,http://localhost:8081. This configuration is required to run REST clients in an MDS environment.

  • Type: List

  • Default: “”

  • Importance: high

confluent.metadata.enable.server.urls.refresh

This option enables metadata server URLs to be refreshed when REST clients (for example, Schema Registry, Connect, and ksqlDB) are communicating with MDS. Set to false when using a DNS-based load balancer with MDS; in such cases clients do not need to look up the IP addresses of the MDS nodes.

  • Type: Boolean

  • Default: true

  • Importance: low

confluent.metadata.server.urls.fail.on.401

When set to true, the client will fail immediately upon receipt of a 401 status while trying to refresh metadata server URLs. This option is useful if repeated 401 responses cause issues in the backing LDAP server.

  • Type: string

  • Default: false

confluent.metadata.server.urls.max.retries

The number of retries to force a refresh of metadata server URLs.

  • Type: int

  • Default: 5

HTTP Response Headers

Metadata Service (MDS) responses can be configured to include specific HTTP headers, which is often a requirement for security audits and compliance. This allows security administrators to enforce policies such as HTTP Strict Transport Security (HSTS).

confluent.metadata.server.response.http.headers.config

Configures custom HTTP response headers for the Metadata Service. This property allows adding, setting, or removing specific HTTP headers in MDS responses. This is particularly useful for enforcing security policies like HSTS.

  • Type: string

  • Default: “”

  • Importance: high

For example, to enforce HTTP Strict Transport Security (HSTS) for the MDS endpoint, add the following to your MDS configuration (e.g., in server.properties):

confluent.metadata.server.response.http.headers.config="add Strict-Transport-Security: max-age=63072000"

This configuration instructs browsers to only interact with the MDS using HTTPS for the specified duration (63072000 seconds, which is 2 years).

For comprehensive HSTS enforcement across all Confluent Platform REST endpoints (for example, Kafka REST Proxy), you may also need to configure confluent.http.server.response.http.headers.config in the respective component’s kafka-rest.properties file.