REST Proxy Configuration Options

In addition to the settings specified here, the Confluent REST Proxy accepts the settings for the Java admin, consumer and producer. Use these to override the default settings of producers and consumers in the REST Proxy. Use the client. prefix to override the default settings of admins, consumers and producers in Confluent REST Proxy. If you want the configuration to apply only to admins, consumers or producers, then replace the prefix with admin., consumer. or producer., respectively. When configuration options are exposed in the Confluent REST Proxy API, priority is given to settings in the user request, then to overrides provided as configuration options, and finally falls back to the default values provided by the Java Kafka clients.

General

id

Unique ID for the Confluent REST Proxy server instance. This is used in generating unique IDs for consumers that do not specify their ID. The ID is empty by default, which makes a single server setup easier to get up and running, but is not safe for multi-server deployments where automatic consumer IDs are used.

  • Type: string

  • Default: “”

  • Importance: high

bootstrap.servers

A list of Kafka brokers to connect to. For example, PLAINTEXT://hostname:9092,SSL://hostname2:9092. This configuration is particularly important when Kafka security is enabled, because Kafka may expose multiple endpoints that all will be stored in ZooKeeper, but REST Proxy may need to be configured with just one of those endpoints. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. Because these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).

listeners

Comma-separated list of listeners that listen for API requests over either HTTP or HTTPS. If a listener uses HTTPS, the appropriate SSL configuration parameters need to be set as well.

  • Type: list

  • Default: http://0.0.0.0:8082

  • Importance: high

schema.registry.url

The base URL for Schema Registry that should be used by the serializer.

  • Type: string

  • Default: http://localhost:8081

  • Importance: high

consumer.request.max.bytes

Maximum number of bytes in unencoded message keys and values returned by a single request. This can be used by administrators to limit the memory used by a single consumer and to control the memory usage required to decode responses on clients that cannot perform a streaming decode. Note that the actual payload will be larger due to overhead from base64 encoding the response data and from JSON encoding the entire response.

  • Type: long

  • Default: 67108864

  • Importance: medium

consumer.threads

The maximum number of threads to run consumer requests on. Note that this must be greater than the maximum number of consumers in a single consumer group. The sentinel value of -1 allows the number of threads to grow as needed to fulfill active consumer requests. Inactive threads will ultimately be stopped and cleaned up.

  • Type: int

  • Default: 50

  • Importance: medium

consumer.request.timeout.ms

The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached.

  • Type: int

  • Default: 1000

  • Importance: medium

host.name

The host name used to generate absolute URLs in responses. If empty, the default canonical hostname is used.

  • Type: string

  • Default: “”

  • Importance: medium

access.control.allow.methods

Set value to Jetty Access-Control-Allow-Origin header for specified methods.

  • Type: string

  • Default: “”

  • Importance: low

access.control.allow.origin

Set value for Jetty Access-Control-Allow-Origin header. You may use * for any origin, or you can specify multiple origins separated by commas.

  • Type: string

  • Default: “”

  • Importance: low

response.http.headers.config

Use to select which HTTP headers are returned in the HTTP response for Confluent Platform components. Specify multiple values in a comma-separated string using the format [action][header name]:[header value] where [action] is one of the following: set, add, setDate, or addDate. You must use quotation marks around the header value when the header value contains commas. For example:

response.http.headers.config="add Cache-Control: no-cache, no-store, must-revalidate", add X-XSS-Protection: 1; mode=block, add Strict-Transport-Security: max-age=31536000; includeSubDomains, add X-Content-Type-Options: nosniff

  • Type: string

  • Default: “”

  • Importance: low

reject.options.request

Boolean indicating whether OPTIONS requests on REST proxy and API V3 are rejected. By default, sending a request with the OPTIONS method to all REST services from Confluent Platform (Connect worker, REST Proxy, Confluent Control Center (Legacy) REST endpoint, and so on) returns the list of available methods on the specified endpoint; for example: curl -X OPTIONS http://localhost:8083. When reject.options.request is set to true, requests with -X OPTIONS will be rejected; available methods will not be returned. Setting reject.options.request to true thereby allows you to protect API endpoints that are not specifically used by applications, so as to reduce the attack surface.

  • Type: boolean

  • Default: false

  • Importance: low

consumer.instance.timeout.ms

Amount of idle time before a consumer instance is automatically destroyed.

  • Type: int

  • Default: 300000

  • Importance: low

consumer.iterator.backoff.ms

Amount of time to backoff when an iterator runs out of data. If a consumer has a dedicated worker thread, this is effectively the maximum error value for the entire request timeout. It should be small enough to closely target the timeout, but large enough to avoid busy waiting.

  • Type: int

  • Default: 50

  • Importance: low

fetch.min.bytes

Minimum number of bytes in message keys and values returned by a single request before the timeout of consumer.request.timeout.ms passes. The special sentinel value of -1 disables this functionality.

  • Type: int

  • Default: -1

  • Importance: medium

consumer.iterator.timeout.ms

Timeout for blocking consumer iterator operations. This should be set to a small enough value that it is possible to effectively peek() on the iterator.

  • Type: int

  • Default: 1

  • Importance: low

debug

Boolean indicating whether extra debugging information is generated in some error response entities.

  • Type: boolean

  • Default: false

  • Importance: low

metric.reporters

A list of classes to use as metrics reporters. Implementing the MetricReporter interface allows plugging in classes that will be notified of new metric creation. The JmxReporter is always included to register JMX statistics.

  • Type: list

  • Default: []

  • Importance: low

metrics.jmx.prefix

Prefix to apply to metric names for the default JMX reporter.

  • Type: string

  • Default: kafka.rest

  • Importance: low

metrics.num.samples

The number of samples maintained to compute metrics.

  • Type: int

  • Default: 2

  • Importance: low

metrics.sample.window.ms

The metrics system maintains a configurable number of samples over a fixed window size. This configuration controls the size of the window. For example, you might maintain two samples each measured over a 30 second period. When a window expires, you erase and overwrite the oldest window.

  • Type: long

  • Default: 30000

  • Importance: low

port

DEPRECATED: port to listen on for new connections. Use listeners instead.

  • Type: int

  • Default: 8082

  • Importance: low

producer.threads

Number of threads to run produce requests on.

  • Type: int

  • Default: 5

  • Importance: low

request.logger.name

Name of the SLF4J logger to write the NCSA Common Log Format request log.

  • Type: string

  • Default: io.confluent.rest-utils.request

  • Importance: low

response.mediatype.default

The default response media type that should be used if no specify types are requested in an Accept header.

  • Type: string

  • Default: application/json

  • Importance: low

response.mediatype.preferred

An ordered list of the server’s preferred media types used for responses, from most preferred to least.

  • Type: list

  • Default: [application/json, application/vnd.kafka.v2+json]

  • Importance: low

shutdown.graceful.ms

Amount of time to wait after a shutdown request for outstanding requests to complete.

  • Type: int

  • Default: 1000

  • Importance: low

kafka.rest.resource.extension.class

A list of classes to use as RestResourceExtension. Implementing the interface RestResourceExtension allows you to inject user defined resources like filters to REST Proxy. Typically used to add custom capabilities like logging, security, etc.

  • Type: list

  • Default: “”

  • Importance: low

advertised.listeners

List of advertised listeners. This configuration is used to generate absolute URLs in V3 responses. The HTTP and HTTPS protocols are supported. Each listener must include the protocol, hostname, and port. For example: http://myhost:8080 and https://0.0.0.0:8081.

  • Type: list

  • Default: “”

  • Importance: low

confluent.resource.name.authority

The authority where the governance of the name space is delegated to. This value is defined by the remainder of the CRN. This is used when generating Confluent resource names. Examples: confluent.cloud and mds-01.example.com.

  • Type: string

  • Default: “”

  • Importance: low

Configuration Options for HTTPS

ssl.keystore.location

Used for HTTPS. Location of the keystore file to use for TLS/SSL.

Important

Jetty requires that the key’s CN, stored in the keystore, must match the FQDN.

  • Type: string

  • Default: “”

  • Importance: high

ssl.keystore.password

Used for HTTPS. The store password for the keystore file.

  • Type: password

  • Default: “”

  • Importance: high

ssl.key.password

Used for HTTPS. The password of the private key in the keystore file.

  • Type: password

  • Default: “”

  • Importance: high

ssl.keystore.type

Used for HTTPS. The type of keystore file.

  • Type: string

  • Default: JKS

  • Importance: medium

ssl.protocol

Used for HTTPS. The TLS/SSL protocol used to generate the SslContextFactory.

  • Type: string

  • Default: TLS

  • Importance: medium

ssl.provider

Used for HTTPS. The TLS/SSL security provider name. Leave blank to use Jetty’s default.

  • Type: string

  • Default: “” (Jetty’s default)

  • Importance: medium

ssl.enabled.protocols

Used for HTTPS. A comma-separated list of protocols enabled for TLS/SSL connections. Leave blank to use Jetty’s defaults.

  • Type: list

  • Default: “” (Jetty’s default)

  • Importance: medium

ssl.keymanager.algorithm

Used for HTTPS. The algorithm used by the key manager factory for TLS/SSL connections. Leave blank to use Jetty’s default.

  • Type: string

  • Default: “” (Jetty’s default)

  • Importance: low

ssl.trustmanager.algorithm

Used for HTTPS. The algorithm used by the trust manager factory for TLS/SSL connections. Leave blank to use Jetty’s default.

  • Type: string

  • Default: “” (Jetty’s default)

  • Importance: low

ssl.cipher.suites

Used for HTTPS. A comma-separated list of TLS/SSL cipher suites. Leave blank to use Jetty’s defaults.

  • Type: list

  • Default: “” (Jetty’s default)

  • Importance: low

ssl.endpoint.identification.algorithm

Used for HTTPS. The endpoint identification algorithm to validate the server hostname using the server certificate. Leave blank to use Jetty’s default.

  • Type: string

  • Default: https

  • Importance: low

Configuration Options for TLS/SSL Encryption between REST Proxy and Apache Kafka Brokers

Note that all the TLS/SSL configurations (for REST Proxy to Broker communication) are prefixed with client.. If you want the configuration to apply just to admins, consumers or producers, you can replace the prefix with admin., consumer. or producer. respectively.

In addition to these configurations, make sure bootstrap.servers configuration is set with SSL://host:port end-points, or you’ll accidentally open a TLS/SSL connection to a non-SSL port.

Keep in mind that authenticated and encrypted connection to Kafka Brokers will only work when Kafka is running with appropriate security configuration. For details, see Kafka Security.

client.security.protocol

Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL.

  • Type: string

  • Default: PLAINTEXT

  • Importance: high

client.ssl.key.password

The password of the private key in the key store file. This is optional for client.

  • Type: password

  • Default: null

  • Importance: high

client.ssl.keystore.location

The location of the key store file. This is optional for client and can be used for two-way authentication for client.

  • Type: string

  • Default: null

  • Importance: high

client.ssl.keystore.password

The store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured.

  • Type: password

  • Default: null

  • Importance: high

client.ssl.truststore.location

The location of the trust store file.

  • Type: string

  • Default: null

  • Importance: high

client.ssl.truststore.password

The password for the trust store file.

  • Type: string

  • Default: null

  • Importance: high

client.ssl.enabled.protocols

The list of protocols enabled for SSL connections.

  • Type: list

  • Default: TLSv1.2,TLSv1.1,TLSv1

  • Importance: medium

client.ssl.keystore.type

The file format of the key store file. This is optional for client.

  • Type: string

  • Default: JKS

  • Importance: medium

client.ssl.protocol

The TLS/SSL protocol used to generate the SSLContext. Default setting is TLS, which is fine for most cases. Allowed values in recent JVMs are TLS, TLSv1.1 and TLSv1.2. SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.

  • Type: string

  • Default: TLS

  • Importance: medium

client.ssl.provider

The name of the security provider used for TLS/SSL connections. Default value is the default security provider of the JVM.

  • Type: string

  • Default: null

  • Importance: medium

client.ssl.truststore.type

The file format of the trust store file.

  • Type: string

  • Default: JKS

  • Importance: medium

client.ssl.cipher.suites

A list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using the TLS/SSL network protocol. By default all the available cipher suites are supported.

  • Type: list

  • Default: null

  • Importance: low

client.ssl.endpoint.identification.algorithm

The endpoint identification algorithm to validate server hostname using server certificate.

  • Type: string

  • Default: null

  • Importance: low

client.ssl.keymanager.algorithm

The algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.

  • Type: string

  • Default: SunX509

  • Importance: low

client.ssl.secure.random.implementation

The SecureRandom PRNG implementation to use for SSL cryptography operations.

  • Type: string

  • Default: null

  • Importance: low

client.ssl.trustmanager.algorithm

The algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.

  • Type: string

  • Default: PKIX

  • Importance: low

Configuration Options for SSL Encryption between REST Proxy and Schema Registry

Keep in mind that authenticated and encrypted connection to Schema Registry will only work when Schema Registry is running with appropriate security configuration. For details, see Schema Registry.

schema.registry.ssl.key.password

The password of the private key in the key store file. This is optional.

  • Type: password

  • Default: null

  • Importance: high

schema.registry.ssl.keystore.location

The location of the key store file. This is optional and can be used for two-way SSL authentication.

  • Type: string

  • Default: null

  • Importance: high

schema.registry.ssl.keystore.password

The store password for the key store file. This is optional and only needed if ssl.keystore.location is configured.

  • Type: password

  • Default: null

  • Importance: high

schema.registry.ssl.truststore.location

The location of the trust store file.

  • Type: string

  • Default: null

  • Importance: high

schema.registry.ssl.truststore.password

The password for the trust store file.

  • Type: string

  • Default: null

  • Importance: high

Interceptor configuration options

REST Proxy supports interceptor configurations as part of Java new producer and consumer settings.

producer.interceptor.classes

Producer interceptor classes.

  • Type: string

  • Default: “”

  • Importance: low

consumer.interceptor.classes

Consumer interceptor classes.

  • Type: string

  • Default: “”

  • Importance: low

For example, to enable Confluent Control Center (Legacy) monitoring interceptors:

consumer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor producer.interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor

For more details about monitoring interceptors, see Confluent Monitoring Interceptors in Control Center (Legacy).

License configuration options

Important

License configurations are only required if you are using principal propagation. For more information, see REST Proxy Authentication.

confluent.license

Confluent will issue a license key to each subscriber. The license key will be a short snippet of text that you can copy and paste. Without the license key, you can use Confluent security plugins for a 30-day trial period. If you are a subscriber and don’t have a license key, please contact Confluent Support at support@confluent.io.

  • Type: string

  • Default: “”

  • Importance: high