Monitoring and Metrics in ksqlDB for Confluent Platform

Context

ksqlDB publishes metrics with Java Management Extensions (JMX), which help you monitor what is happening inside of ksqlDB’s servers. For a comprehensive list of metrics, see the reference section.

Enable monitoring

You must enable monitoring explicitly on each ksqlDB server. To enable it in a Docker-based deployment, export an environment variable named KSQL_JMX_OPTS with your JMX configuration and expose the port that JMX will communicate over.

By default, ksqlDB starts with JMX monitoring bound to localhost only, with no remote listener. To enable remote JMX, set the KSQL_JMX_OPTS environment variable with authentication and TLS configured explicitly, as shown in the following example.

The following Docker Compose example shows how you can configure remote monitoring for a ksqlDB server. The surrounding components, like the broker and CLI, are omitted for brevity. You can see an example of a complete setup in the ksqlDB Quick Start.

ksqldb-server:
  image: confluentinc/cp-ksqldb-server:8.1.6
  hostname: ksqldb-server
  container_name: ksqldb-server
  depends_on:
    - broker
    - schema-registry
  ports:
    - "8088:8088"
    - "1099:1099"
  environment:
    KSQL_LISTENERS: "http://0.0.0.0:8088"
    KSQL_BOOTSTRAP_SERVERS: "broker:9092"
    KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
    KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
    KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
    KSQL_KSQL_QUERY_PULL_METRICS_ENABLED: "true"
    KSQL_JMX_OPTS: >
      -Djava.rmi.server.hostname=localhost
      -Dcom.sun.management.jmxremote
      -Dcom.sun.management.jmxremote.port=1099
      -Dcom.sun.management.jmxremote.rmi.port=1099
      -Dcom.sun.management.jmxremote.authenticate=true
      -Dcom.sun.management.jmxremote.password.file=/etc/ksqldb/jmxremote.password
      -Dcom.sun.management.jmxremote.access.file=/etc/ksqldb/jmxremote.access
      -Dcom.sun.management.jmxremote.ssl=true

With respect to monitoring, here it what this does:

• The environment variable KSQL_JMX_OPTS is supplied to the server with various arguments. The > character lets you write a multi-line string in YAML, which makes this long argument easier to read. This example shows JMX authentication using a password file, a JMX access-control file, and SSL. Adjust the advertised hostname and ports for your environment. JMX has a wide range of configuration options.

• The jmxremote.password.file and jmxremote.access.file must be mounted into the container with the following permissions:

  • Password file: 0600 - Owner can read and write

  • Access file: 0644 - Everyone can read, owner can write

  For more about these files, see the JMX documentation.

• Port 1099 is exposed, which corresponds to the JMX port set in the KSQL_JMX_OPTS configuration. This enables remote monitoring tools to communicate into ksqlDB’s process.

Verifying your monitoring setup

An easy way to check that ksqlDB is properly emitting metrics is by using jconsole. JConsole is a graphical monitoring tool to monitor the JVM, and it is included in Oracle JDK installations.

On your host machine, run the command:

jconsole

You will be prompted for a host and port. If you used the example configuration above, then entering localhost:1099 will allow JConsole to establish the connection. You should see a series of graphs showing resource utilization. If you don’t, make sure the networking between your machine and the Docker container is configured correctly.