.. meta:: :title: Kafka Monitoring and Metrics Using JMX with Docker :description: Connect JMX to Kafka in Confluent Plaform running in Docker. Manage clusters, collect broker/client metrics, and monitor Kafka system health in predefined dashboards with real-time alerting. .. _use-jmx-monitor-docker-deployments: Monitor and Track Metrics in Docker with JMX in |cp| #################################################### You can monitor |cp| deployments by using Java Management Extensions (JMX) and MBeans. .. note:: Confluent offers some alternatives to using JMX monitoring. - **Health+**: Consider monitoring and managing your environment with :ref:`health-plus`. Ensure the health of your clusters and minimize business disruption with intelligent alerts, monitoring, and proactive support based on best practices created by the inventors of |ak|. - **Confluent Control Center**: You can deploy :ref:`Control Center ` for out-of-the-box |ak| cluster monitoring so you don't have to build your own monitoring system. |c3-short| makes it easy to manage the entire |cp| deployment. |c3-short| is a web-based application that allows you to manage your cluster and to alert on triggers. Additionally, |c3-short| measures how long messages take to be delivered, and determines the source of any issues in your cluster. .. _configure-environment: Configure environment variables ******************************* JMX is enabled for |ak| by default. You can set the following JVM environment variables to configure JMX monitoring for your Docker image in a Compose file, a Dockerfile, or from the command line when you run |ak|. For an example of how to set the environment variables when you run |ak|, see :ref:`start-kafka-docker`. ``KAFKA_JMX_PORT`` The JMX Port. ``KAFKA_JMX_HOSTNAME`` The hostname associated with locally created remote objects. ``KAFKA_JMX_OPTS`` JMX options. Use this variable to override the default JMX options such as whether authentication is enabled. A JMX client must be able to connect to the ``java.rmi.server.hostname`` specified in the ``KAFKA_JMX_OPTS``. The default options are set as follows: .. codewithvars:: bash -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false The default for a bridged network is the bridged IP address, which means that you will only be able to connect to it from another Docker container. For the host network, this is the IP that the hostname on the host resolves to. Use ``hostname -i`` to see the what the hostname is set to in the Docker container. If you have more than one network configured for the container, ``hostname -i`` gives you all the IPs, and the default is the first IP address. Monitoring options ******************* For a list of JMX MBeans you can monitor, see :ref:`kafka_monitoring`. .. include:: ../../../includes/jmx-configure-security.rst .. _start-kafka-docker: Run |ak| with JMX enabled ************************* You run |ak| with JMX enabled in the same way that you normally start it, but you specify the ``KAFKA_JMX_PORT`` and ``KAFKA_JMX_HOSTNAME`` environment variables. If you are running |ak| in |zk| mode, specify ``KAFKA_JMX_PORT`` and ``KAFKA_JMX_HOSTNAME`` environment variables when you start both Kafka and |zk|. Following are example Docker ``run`` commands for Kafka running in |kraft| or |zk| mode with JMX configured: .. tabs:: .. tab:: |kraft| .. important:: For more information about configuring |kraft| in production, see :ref:`configure-kraft`. .. codewithvars:: bash docker run -d \ --name=kafka-jmx \ -h kafka-jmx \ -p 9101:9101 \ -e KAFKA_NODE_ID=1 \ -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP='CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT' \ -e KAFKA_ADVERTISED_LISTENERS='PLAINTEXT://kafka-jmx:29092,PLAINTEXT_HOST://localhost:9092' \ -e KAFKA_JMX_PORT=9101 \ -e KAFKA_JMX_HOSTNAME=localhost \ -e KAFKA_PROCESS_ROLES='broker,controller' \ -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ -e KAFKA_CONTROLLER_QUORUM_VOTERS='1@kafka-jmx:29093' \ -e KAFKA_LISTENERS='PLAINTEXT://kafka-jmx:29092,CONTROLLER://kafka-jmx:29093,PLAINTEXT_HOST://0.0.0.0:9092' \ -e KAFKA_INTER_BROKER_LISTENER_NAME='PLAINTEXT' \ -e KAFKA_CONTROLLER_LISTENER_NAMES='CONTROLLER' \ -e CLUSTER_ID='MkU3OEVBNTcwNTJENDM2Qk' \ confluentinc/cp-kafka:|release| .. tab:: |zk| .. include:: ../../../includes/zk-deprecation.rst .. codewithvars:: bash docker run -d \ --name=zk-jmx \ --net=host \ -e ZOOKEEPER_TICK_TIME=2000 \ -e ZOOKEEPER_CLIENT_PORT=32181 \ -e KAFKA_JMX_PORT=9101 \ confluentinc/cp-zookeeper:|release| docker run -d \ --name=kafka-jmx \ --net=host \ -e KAFKA_BROKER_ID=1 \ -e KAFKA_ZOOKEEPER_CONNECT=localhost:32181/jmx \ -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:39092 \ -e KAFKA_JMX_PORT=9101 \ -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \ confluentinc/cp-kafka:|release| View MBeans with JConsole *************************** To confirm you have successfully configured JMX monitoring with a Docker container, you can start `JConsole `__, a command-line utility provided with Java. To start JConsole, use the ``jconsole`` command, and connect to the Kafka process. For this example, the JMX settings for a Docker container running locally might look like the following: .. codewithvars:: bash KAFKA_JMX_PORT: 9101 KAFKA_JMX_HOSTNAME: localhost After JConsole starts, under **Remote Process**, enter the hostname and port you specified in your JMX configuration, and click **Connect**. .. image:: ../../../images/docker-jmx-jconsole.png :width: 500 If you have not configured authentication, you may be prompted to make an **Insecure connection**. .. image:: ../../../images/docker-jmx-insecure.png :width: 300 After JConsole is running, you can select the **MBeans** tab and expand the folders to see the JMX events and attributes for those events. .. image:: ../../../images/docker-jmx-mbeans.png :width: 500 .. _jmx-overrides-for-cp: .. include:: ../../../includes/jmx-cp-components.rst Related content *************** - :ref:`cpdocker_intro` - :ref:`image_reference` - :ref:`docker_operations_logging` - :ref:`external_volumes`