<a id="docker-operations-logging"></a>

# Configure Kafka Logs for Docker in Confluent Platform

To configure Apache Kafka® logging in a Confluent Platform Docker container, set
`KAFKA_LOG4J_ROOT_LOGLEVEL` for the root log level and
`KAFKA_LOG4J_LOGGERS` for individual loggers. Kafka uses the
Java-based logging utility [Apache Log4j 2](https://logging.apache.org/log4j/2.12.x/). The same pattern
applies to other Confluent Platform components.

#### IMPORTANT
Confluent Platform and its components, version 7.9 and earlier, use Log4j 1.x
and not Log4j 2.x. This affects how you configure logging and the
format of the logging configuration file. For information about
logging in Confluent Platform 7.9 and earlier, see
[Configure Confluent Platform Logging (7.8)](https://docs.confluent.io/platform/7.8/monitor/cp-logging.html).

## Loggers for Confluent Platform components

Each Confluent Platform component has a Log4j environment-variable prefix. Replace
`{COMPONENT}` with the prefix for the component you are
configuring. For example, use `CONNECT_LOG4J_LOGGERS` to set loggers
for Kafka Connect. The following table lists each component and
its prefix.

To add or override default loggers, use the `{COMPONENT}_LOG4J_LOGGERS` environment variable.
This variable accepts comma-separated values that specify the logger and log level for that logger.

For more information on Log4j 2 settings, see [Log4j 2
Configuration](https://logging.apache.org/log4j/2.12.x/manual/configuration.html).

| Component                | {COMPONENT} or variable            |
|--------------------------|------------------------------------|
| Kafka                    | `KAFKA`                            |
| Confluent Control Center | `CONTROL_CENTER`                   |
| Schema Registry          | `SCHEMA_REGISTRY`                  |
| REST Proxy               | `KAFKA_REST`                       |
| Kafka Connect            | `CONNECT`                          |
| ksqlDB                   | `KSQL`                             |
| Replicator               | `REPLICATOR`                       |
| default loggers          | `{COMPONENT}_LOG4J_LOGGERS`        |
| root log level           | `{COMPONENT}_LOG4J_ROOT_LOGLEVEL`  |
| tools log level          | `{COMPONENT}_LOG4J_TOOLS_LOGLEVEL` |

## Log levels

You might choose to increase the log level if an issue occurs
and you are not getting sufficient log output to help you identify the cause. Note that increasing the log level can slow
service operation due to increased I/O load. In addition, increasing the log level can use more disk space.
If you have a size limit on the log files, you could lose older entries, which might be helpful for in debugging.

Following is a list of the supported log levels.

| Level   | Description                                                                       |
|---------|-----------------------------------------------------------------------------------|
| OFF     | Turns off logging.                                                                |
| FATAL   | Severe errors that cause premature termination.                                   |
| ERROR   | Other runtime errors or unexpected conditions.                                    |
| WARN    | Runtime situations that are undesirable or unexpected, but not necessarily wrong. |
| INFO    | Runtime events of interest at startup and shutdown.                               |
| DEBUG   | Detailed diagnostic information about events.                                     |
| TRACE   | Detailed diagnostic information about everything.                                 |

## Default loggers for Kafka

Kafka ships with default loggers. Setting a log level on a default
logger overrides the root logger level and changes the log output for
that logger. The following table lists each default logger.

| Logger                    | Description                                                                                                                                |
|---------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
| `kafka.request.logger`    | Displays all requests being served by the broker.<br/>A broker serving many requests has a high log volume when this is set to INFO level. |
| `kafka.controller`        | Provides information on state changes in the kafka cluster and is not verbose for a healthy cluster.                                       |
| `kafka.log.LogCleaner`    | Describes how and when log segment cleanup is occurring for topics that are using log compaction.                                          |
| `kafka.authorizer.logger` | Provided by the pluggable security authorizer. Increase verbosity of this log to help debug issues with authorization.                     |
| `state.change.logger`     | Tracks the state changes in the cluster. Typically<br/>not verbose in a healthy cluster.                                                   |

To see the default logger settings for the Kafka Docker image provided by Confluent, see
the [log4j2.yaml.template](https://github.com/confluentinc/kafka-images/blob/master/kafka/include/etc/confluent/docker/log4j2.yaml.template) on GitHub.

## Kafka log level examples

To override the log levels for the Kafka root, controller, and other default loggers,
use the following code:

```bash
KAFKA_LOG4J_ROOT_LOGLEVEL: 'WARN'
KAFKA_LOG4J_LOGGERS: 'kafka=WARN,kafka.controller=WARN,kafka.log.LogCleaner=WARN,state.change.logger=WARN,kafka.producer.async.DefaultEventHandler=WARN'
```

The following command changes the log levels for Kafka running in KRaft combined mode:

```bash
docker run -d \
    --name=kafka-log-example \
    --net=host \
    -e KAFKA_PROCESS_ROLES='broker,controller' \
    -e KAFKA_NODE_ID=1 \
    -e KAFKA_LISTENERS='PLAINTEXT://localhost:29092,CONTROLLER://localhost:29093' \
    -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP='CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT' \
    -e KAFKA_ADVERTISED_LISTENERS='PLAINTEXT://localhost:29092' \
    -e KAFKA_INTER_BROKER_LISTENER_NAME='PLAINTEXT' \
    -e KAFKA_CONTROLLER_LISTENER_NAMES='CONTROLLER' \
    -e KAFKA_CONTROLLER_QUORUM_VOTERS='1@localhost:29093' \
    -e KAFKA_JMX_PORT=39999 \
    -e KAFKA_LOG4J_LOGGERS="kafka=WARN,kafka.controller=WARN,kafka.log.LogCleaner=WARN,state.change.logger=WARN,kafka.producer.async.DefaultEventHandler=WARN" \
    -e KAFKA_LOG4J_ROOT_LOGLEVEL=WARN \
    -e KAFKA_LOG4J_TOOLS_LOGLEVEL=ERROR \
    -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
    -e CLUSTER_ID='MkU3OEVBNTcwNTJENDM2Qk' \
    confluentinc/cp-kafka:8.3.1
```

## Log4j log outputs

All log4j logs are sent to `stdout` by default. You can change this by
[extending the images](../development.md#log-to-external-volumes).

## Related content

- [Install Confluent Platform Using Docker](../installation.md#cpdocker-intro)
- [Configure Confluent Platform Logging](../../../monitor/cp-logging.md#cp-logging)
- [Kafka Connect Logging for Confluent Platform](../../../connect/logging.md#connect-logging)
- [Docker Image Reference for Confluent Platform](../image-reference.md#image-reference)
- [Monitor and Track Metrics in Docker with JMX in Confluent Platform](monitoring.md#use-jmx-monitor-docker-deployments)
- [Mount Docker External Volumes in Confluent Platform](external-volumes.md#external-volumes)
- Confluent Support: [StringMatchFilter To Exclude Messages From a Log](https://support.confluent.io/hc/en-us/articles/360030978652)