Manage Control Center Logs for Confluent Platform

Use this topic to enable and manage logs for Control Center.

Control Center logging

Control Center and other Confluent Platform components use the Java-based logging utility Apache Log4j2 to collect runtime data and record component events. The following table describes each log level.

Level

Description

OFF

Turns off logging.

FATAL

Severe errors that cause premature termination.

ERROR

Other runtime errors or unexpected conditions.

WARN

Runtime situations that are undesirable or unexpected, but not necessarily wrong.

INFO

Runtime events of interest at startup and shutdown.

DEBUG

Detailed diagnostic information about events.

TRACE

Detailed diagnostic information about everything.

By default, Control Center writes INFO, WARN, ERROR, and FATAL information to standard output (stdout). If you want to see all events, you can set DEBUG or TRACE logging levels.

You can select your desired log4j2 configuration by setting the CONTROL_CENTER_log4j2_OPTS environment variable when starting Control Center.

Considerations:

  • Control Center logging has migrated to Log4j2 library.

  • Log4j library is no longer supported.

  • The Log4j2 library uses the .yaml format instead of .properties.

  • You must update any custom logging configurations to use the new Log4j2 yaml format.

log4j2 yaml files

A default Control Center log4j2 template file is provided with Control Center. This yaml file is located in the directory: etc/confluent-control-center/log4j2.yaml

The following shows an example of what the default log file contains:

 1Configuration:
 2  status: WARN
 3  Appenders:
 4    Console:
 5      - name: Stdout
 6        Target: SYSTEM_OUT
 7        PatternLayout:
 8          pattern: "[%d{yyyy-MM-dd HH:mm:ss,SSS}] %p [%t] %m (%c)%n"
 9
10      - name: Streams
11        PatternLayout:
12          pattern: "[%d{yyyy-MM-dd HH:mm:ss,SSS}] %p [%t] %m (%c)%n"
13        Filters:
14          - Log4jRateFilter:
15              level: WARN
16              rate: 25
17
18  Loggers:
19    Root:
20      level: info
21      AppenderRef:
22        - ref: Stdout
23
24    Logger:
25      - name: org.apache.kafka.streams
26        level: info
27        AppenderRef:
28          - ref: Streams
29        additivity: false
30
31      - name: io.confluent.controlcenter.streams
32        level: info
33        AppenderRef:
34          - ref: Streams
35        additivity: false
36
37      - name: kafka
38        level: info
39        AppenderRef:
40          - ref: Stdout
41
42      - name: org.apache.kafka
43        level: info
44        AppenderRef:
45          - ref: Stdout
46
47      - name: org.apache.kafka.clients.consumer
48        level: info
49        AppenderRef:
50          - ref: Stdout
51
52      - name: org.apache.zookeeper
53        level: warn
54        AppenderRef:
55          - ref: Stdout
56
57      - name: org.I0Itec.zkclient
58        level: warn
59        AppenderRef:
60          - ref: Stdout

Confluent also provides etc/confluent-control-center/log4j2-rolling.yaml as an example of setting up Control Center with rolling log files that simplify log management. The rolling log file creates three, size-limited log files that isolate information:

  • control-center.log - Control Center, HTTP activity, anything not related to streams, REST API calls

  • control-center-streams.log - Streams

  • control-center-kafka.log - Client, ZooKeeper, and Kafka

The following shows an example of what the rolling log file contains:

 1Configuration:
 2  status: WARN
 3  name: Log4j2Config
 4
 5  Appenders:
 6    RollingFile:
 7    - name: Main
 8      fileName: ${sys:confluent.controlcenter.log.dir}/control-center.log
 9      filePattern: ${sys:confluent.controlcenter.log.dir}/control-center-%i.log
10      PatternLayout:
11        pattern: "[%d{yyyy-MM-dd HH:mm:ss,SSS}] %p [%t] %m (%c)%n"
12      Policies:
13        SizeBasedTriggeringPolicy:
14          size: 10MB
15      DefaultRolloverStrategy:
16        max: 5
17
18    - name: Streams
19      fileName: ${sys:confluent.controlcenter.log.dir}/control-center-streams.log
20      filePattern: ${sys:confluent.controlcenter.log.dir}/control-center-streams-%i.log
21      PatternLayout:
22        pattern: "[%d{yyyy-MM-dd HH:mm:ss,SSS}] %p [%t] %m (%c)%n"
23      Policies:
24        SizeBasedTriggeringPolicy:
25          size: 10MB
26      DefaultRolloverStrategy:
27        max: 5
28      Filters:
29        - Log4jRateFilter:
30            level: WARN
31            rate: 25
32
33    - name: Kafka
34      fileName: ${sys:confluent.controlcenter.log.dir}/control-center-kafka.log
35      filePattern: ${sys:confluent.controlcenter.log.dir}/control-center-kafka-%i.log
36      PatternLayout:
37        pattern: "[%d{yyyy-MM-dd HH:mm:ss,SSS}] %p [%t] %m (%c)%n"
38      Policies:
39        SizeBasedTriggeringPolicy:
40          size: 10MB
41      DefaultRolloverStrategy:
42        max: 5
43
44  Loggers:
45    Root:
46      level: info
47      AppenderRef:
48        - ref: Main
49
50    Logger:
51      - name: org.apache.kafka.streams
52        level: info
53        AppenderRef:
54          - ref: Streams
55        additivity: false
56
57      - name: io.confluent.controlcenter.streams
58        level: info
59        AppenderRef:
60          - ref: Streams
61        additivity: false
62
63      - name: kafka
64        level: info
65        AppenderRef:
66          - ref: Kafka
67        additivity: false
68
69      - name: org.apache.kafka
70        level: info
71        AppenderRef:
72          - ref: Kafka
73        additivity: false
74
75      - name: org.apache.zookeeper
76        level: warn
77        AppenderRef:
78          - ref: Kafka
79        additivity: false
80
81      - name: org.I0Itec.zkclient
82        level: warn
83        AppenderRef:
84          - ref: Kafka
85        additivity: false

You can add additional entries and change log levels by updating these configuration files.

Enable logs for Control Center

You set your logging configuration with a Control Center environment variable at start up. Use the following environment variable to log4j2 configuration:

  • CONTROL_CENTER_log4j2_OPTS

To set the logging configuration for Control Center:

  • Set and export the CONTROL_CENTER_log4j2_OPTS environment variable similar to this example:

    export CONTROL_CENTER_log4j2_OPTS='-Dlog4j2.configuration=file:etc/confluent-control-center/<log4j2.yaml>'

You must restart Control Center to apply changes to logging with an environment variable. For more information, see Restarting Control Center.

Change log levels for Control Center

You can change log levels to help debug issues. The following commands show how to set root logging levels to DEBUG or TRACE.

  • To enable debug logging, change the log level to DEBUG at the root level:

    1  Loggers:
2    Root:
3      level: debug
4      AppenderRef:
5        - ref: Stdout

  • To enable trace logging, change the root logger to TRACE at the root level:

    1  Loggers:
2    Root:
3      level: trace
4      AppenderRef:
5        - ref: Stdout

When you are done debugging and tracing, reset the log levels back to their defaults and restart Control Center. For more information, see Restarting Control Center.

Set custom log directory location

The location the logs are written depends on the type of installation.

  • RPM/Debian installation: /var/log/kafka

  • ZIP/TAR installation: $base_dir/logs

You set a custom logging location with an environment variable at start up. To set a custom location, use the following environment variable:

  • LOG_DIR

To set the custom location for logs:

  • Set and export the LOG_DIR environment variable similar to this example:

    export LOG_DIR='/my/logs/'

You must restart Control Center to apply changes to logging with an environment variable. For more information, see Restarting Control Center.

If you are using Ansible Playbooks for Confluent Platform, use the variable control_center_log_dir to write Control Center logs to a custom directory. For more information, see Ansible Playbooks for Confluent Platform.

Related content