Use Confluent Platform systemd Service Unit Files

You can use Confluent Platform systemd service unit files to configure automatic startup and monitoring of services during system boot. These files are analogous to init scripts or startup scripts and they provide a standardized method for automatically starting Confluent Platform server components in the correct order on startup.

The systemd service unit files are optimized for staging and production deployments on Linux-based systems.

Overview

The systemd service unit files are included in the RPM and Debian packages for the following Confluent Platform components:

  • Control Center (control-center)
  • Apache Kafka® (kafka)
  • Kafka Connect (kafka-connect)
  • Confluent REST Proxy (kafka-rest)
  • ksqlDB (ksql)
  • Schema-Registry (schema-registry)
  • ZooKeeper (zookeeper)

Each component runs under its own user and a common confluent group that are set up during package installation. This configuration ensures proper security separation between components that are running on the same system. The usernames are prefixed with cp- followed by the component name. For example, cp-kafka and cp-schema-registry.

For components with persistent storage, such as Kafka and Confluent Control Center, the default component configuration file points to component-specific data directories /var/lib/<component>. For example, Kafka points to /var/lib/kafka and ZooKeeper points to /var/lib/zookeeper.

Usage

Enable automatic startup during boot

During installation, a component service is not enabled for automatic start on boot by default. To enable startup on boot, but not immediately, run this command.

sudo systemctl enable confluent-<component>

For example, you can enable the Confluent REST Proxy with the command:

sudo systemctl enable confluent-kafka-rest

Start service immediately

To start the service now, run this command:

sudo systemctl start confluent-<component>

This command will return immediately, although the service has not been started. The actual starting of the service is performed in the background by the systemd daemon.

Check service status

To check the status of a service, run this command:

systemctl status confluent-<component>

Look for the Active: line, which should say active.

Stop a service

To stop a service, run this command:

sudo systemctl stop confluent-<component>

Disable a service

To disable a service to prevent it from being automatically started on boot, run this command:

sudo systemctl disable confluent-<component>

Logging

Log files are located in component-specific directories /var/log/confluent/. For example, Schema Registry logs are located in /var/log/confluent/schema-registry.

The standard output of services is written to the log journal of a service. You can read the log journal with this command:

sudo journalctl -u confluent-<component>

To update the log4j file used for a component, set the <component>_LOG4J_OPTS environment variable.

To update the logging directory path used by the default log4j files, update the LOG_DIR environment variable.

For example, to change the log file directory for Confluent Control Center, you can set one of the following and restart Confluent Control Center:

export Environment="LOG_DIR=/new/path/to/log/confluent/control-center"

export Environment="CONTROL_CENTER_LOG4J_OPTS=-Dconfluent.controlcenter.log.dir=/new/path/to/log/confluent/control-center"

Follow the log

To follow the log in real-time, run this command:

sudo journalctl -f -u confluent-<component>

Modify configuration

Modify service unit configuration settings

To make changes to the service unit configuration settings, you should append configuration settings that overwrite previous defaults, rather than change existing configuration lines. This assures that the base service unit files can be upgraded cleanly.

The systemd component configuration settings are minimal and only control how the component is started, run, and stopped. The component configuration is managed through its existing /etc/<component>/<component>.properties configuration file, for example /etc/kafka/server.properties for the Kafka broker.

Modify the systemd component configuration

To modify the configuration settings for a service using the systemd builtin configuration editor, follow these steps:

  1. View the current systemd component configuration:

    systemctl cat confluent-<component>

  2. Run the edit command:

    sudo systemctl edit confluent-<component>

  3. Optional: Inspect the result.

    systemctl cat confluent-<component>

  4. Reload systemd configuration and restart the component.

    sudo systemctl reload
sudo systemctl restart confluent-<component>

To reconfigure the Confluent Platform component, refer to the component documentation and its configuration file. The configuration files are located in <path-to-confluent>/etc/<component>/<component>.properties. After reconfiguring the component, restart it with the following command to have the changes take affect:

sudo systemctl restart confluent-<component>

Additional resources