Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Control Center System Requirements

This section is not an exhaustive guide to running Control Center in production, but it covers the key things to consider before going live.

Hardware

As of this release, Control Center must run on a single machine. The resources needed for this machine depend primarily on how many producers are monitored and how many partitions each producer writes to. The Stream Monitoring functionality of Control Center is implemented as a Kafka Streams application and consequently benefits from having a lot of memory to work with for RocksDB caches and OS page cache.

Memory

The more memory you give Control Center the better but we recommend at least 32 GB of RAM. The JVM heap size can be fairly small (defaults to 6 GB) but the application needs the additional memory for RocksDB in-memory indexes and caches as well as OS page cache for faster access to persistent data.

CPUs

The Stream Monitoring functionality of Control Center requires significant CPU power for data verification and aggregation. We recommend at least 8 cores. If you have more cores available, you can increase the number of threads in the Kafka Streams pool (confluent.controlcenter.streams.num.stream.threads) and increase the number of partitions on internal Control Center topics (confluent.controlcenter.internal.topics.partitions) for greater parallelism.

Disks

Control Center relies on local state in RocksDB. We recommend at least 300 GB of storage space, preferably SSDs. All local data is kept in the directory specified by the confluent.controlcenter.data.dir config parameter.

Network

Control Center relies heavily on Kafka, so fast and reliable network is important for performance. Modern data-center networking (1 GbE, 10 GbE) should be sufficient.

OS

Control Center needs many open RocksDB files. Make sure the ulimit for the number of open files (ulimit -n) is at least 16384.

JVM

We recommend running the latest version of JDK 1.8 with a 6 GB max heap size. JDK 1.7 is also supported.

Control Center stores cluster metadata and user data (triggers/actions) in the _confluent-command topic. This topic is not changed during an upgrade. To reset it, change the confluent.controlcenter.command.topic config to something else (e.g. _confluent-command-2) and restart Control Center, this will re-index the cluster metadata and remove all triggers/actions

Kafka

The amount of storage space needed in Kafka depends on how many producers and consumers are being monitored as well as the configured retention and replication settings.

By default, Control Center keeps 3 days worth of data for the monitoring topic _confluent-monitoring and metrics topic _confluent-metrics, and 24 hours of data of all of it’s internal topics. This means that you can take Control Center down for maintenance for as long as 24 hours without data loss. You can change these values by setting the following config parameters

  • confluent.monitoring.interceptor.topic.retention.ms
  • confluent.metrics.topic.retention.ms
  • confluent.controlcenter.internal.topics.retention.ms

Control Center also has a number of internal topics that it uses for aggregations. Data on these topics is kept with different retention periods based on the data type.

  • Streams Monitoring data is held at two retention levels, 96 hours for granular data, and 700 days for historical data. For example, if you have the same number of clients reading and writing granular data from the same number of topics, the amount of space that is required is about twice the amount needed for running at 96 hours.
  • Metrics data has a retention period of 7 days. With a constant number of topic partitions in a cluster, the amount of data that is used for metrics data should grow linearly and max out after 7 days of accumulation.

By default, Control Center stores 3 copies on all topic partitions for availability and fault tolerance.

The full set of configuration options are documented in Control Center Parameter Reference.