Confluent Cloud Cluster and Topic Configuration Settings¶
This topic describes the default Apache Kafka® cluster and topic configuration settings in Confluent Cloud as well as the topic settings that can be edited. For a complete description of all Kafka configurations, see Confluent Platform Configuration Reference.
When editing topic and cluster settings, remember the following:
- You cannot edit cluster settings on Confluent Cloud on Basic or Standard clusters, but you can edit certain topic configurations after a topic has been created. For a list of editable topic settings, see Custom topic settings for all cluster types.
- You can change some configuration settings on Dedicated clusters using the Kafka CLI. See Change cluster settings for Dedicated clusters.
- You cannot access internal Kafka topics, and therefore they cannot be edited. For example, the internal topic
__consumer_offsetsis not accessible. Also, topics that are not accessible do not count toward partition limits or partition billing charges. Topics created by managed Connectors and ksqlDB are accessible.
Access cluster settings in the Confluent Cloud Console¶
You can access settings for your clusters with the Cloud Console.
To do so:
Sign in to your Confluent account.
Select an environment and choose a cluster.
In the navigation menu, select Cluster Overview > Cluster settings and the cluster settings page displays.
Note
You may also see Networking and Security tabs depending on the type of cluster and how it is configured.
Custom topic settings for all cluster types¶
The following table lists default parameter values for custom topics. The table also includes minimum and maximum values where they are relevant, as well as whether the parameters are editable. To edit topic settings, see Edit a Topic.
Important
The following limitations apply to changing parameter values after a topic has been created:
- You cannot change the
cleanup.policyfromdeletetocompactafter a topic has been created. - The
num.partitionsvalue can only be increased and not decreased after a topic is created. - All other editable settings can be changed after topic creation, but the limits that applied at topic creation still apply.
| Parameter Name | Default | Editable | Min/Max |
|---|---|---|---|
| cleanup.policy | Delete | No | |
| compression.type | producer | No | |
| connection.failed.authentication.delay.ms | 5000 | No | |
| default.replication.factor | 3 | No | |
| delete.retention.ms | 86400000 | Yes | Max: 63113904000 |
| file.delete.delay.ms | 60000 | No | |
| flush.messages | 9223372036854775807 | No | |
| flush.ms | 9223372036854775807 | No | |
| group.max.session.timeout.ms | 1200000 | No | |
| index.interval.bytes | 4096 | No | |
| max.message.bytes (Dedicated) | 2097164 | Yes | Max : 20971520 |
| max.message.bytes (Standard, Basic) | 2097164 | Yes | Max : 8388608 |
| max.compaction.lag.ms | 9223372036854775807 | Yes | Min: 604800000 |
| message.downconversion.enable | true | No | |
| message.format.version | 2.3-IV1 | No | |
| message.timestamp.difference.max.ms | 9223372036854775807 | Yes | |
| message.timestamp.type | CreateTime | Yes | |
| min.cleanable.dirty.ratio | 0.5 | No | |
| min.compaction.lag.ms | 0 | Yes | |
| min.insync.replicas | 2 | Yes | |
| num.partitions | 6 | Yes, see Increase partitions | Varies, see: Confluent Cloud Features and Limits by Cluster Type |
| offsets.retention.minutes | 10080 | No | |
| preallocate | false | No | |
| retention.bytes | -1 | Yes | |
| retention.ms | 604800000 | Yes | |
| segment.bytes | 104857600 | Yes | Min: 52428800 Max: 1073741824 |
| segment.index.bytes | 10485760 | No | |
| segment.jitter.ms | 0 | No | |
| segment.ms | 604800000 | Yes | Min: 600000 |
| unclean.leader.election.enable | false | No |
Increase partitions¶
You can change the number of partitions for an existing topic (num.partitions) for all cluster types on a per-
topic basis.
To change the number of partitions, you use the kafka-topic script that is a part
of the Kafka command line tools.
(installed with Confluent Platform) with the following command.
bin/kafka-topics --bootstrap-server <hostname:port> --command-config <config_file> --alter --topic <topic_name> --partitions <number_partitions>``
Important
The only way to change the partition count on an existing topic is with the kafka-topic script mentioned previously.
You cannot change the number of partitions using the Confluent CLI or with Confluent Cloud APIs.
For more details, sign in to the the Confluent support portal and search for “How to increase the partition count for a Confluent Cloud hosted topic.”
Change cluster settings for Dedicated clusters¶
Some settings for Dedicated clusters can be modified using the kafka-configs script that
is a part of the
Kafka command line tools (installed as a part of Confluent Platform).
You can change:
- auto.create.topics.enable
- ssl.cipher.suites
- num.partitions
- log.cleaner.max.compaction.lag.ms
- log.retention.ms
Changes to the following settings are applied to your Confluent Cloud cluster without additional action on your part and are persistent until the setting is explicitly changed again.
Important
These settings apply only to Dedicated clusters and cannot be modified on Basic or Standard clusters.
Enable automatic topic creation¶
Automatic topic creation (auto.create.topics.enable) is disabled (false) by default to help prevent unexpected costs. You can enable it with the following
command.
bin/kafka-configs --bootstrap-server <bootstrap> --command-config <config-properties> --entity-type brokers --entity-default --alter --add-config auto.create.topics.enable=true
Restrict cipher suites¶
Restrict allowed TLS/SSL cipher suites (ssl.cipher.suites) with the following command.
bin/kafka-configs --bootstrap-server <bootstrap> --command-config config.properties --entity-type brokers --entity-default --alter --add-config ssl.cipher.suites=["list-item-1,list-item-2"]
The valid list of ciphers includes:
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
Change the default number of partitions for new topics¶
You can change the default number of partitions (num.partitions) for newly created topics with the following command.
bin/kafka-configs --bootstrap-server <bootstrap> --command-config <auth-configs> --entity-type brokers --entity-default --alter --add-config num.partitions=<int>
To change the number of partitons once a topic has been created, see Increase partitions.
Change max compaction log time¶
Set the default maximum log compaction time for new topics (log.cleaner.max.compaction.lag.ms)
with the following command.
bin/kafka-configs --bootstrap-server <bootstrap> --command-config <auth-configs> --entity-type brokers --entity-default --alter --add-config log.cleaner.max.compaction.lag.ms=<int>
Change log retention time¶
Set the default log retention time for new topics (log.retention.ms) with the following command.
bin/kafka-configs --bootstrap-server <bootstrap> --command-config <auth-configs> --entity-type brokers --entity-default --alter --add-config log.retention.ms=<int>