<a id="streams-developer-guide-topics"></a>

# Manage Kafka Streams Application Topics in Confluent Platform

A Kafka Streams application uses two kinds of Apache Kafka® topics: *user topics*,
which you create and manage to hold input and output data, and *internal
topics*, which Kafka Streams auto-creates for state store changelogs and
repartitioning. You manage user topics with standard Kafka topic tools,
while internal topics are created and named automatically by the
application based on its topology.

Kafka Streams distinguishes between [user topics](#streams-developer-guide-topics-user) and
[internal topics](#streams-developer-guide-topics-internal).

<a id="streams-developer-guide-topics-user"></a>

## User topics

User topics exist externally to an application. The application reads from or writes to user topics, including:

Input topics
: Topics that are specified by source processors in the application’s topology; for example, by `StreamsBuilder#stream()`, `StreamsBuilder#table()` and `Topology#addSource()`.

Output topics
: Topics that are specified by sink processors in the application’s topology; for example, by
  `KStream#to()`, `KTable.to()` and `Topology#addSink()`.

User topics must be created and manually managed ahead of time (for example, by using the
[topic tools](../../kafka/post-deployment.md#kafka-operations-admin)). If user topics are shared among multiple applications for reading and
writing, the application users must coordinate topic management. If user topics are centrally managed, application
users do not need to manage topics themselves; they only obtain access to them.

Don’t use the auto-create topic feature on the brokers to create user topics,
for the following reasons:

- Auto-creation of topics might be disabled in your Kafka cluster.
- Auto-creation automatically applies the default topic settings, such as the
  replication factor. These default settings might not be what you want for
  certain output topics, for example, `auto.create.topics.enable=true` in the
  [Kafka broker configuration](../../installation/configuration/broker-configs.md#cp-config-brokers).

<a id="streams-developer-guide-topics-internal"></a>

## Internal topics

Internal topics are used internally by the Kafka Streams application while executing, for example the
changelog topics for state stores. These topics are created by the application and are only used by that stream application.

From the broker perspective, internal topics are regular topics, in contrast to
“broker internal” topics like `__consumer_offsets` or `__transaction_state`.

If security is enabled on the Kafka brokers, you must grant the underlying clients admin permissions so that they can
create internal topics. For more information, see [Secure Deployment for Kafka Streams in Confluent Platform](security.md#streams-developer-guide-security).

The internal topics follow the naming convention `<application.id>-<operatorName>-<suffix>`,
but this convention isn’t guaranteed for future releases. For more information
about configuring parameters for internal topics, see
[Internal topic parameters](config-streams.md#streams-developer-guide-internal-topic-params).

If automatic topic creation has been disabled, Kafka Streams applications continue to work.
Kafka Streams applications use the Admin Client, so internal topics are still created.
Even if automatic topic creation is enabled, internal topics are created with
a specific number of partitions and a replication factor as specified in the `StreamsConfig`.

The following settings apply to the default configuration for internal topics:

- For all internal topics, `message.timestamp.type` is set to `CreateTime`.
- For internal repartition topics, the compaction policy is `delete` and
  the retention time is `-1` (infinite).
- For internal changelog topics for key-value stores, the compaction policy is
  `compact`.
- For internal changelog topics for windowed key-value stores, the compaction
  policy is `delete,compact`. The retention time is set to 24 hours plus your
  setting for the windowed store.
- For internal changelog topics for versioned state stores, the cleanup policy
  is `compact`, and `min.compaction.lag.ms` is set to 24 hours plus the
  store’s `historyRetentionMs` value.

#### NOTE
This website includes content developed at the [Apache Software Foundation](https://www.apache.org/)
under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).
