Frequently Asked Questions for Kafka Streams in Confluent Cloud

Q: Why don’t my Kafka Streams applications show up on the Kafka Streams page?

The most common causes are: Client library version. The Kafka Streams page requires applications built with Kafka Streams and Apache Kafka® client libraries version 4.0.0 or later. Metrics collection delay. After an application starts, Confluent Cloud can take a few minutes to collect and display its metrics. Self-managed clusters. If your application connects to a Confluent Platform cluster registered with Confluent Cloud through Unified Stream Manager, the application must use a Kafka Streams Java library based on Kafka 4.2.0 or later, because USM identifies active applications using the application ID label that Kafka adds starting in that version. Earlier client versions don’t appear in the monitoring views.

This FAQ answers common questions about the Confluent Cloud Console capabilities for Kafka Streams applications, including monitoring, application status, and the Kafka Streams Add-On. For general questions about building Kafka Streams applications, running them at scale, state stores, or troubleshooting application code, see the Kafka Streams FAQ for Confluent Platform.

Does Confluent Cloud run my Kafka Streams application for me?

No. Kafka Streams applications are client-side applications that you build and run on your own infrastructure. They don’t run inside Confluent Cloud or the Kafka brokers. What Confluent Cloud provides is a Kafka Streams page in the Confluent Cloud Console that visualizes the health and performance of applications connected to your cluster. For more information, see Monitor Kafka Streams Applications in Confluent Cloud.

How do I see my Kafka Streams applications in the Confluent Cloud Console?

Navigate to your cluster’s overview page in the Confluent Cloud Console and click Kafka Streams. The page lists every Kafka Streams application connected to the cluster, with aggregate metrics in the header and a table of per-application metrics. Click an application name to open its detail page, which shows per-thread status and performance charts. For step-by-step instructions, see Monitor Kafka Streams Applications in Confluent Cloud.

Why don’t my Kafka Streams applications show up on the Kafka Streams page?

The most common causes are:

Client library version. The Kafka Streams page requires applications built with Kafka Streams and Apache Kafka® client libraries version 4.0.0 or later.
Metrics collection delay. After an application starts, Confluent Cloud can take a few minutes to collect and display its metrics.
Self-managed clusters. If your application connects to a Confluent Platform cluster registered with Confluent Cloud through Unified Stream Manager, the application must use a Kafka Streams Java library based on Kafka 4.2.0 or later, because USM identifies active applications using the application ID label that Kafka adds starting in that version. Earlier client versions don’t appear in the monitoring views.

What metrics does the Kafka Streams page show?

The page shows a curated set of metrics for monitoring in the Confluent Cloud Console, not the full set of Java Management Extensions (JMX) metrics that the Kafka Streams library exposes. At the application level, you see running threads, production and consumption totals, memtable size, estimated key count, block cache usage, end-to-end latency, and a CPU ratio. The CPU ratio breaks down into commit, poll, process, and punctuate operations. At the thread level, you see status and a table of assigned tasks, including per-task consumer lag. For the complete set of JMX metrics available from the Kafka Streams library itself, which you can collect with your own metrics reporters, see Kafka Streams Metrics in Confluent Cloud.

What do the application statuses mean?

The Cloud Console derives an application’s status from the state of its threads:

Running: All threads are actively processing records.
Rebalancing: The application is reassigning tasks among its threads, for example, because an instance started or stopped.
Restoring: One or more threads are restoring state from a changelog topic, for example, after a new instance joins the application or after local state is lost.

Rebalancing and restoring are normal, typically short-lived phases of a Kafka Streams application’s lifecycle. If an application shows a Rebalancing or Restoring status for an extended period, check the application logs for errors. For more information, see Application status.

What is the Kafka Streams Add-On?

The Kafka Streams Add-On unlocks deeper diagnostics for applications that use the Streams Rebalance Protocol (group.protocol=streams). With the Add-On, the application overview page and thread detail panel show more diagnostics during rebalances and restores, such as per-thread task reassignment progress and state restoration progress. Without the Add-On, or if your application uses the classic group protocol, the application list, overview page, and thread detail panel are still available. However, Cloud Console shows a banner prompting you to get the Add-On. To request it, click Get Add-On on an application’s overview page, or contact your Confluent account team.

Can I monitor Kafka Streams applications running against a self-managed cluster?

Yes, if the cluster is Confluent Platform and registered with Confluent Cloud through Unified Stream Manager. This lets you view the same Kafka Streams page for applications connected to your on-premises Confluent Platform clusters. It requires Confluent Platform running in KRaft mode on version 8.2.2 or later, deployed with Confluent for Kubernetes 3.3.0 or later or Confluent Ansible 8.2.2 or later. You must also enable client monitoring on the Confluent Platform cluster, which collects Kafka Streams application metrics along with producer and consumer metrics. For details, see Monitor Confluent Platform Kafka Streams Applications with Unified Stream Manager.

Can I use Control Center and Unified Stream Manager client monitoring at the same time?

No. A Confluent Platform cluster can’t have both Control Center client metrics and USM client monitoring enabled simultaneously.

Does monitoring my Kafka Streams applications increase network traffic?

It can increase network traffic, depending on how your applications authenticate. Kafka Streams applications fetch metadata for every topic in the cluster, not just the topics they produce to or consume from. The Kafka Streams framework requires this broader metadata for its internal operations. If your applications use service accounts with the DataDiscovery or DataSteward role, this can increase network traffic compared to applications that use more restrictive service accounts. Monitor network usage and the request_count metric using the Metrics API, and determine whether a more restrictive role fits your use case. This behavior is independent of, and not caused by, the Kafka Streams monitoring page itself. For more information, see View thread and task details.

Where can I get help or support?

If you have questions or need support, see the Kafka Streams documentation for Confluent Cloud, ask in the Confluent Community Forum, or contact Confluent Support if you have a support plan.