Actionable Insights for Confluent Cloud
Use actionable insights to get recommendations for your Confluent Cloud resources, delivered as notifications. Each insight describes a condition detected on a resource and a recommended action, so you don’t need to correlate multiple metrics yourself.
Note
The actionable insights feature is an Early Access Program feature in Confluent Cloud. To sign up, contact Confluent Support.
An Early Access feature is a component of Confluent Cloud introduced to gain feedback. This feature should be used only for evaluation and non-production testing purposes or to provide feedback to Confluent, particularly as it becomes more widely available in follow-on preview editions.
Early Access Program features are intended for evaluation use in development and testing environments only, and not for production use. Early Access Program features are provided: (a) without support; (b) “AS IS”; and (c) without indemnification, warranty, or condition of any kind. No service level commitment will apply to Early Access Program features. Early Access Program features are considered to be a Proof of Concept as defined in the Confluent Cloud Terms of Service. Confluent may discontinue providing preview releases of the Early Access Program features at any time in Confluent’s sole discretion.
Actionable insights build on the Metrics API, which returns raw telemetry data such as throughput, latency, lag, and utilization. Actionable insights evaluate that data at regular intervals and translate patterns into recommendations. Use the Metrics API to see what happened. Then you can use actionable insights to learn what it means and what to do.
Actionable insights use only metadata about your resources, such as topic names, cluster IDs, metric trends, and partition counts. They never access or process message payloads.
Available insights
Four insights are available, in three themes.
Insight |
Theme |
What it detects |
|---|---|---|
Idle topics detected |
Resource optimization |
Topics that contain data and have zero producers and zero consumers for seven days. |
Topics with high empty partitions |
Resource optimization |
Topics where 80% or more of the partitions are empty, evaluated over one day. |
Cluster projected to reach capacity |
Capacity planning |
Dedicated clusters projected to reach 80% capacity utilization within three days, based on the trend over the last seven days. |
Uneven paritions traffic with consumer lag |
Troubleshooting |
Topics where one partition receives a disproportionate share of produce traffic and consumer lag on that partition is increasing, evaluated over one day. |
Idle topics detected
The Idle topics detected insight flags a topic only when it contains data and has no producer activity and no consumer activity for the entire seven-day period. Review a flagged topic to decide whether you still need it. If you don’t, reclaim its resources.
Topics with high empty partitions
The Topics with high empty partitions insight flags topics where most partitions carry no traffic. A high percentage of empty partitions points to over-partitioning or poor key distribution. Review the partition count or the key strategy for the topic.
Cluster projected to reach capacity
The Cluster projected to reach capacity insight applies to Dedicated clusters only. It uses hourly cluster load data from the last seven days to project whether the cluster reaches 80% capacity utilization within the next three days. Plan a capacity expansion before the cluster reaches the threshold. For more about cluster types, see Kafka Cluster Types in Confluent Cloud.
Uneven partition traffic with consumer lag
The Uneven partition traffic with consumer lag insight flags a topic when one partition receives a disproportionate share of the topic’s produce traffic and consumer lag on that partition is increasing. Only one consumer in a consumer group reads each partition. So when traffic is concentrated on one partition, that consumer can fall behind even while the rest of the group keeps up. Review a flagged topic to decide whether to change the partition key strategy so traffic is spread more evenly, or to increase the processing throughput of the consumer reading that partition.
Limitations
Actionable insights have the following limitations:
You can’t change the detection thresholds. Confluent determines them.
Insights are evaluated using seven days of metric history. Insights based on a shorter history might be less accurate.
Receive insight notifications
To receive insights, subscribe to notifications in the Insights category. Confluent Cloud delivers insights through the notification system and lists the Insights category alongside the other notification categories. You can manage your preferences for the Insights category independently of other categories.
Insights are resource-level notifications. An insight is generated for a specific resource, such as a cluster or a topic, and goes only to users who have access to that resource.
To learn how to subscribe, choose delivery channels, and set up integrations, see Notifications for Confluent Cloud and Configure integrations and subscriptions.