Stream Designer for Confluent Cloud¶
Confluent Stream Designer is a high-productivity, easy to use, visual designer for building, testing, running, and monitoring data pipelines. With Stream Designer, you build pipelines end-to-end by using a non-programming based, low-code approach, with augmentation by code when you need it. Stream Designer is built on key Kafka components, like Core Kafka, Connect, ksqlDB, and Schema Registry. Stream Designer provides UI, CLI, and API interfaces.
Features¶
- Use the design canvas to build pipelines by using UI gestures only, no code.
- Enrich your pipelines with joins, filters, mappings, and custom partitions.
- Use the whole library of managed connectors to ingest data to your pipelines and sink data to other systems.
- Switch easily between the design surface and the code view of a pipeline.
- Activate and deactivate pipelines with a click.
- Save pipelines to SQL files and load them in the source code editor.
- Manage pipelines by using the Confluent CLI or the Pipelines REST API.
- Confluent CLI version 2.38.0 and later supports pipeline secrets and SQL code.
Known limitations¶
- When you import source code, you must tag existing topics manually. The Stream Designer backend can’t determine if any of the topics in the source code exist currently. However, an existing topic shows a warning that a topic with this name exists already.
- When you import source code, origin streams or tables must have schemas defined.
- Activating connectors may take a few minutes, which can cause downstream components to fail to activate. Once the connector is in the Running state, click Reactivate to activate downstream components.
- You can use Stream Designer only on Kafka clusters that are publicly available. You can’t use Stream Designer on Kafka clusters that use private networking (private link or VPC Peering).
If you enounter errors while creating or activating your pipelines, see Troubleshoot a Pipeline for Stream Designer on Confluent Cloud.