documentation
Get Started Free
  • Get Started Free
  • Courses
      What are the courses?

      Video courses covering Apache Kafka basics, advanced concepts, setup and use cases, and everything in between.

      Learning pathways (24)
      New Courses
      NewDesigning Event-Driven Microservices
      NewApache Flink® 101
      NewBuilding Flink® Apps in Java
      NewHybrid and Multicloud Architecture
      NewMastering Production Data Streaming Systems with Apache Kafka®
      Featured Courses
      Kafka® 101
      Kafka® Connect 101
      Kafka Streams 101
      Schema Registry 101
      ksqlDB 101
      Data Mesh 101
  • Learn
      Pick your learning path

      A wide range of resources to get you started

      Start Learning
      Articles

      Deep-dives into key concepts
      Patterns

      Architectures for event streaming
      FAQs

      Q & A about Kafka® and its ecosystem
      Blog

      The Confluent blog
      NewLearn More

      Our podcast, Streaming Audio
  • Build
      Design. Build. Run.

      Build a client app, explore use cases, and build on our demos and resources

      Start Building
      Language Guides

      Build apps in your favorite language
      Tutorials

      Hands-on stream processing examples
      Demos

      More resources to get you started
  • Community
      Join the Community

      Confluent proudly supports the global community of streaming platforms, real-time data streams, Apache Kafka®️, and its ecosystems

      Learn More
      Meetups & Events

      Kafka and data streaming community
      Ask the Community

      Community forums and Slack channels
      Community Catalysts

      Sharing expertise with the community
      DevX Newsletter

      Bi-weekly newsletter with Apache Kafka® resources, news from the community, and fun links.
      Data Streaming Awards

      Nominate amazing use cases and view previous winners
      NewCommunity Use Cases

      Real-world Examples of Apache Kafka® and Flink® in action
      NewKafka Summit 2024 - Bangalore

      Register now!
      NewKafka Summit 2024 - London
      NewCurrent 2024
      Current 2023

      View sessions and slides from Current 2023
      Kafka Summit 2023

      View sessions and slides from Kafka Summit 2023
Courses
What are the courses?

Video courses covering Apache Kafka basics, advanced concepts, setup and use cases, and everything in between.

Learning pathways (24)
New Courses
NewDesigning Event-Driven Microservices
NewApache Flink® 101
NewBuilding Flink® Apps in Java
NewHybrid and Multicloud Architecture
NewMastering Production Data Streaming Systems with Apache Kafka®
Featured Courses
Kafka® 101
Kafka® Connect 101
Kafka Streams 101
Schema Registry 101
ksqlDB 101
Data Mesh 101
Learn
Pick your learning path

A wide range of resources to get you started

Start Learning
Articles

Deep-dives into key concepts
Patterns

Architectures for event streaming
FAQs

Q & A about Kafka® and its ecosystem
Blog

The Confluent blog
NewLearn More

Our podcast, Streaming Audio
Build
Design. Build. Run.

Build a client app, explore use cases, and build on our demos and resources

Start Building
Language Guides

Build apps in your favorite language
Tutorials

Hands-on stream processing examples
Demos

More resources to get you started
Community
Join the Community

Confluent proudly supports the global community of streaming platforms, real-time data streams, Apache Kafka®️, and its ecosystems

Learn More
Meetups & Events

Kafka and data streaming community
Ask the Community

Community forums and Slack channels
Community Catalysts

Sharing expertise with the community
DevX Newsletter

Bi-weekly newsletter with Apache Kafka® resources, news from the community, and fun links.
Data Streaming Awards

Nominate amazing use cases and view previous winners
NewCommunity Use Cases

Real-world Examples of Apache Kafka® and Flink® in action
NewKafka Summit 2024 - Bangalore

Register now!
NewKafka Summit 2024 - London
NewCurrent 2024
Current 2023

View sessions and slides from Current 2023
Kafka Summit 2023

View sessions and slides from Kafka Summit 2023
Developer
Get Started Free
Confluent Documentation
  1. Home
  2. Platform
  3. ksqlDB and Kafka Streams

CONFLUENT PLATFORM

  • Overview
  • Get Started
    • What is Confluent Platform?
    • Quick Start for Confluent Platform
    • Kafka Basics on Confluent Platform
    • Introduction to Kafka
    • Kafka Design
    • Videos, Demos, and Reading Material
      • Scripted Confluent Platform Demo
        • Overview
        • Deploy Confluent Platform Environment
        • Deploy Hybrid Confluent Platform and Cloud Environment
        • End Demo
        • Troubleshooting
      • Tutorial: Introduction to Streaming Application Development
      • Clickstream Data Analysis Pipeline Using ksqlDB
      • RBAC Example for Confluent Platform
      • Replicator Schema Translation Example for Confluent Platform
      • DevOps for Kafka with Kubernetes and GitOps
        • Overview
        • Kafka DevOps Case Studies
          • Case Study: Graduated Environments
          • Case Study: Manage Cloud Secrets
          • Case Study: Kafka Connect management with GitOps
    • Course: Apache Kafka 101
  • Install and Upgrade
    • On-Premises Deployments
      • Overview
      • ZIP and TAR
      • Ubuntu and Debian
      • RHEL and CentOS
      • Docker
        • Install using Docker
        • Docker Configuration Parameters
        • Docker Image Reference
        • Docker Security
        • Docker Developer Guide
    • Confluent System Requirements
    • Confluent Platform Licenses
    • Confluent Platform Upgrade Checklist
    • Upgrade
    • Supported Versions and Interoperability
    • Using Confluent Platform systemd Service Unit Files
    • Confluent Platform Packages
    • Migrate to Confluent Platform
    • Migrate an Existing Kafka Deployment
    • Migrate to Confluent Server
  • Build Client Applications
    • Kafka Clients
      • Overview
      • Kafka Consumer
      • Kafka Producer
      • Client Guides
        • Python Client
        • .NET Client
        • Node.js Client
        • Go Client
        • Java Client
        • C++ Client
        • JMS Client
      • Client Examples
        • Overview
        • C/C++ Example
        • .NET Example
        • Go Example
        • Spring Boot Example
        • Java Example
        • KafkaProducer Example
        • Python Example
        • REST Example
        • Node.js Example
        • Clojure Example
        • Groovy Example
        • Kafka Connect Datagen Example
        • kafkacat Example
        • Kotlin Example
        • Ruby Example
        • Rust Example
        • Scala Example
      • Client APIs
        • Kafka C++ Client API
        • Kafka Python Client API
        • Kafka Go Client API
        • Kafka .NET Client API
        • JMS Client
          • Overview
          • JMS Client Installation
          • JMS Client Development Guide
    • Application Development
    • MQTT Proxy
      • Introduction
      • Communication Security Settings
      • MQTT Proxy Configuration Options
    • Command Line Tools
      • CLI Tools for Confluent Platform
      • Confluent CLI
    • kcat (formerly kafkacat) Utility
  • Confluent REST APIs
    • Overview
    • Quick Start
    • API Reference
    • Production Deployment
      • Confluent Server
        • Configuration
        • Security
      • REST Proxy
        • Overview
        • Configuration
        • Monitoring
        • Security
    • Connect to Confluent Cloud
    • REST Proxy Tutorial
  • ksqlDB and Kafka Streams
    • Overview
    • ksqlDB
      • ksqlDB Overview
      • ksqlDB Quickstart
      • Install ksqlDB
      • Operate ksqlDB
      • Upgrade ksqlDB
      • Develop applications for ksqlDB
      • Run ksqlDB in Confluent Cloud
      • Connect ksqlDB to Confluent Cloud
      • Migrate Confluent Cloud ksqlDB applications
      • Run ksqlDB in Confluent Control Center
      • Connect ksqlDB to Confluent Control Center
      • Secure ksqlDB with RBAC
      • Frequently Asked Questions
      • Troubleshoot ksqlDB issues
      • Tutorials and Examples
        • Tutorials overview
        • ksqlDB Quick Start
        • How-to Guides
        • Example Code Snippets
        • Materialized View/cache
        • Streaming ETL Pipeline
        • Event-driven Microservice
        • ksqlDB with Embedded Connect
        • Clickstream Data Analysis Pipeline Using ksqlDB
        • Kafka Tutorials Using ksqlDB
    • Kafka Streams
      • Kafka Streams Overview
      • Introduction
      • Build your first Streams application
      • Tutorial: Introduction to Streaming Application Development
      • Connect Confluent Platform Components to Confluent Cloud
      • Streams Concepts
      • Streams Architecture
      • Streams Code Examples
      • Streams Developer Guide
        • Writing a Streams Application
        • Testing Streams Code
        • Configuring a Streams Application
        • Streams DSL
        • Naming Kafka Streams DSL Topologies
        • Optimizing Kafka Streams Topologies
        • Processor API
        • Data Types and Serialization
        • Interactive Queries
        • Memory Management
        • Running Streams Applications
        • Managing Streams Application Topics
        • Kafka Streams Security
        • Application Reset Tool
      • Pipelining with Kafka Connect and Kafka Streams
      • Streams Operations
        • Capacity Planning and Sizing
        • Monitoring Kafka Streams Applications
      • Streams Upgrade Guide
      • Streams FAQ
      • Streams Javadocs
  • Connect to External Systems
    • Overview
    • Get Started
    • Kafka Connect 101
    • Connectors
    • Confluent Hub
      • Overview
      • Confluent Hub Client
      • Command Reference
        • Overview
        • confluent-hub help
        • confluent-hub install
      • Component Archive Specification
      • Contribute to Confluent Hub
    • Install
    • License
    • Supported
    • Preview
    • Configure
    • Concepts
    • Monitor
    • Logging
    • Connect to Confluent Cloud
    • Developer Guide
    • Tutorial: Moving Data In and Out of Kafka
    • Reference
      • Connect Javadocs
      • Connect REST Interface
      • Worker Configuration Properties
      • Connector Configuration Properties
    • Transformation
      • Overview
      • Cast
      • Drop
      • DropHeaders
      • EventRouter
      • ExtractField
      • ExtractTopic
      • Filter (Kafka)
      • Filter (Confluent)
      • Flatten
      • GzipDecompress
      • HeaderFrom
      • HoistField
      • InsertField
      • InsertHeader
      • MaskField
      • MessageTimestampRouter
      • RegexRouter
      • ReplaceField
      • SetSchemaMetadata
      • TimestampConverter
      • TimestampRouter
      • TombstoneHandler
      • TopicRegexRouter
      • ValueToKey
    • Security
      • Kafka Connect Security Basics
      • Kafka Connect and RBAC
        • Getting started with RBAC and Kafka Connect
        • Configuring RBAC for a Connect cluster
        • Configuring RBAC for a Connect worker
        • RBAC for self-managed connectors
        • Connect Secret Registry
        • Example Connect role-binding sequence
    • Design
    • Add Connectors and Software
    • Install Community Connectors
    • Upgrade
    • FileStream Connectors
    • FAQ
  • Schema Management
    • Overview
    • Installing and Configuring
      • Overview
      • Configuration Options
      • Run in Production
      • Connect to Confluent Cloud
      • Migrate Schemas
    • Schema Registry Tutorials
      • Tutorials Overview
      • On-Premises
      • Confluent Cloud
    • Schema Contexts
    • Schema Linking
    • Schema ID Validation on Confluent Server
    • Monitoring
    • Single and Multi-Datacenter Setup
    • Schema Evolution and Compatibility
    • Schemas in Control Center
    • Schemas on Confluent Cloud
    • Migrate Schemas
    • Deleting Schemas
    • Security
      • Schema Registry Security Overview
      • Role-Based Access Control
      • Schema Registry Security Plugin
        • Overview
        • Install and Configure
        • Schema Registry Authorization
          • Supported Operations and Resources
          • Role-Based Access Control
          • Schema Registry ACL Authorizer
          • Topic ACL Authorizer
    • Developer Guide
      • Overview
      • Maven Plugin
      • API Reference
      • API Usage Examples
    • Schema Formats
      • Formats, Serializers, and Deserializers
      • Avro
      • Protobuf
      • JSON Schema
    • Integrate Schemas from Connectors
  • Security
    • General Security
      • Security Overview
      • Security Tutorial
      • Configuring Confluent Server Authorizer
      • Cluster Registry
      • Security Compliance
      • Prefixes for Configuring Security
    • Authentication
      • Authentication Methods Overview
      • Authentication with SASL
        • Authentication with SASL using JAAS
        • Install
        • Configuring GSSAPI
        • Configuring OAUTHBEARER
        • Configuring PLAIN
        • Configuring SCRAM
        • Authentication using Delegation Tokens
        • Configuring Kafka Client Authentication with LDAP
      • Encrypt and Authenticate with TLS
      • HTTP Basic Authentication
      • Adding security to a running cluster
    • Authorization
      • Authorization using Role-Based Access Control
        • RBAC Overview
        • Quick Start
        • Predefined Roles
        • Enable RBAC in a Running Cluster
        • Discover Identifiers for Clusters
        • Configuring Token Authentication
        • Confluent Metadata API Reference
        • RBAC Example for Confluent Platform
      • Configure RBAC using the REST API
      • ACLs
        • Authorization using centralized ACLs
        • Authorization using ACLs
      • Group-Based Authorization Using LDAP
        • Configuring Confluent Server Authorizer
        • Configuring LDAP
        • Tutorial: Group-Based Authorization Using LDAP
    • Data Protection
      • Audit Logs
        • Audit Log Concepts
        • Auditable Events
        • Configure Audit Logs using the Confluent CLI
        • Configure MDS to Manage Centralized Audit Logs
        • MDS API Audit Log Configuration
        • Configure Audit Logs using the Properties File
      • Encrypt with TLS
      • Secrets
        • Secrets Management
        • Tutorial: Secret Protection
      • Redact Confluent Logs
    • Component Security
      • Confluent Control Center Security
        • Overview
        • Configure TLS/SSL
        • Configure SASL
        • Configure HTTP Basic Authentication
        • Authorize with Kafka ACLs
        • Configure LDAP
        • Configure RBAC
        • Manage and View RBAC Roles
          • Log in to Control Center when RBAC enabled
          • Manage RBAC roles with Control Center
          • View your RBAC roles in Control Center
      • Kafka Streams Security
      • Schema Registry Security
      • Kafka Connect Security
        • Kafka Connect Security Basics
        • Kafka Connect and RBAC
          • Getting started with RBAC and Kafka Connect
          • Configuring RBAC for a Connect cluster
          • Configuring RBAC for a Connect worker
          • RBAC for self-managed connectors
          • Connect Secret Registry
          • Example Connect role-binding sequence
      • ksqlDB RBAC
      • REST Proxy Security
        • REST Proxy Security
        • REST Proxy Security Plugins
      • ZooKeeper Security
    • Security Management Tools
      • Ansible Playbooks for Confluent Platform
      • Docker Security for Confluent Platform
  • Multi-DC Deployment Architectures
    • Overview
    • Multi-Data Center Architectures
    • Cluster Linking
      • Cluster Linking
      • Tutorial
      • Hybrid Cloud and Bridge-to-Cloud
      • Data Migration
      • Mirror Topics
      • Commands
      • Configuration Options
      • Metrics and Monitoring
      • Security
      • Frequently Asked Questions (FAQs)
    • Multi-Region Clusters
      • Overview
      • Tutorial: Multi-Region Clusters
      • Transition Standard Active-Passive Data Centers to a Multi-Region Stretched Cluster
    • Replicator for Multi-Datacenter Replication
      • Replicator Overview
      • Tutorial: Replicating Data Across Clusters
      • Active-active Demo
      • Download and Install
      • Configure and Run
      • Tuning Replicator
      • Monitoring Replicator
      • Replicator and Cross-Cluster Failover
      • Configuration Options
      • Migrate from MirrorMaker to Replicator
      • Replicator Verifier
      • Replicator to Cloud Configurations
  • Administer
    • Confluent Control Center
      • Control Center Overview
      • Installing and Configuring Control Center
        • Configuring
        • Configuration Reference
        • Check Control Center Version and Enable Auto-Update
        • Properties File
        • Connecting Control Center to Confluent Cloud
        • Confluent Monitoring Interceptors in Control Center
        • Installing Control Center on Kafka
        • Managing Confluent Platform Licenses
        • Troubleshooting Control Center
        • Upgrading Control Center
      • Clusters
      • Brokers
      • Topics Overview
        • Topics Overview
        • Create
        • View Topic Metrics
        • Message Browser
        • Manage Schemas for Topics
        • Edit Configuration Settings for Topics
        • Delete Topics
      • Connect
      • ksqlDB
      • Consumers
      • Replicators
      • Clusters
      • Control Center Alerts
        • Overview
        • Access Alerts and Alert History
        • Configure Alerts Properties
        • View and Manage Triggers
        • Manage Actions
        • Configure PagerDuty Emails with Alerts
        • REST API for Alerts History
        • Example Triggers and Actions
        • Troubleshooting Alerts
      • Security
    • Configuration Reference
      • Overview
      • Kafka Broker Configurations for Confluent Platform
      • Kafka Topic Configurations for Confluent Platform
      • Kafka Consumer Configurations for Confluent Platform
      • Kafka Producer Configurations for Confluent Platform
      • Kafka Connect Configurations for Confluent Platform
        • Overview
        • Sink Configurations
        • Source Configurations
      • Kafka AdminClient Configurations for Confluent Platform
      • License Configurations for Confluent Platform
      • Kafka Streams Configurations for Confluent Platform
      • Docker Configuration Parameters for Confluent Platform
      • Control Center Configurations
      • Streams Configurations
      • ZooKeeper Configurations
      • ksqlDB Server Configurations
    • CLI Tools for Confluent Platform
      • CLI Tools for Confluent Platform
      • Confluent CLI
    • Dynamic Kafka Configurations
    • Configure Multi-Node Environment
    • Metadata Service (MDS)
      • Configure Metadata Service (MDS)
      • Configure Confluent Platform Components to Communicate with MDS over TLS/SSL
      • Configure mTLS Authentication and RBAC for Kafka Brokers
      • Configure Kerberos Authentication for Brokers Running MDS
      • Configure LDAP Authentication
      • Configure LDAP Group-Based Authorization for MDS
      • Configure MDS to Manage Centralized Audit Logs
      • Metadata Service Configuration Options
      • Confluent Metadata API Reference
    • Confluent Metrics Reporter
    • Confluent Health+
      • Health+ Overview
      • Enable Health+
      • Health+ Intelligent Alerts
      • Health+ Monitoring Dashboard
      • Confluent Telemetry Reporter
      • Telemetry Reporter Metrics
      • Confluent Health+ FAQ
    • Confluent REST APIs
      • Overview
      • Quick Start
      • API Reference
      • Production Deployment
        • Confluent Server
          • Configuration
          • Security
        • REST Proxy
          • Overview
          • Configuration
          • Monitoring
          • Security
      • Connect to Confluent Cloud
      • REST Proxy Tutorial
    • Kafka Operations
      • Overview
      • Running Kafka in Production
      • Dynamic Kafka Configurations
      • Post Kafka Deployment
      • Self-Balancing Clusters
        • Self-Balancing Overview
        • Quick Start Demo (Docker)
        • Tutorial: Adding and Removing Brokers
        • Configuration Options and Commands
        • Performance and Resource Usage
      • Auto Data Balancing
        • Quick Start
        • Tutorial (Docker)
        • Command and Configuration Options
      • Monitor Kafka with JMX
      • Confluent Metrics Reporter
      • Tiered Storage
    • Docker Operations
      • Overview
      • Kafka Monitoring and Metrics Using JMX
      • Configure Docker Logging
      • Mounting Docker External Volumes
    • Post Kafka Deployment
    • ZooKeeper Operations
      • Running ZooKeeper in Production
      • Kafka Raft (KRaft)
    • Kafka Streams Operations
      • Capacity Planning and Sizing
      • Monitoring Kafka Streams Applications
    • ksqlDB Operations
    • DevOps for Kafka with Kubernetes and GitOps
      • Overview
      • Kafka DevOps Case Studies
        • Case Study: Graduated Environments
        • Case Study: Manage Cloud Secrets
        • Case Study: Kafka Connect management with GitOps
  • Confluent Health+
    • Health+ Overview
    • Enable Health+
    • Health+ Intelligent Alerts
    • Health+ Monitoring Dashboard
    • Confluent Telemetry Reporter
    • Telemetry Reporter Metrics
    • Confluent Health+ FAQ
  • Resources
  • Confluent CLI
  • Release Notes
    • Release Notes
    • Component Changelogs
  • API and Javadocs
    • Overview
    • Kafka Java APIs
      • Kafka Java Client APIs
      • Kafka Producer Java API
      • Kafka Consumer Java API
      • Kafka AdminClient Java API
      • Kafka Streams Java API
      • Kafka Connect Java API
    • Kafka Client APIs
      • Kafka C++ Client API
      • Kafka Python Client API
      • Kafka Go Client API
      • Kafka .NET Client API
      • JMS Client
        • Overview
        • JMS Client Installation
        • JMS Client Development Guide
    • Confluent APIs
      • Confluent Schema Registry API
      • ksqlDB API
      • Confluent REST API
      • Confluent Metadata API

ksqlDB¶

  • ksqlDB Overview
  • ksqlDB Quickstart
  • Install ksqlDB
  • Operate ksqlDB
  • Upgrade ksqlDB
  • Develop applications for ksqlDB
  • Run ksqlDB in Confluent Cloud
  • Connect ksqlDB to Confluent Cloud
  • Migrate Confluent Cloud ksqlDB applications
  • Run ksqlDB in Confluent Control Center
  • Connect ksqlDB to Confluent Control Center
  • Secure ksqlDB with RBAC
  • Frequently Asked Questions
  • Troubleshoot ksqlDB issues
  • Tutorials and Examples

Was this doc page helpful?

Give us feedback

Do you still need help?

Confluent support portal Ask the community
Thank you. We'll be in touch!
Be the first to get updates and new content

By clicking "SIGN UP" you agree that your personal data will be processed in accordance with our Privacy Policy.

  • Confluent
  • About
  • Careers
  • Contact
  • Professional Services
  • Product
  • Confluent Cloud
  • Confluent Platform
  • Connectors
  • Flink
  • Stream Governance
  • Developer
  • Free Courses
  • Tutorials
  • Event Streaming Patterns
  • Documentation
  • Blog
  • Podcast
  • Community
  • Forum
  • Meetups
  • Kafka Summit
  • Catalysts
Terms & Conditions Privacy Policy Do Not Sell My Information Modern Slavery Policy Cookie Settings Feedback

Copyright © Confluent, Inc. 2014- Apache, Apache Kafka, Kafka, the Kafka logo, Apache Flink, Flink, the Flink logo, Apache Iceberg, Iceberg, and the Iceberg logo are trademarks of the Apache Software Foundation

On this page: