>
Confluent
  • Product
    • Confluent Platform
    • KSQL
    • Confluent Hub
    • Subscription
    • Professional Services
    • Training
    • Customers
  • Cloud
    • Confluent Cloud
    • Support
    • Sign Up
    • Log In
    • Cloud FAQ
  • Developers
    • What is Kafka?
    • Resources
    • Events
    • Online Talks
    • Meetups
    • Kafka Summit
    • Kafka Tutorials
    • Confluent Developer
  • About Us
    • Company
    • Partners
    • News
    • Events
    • Careers
    • Contact
  • Blog
  • Docs
  • Download
Get Confluent | Sign up for Confluent Cloud or download Confluent Platform
  • Getting Started
    • What is the Confluent Platform?
      • What is included in the Confluent Platform?
        • Overview
        • Confluent Open Source
          • Confluent Connectors
            • Kafka Connect JDBC Connector
            • Kafka Connect HDFS Connector
            • Kafka Connect Elasticsearch Connector
            • Kafka Connect S3 Connector
          • Confluent Clients
            • C/C++ Client Library
            • Python Client Library
            • Go Client Library
            • .Net Client Library
          • KSQL
          • Confluent Schema Registry
          • Confluent REST Proxy
        • Confluent Enterprise
          • Automatic Data Balancing
          • Multi-Datacenter Replication
          • Confluent Control Center
          • JMS Client
        • Confluent Proactive Support
      • Migrating an existing Kafka deployment
        • Migrating developer tools
        • Migrating to Control Center
    • Confluent Platform Quick Start
      • Confluent Enterprise Quick Start
        • Step 1: Download and Start Confluent Platform
        • Step 2: Create Kafka Topics
        • Step 3: Create Sample Data
        • Step 4: Create and Write to a Stream and Table using KSQL
          • Create Streams and Tables
          • Write Queries
        • Step 5: View Your Stream in Control Center
        • Next Steps
      • Confluent Open Source Quick Start
        • Step 1: Download and Install
        • Step 2: Create Kafka Topics
        • Step 3: Create Sample Data
        • Step 4: Create and Write to a Stream and Table using KSQL
          • Create Streams and Tables
          • Write Queries
          • Monitor Streaming Data
        • Next Steps
      • Confluent Cloud Quick Start
    • Installing and Upgrading
      • Install Confluent Platform
        • System Requirements
          • Operating Systems
          • Software
            • Port Configuration
        • Installing Confluent Platform
          • Docker
          • ZIP and TAR archives
          • Debian and Ubuntu
          • RHEL and CentOS
        • Next Steps
      • Packages
        • Platform Packages
          • confluent-platform-<scala_version>
          • confluent-platform-oss-<scala_version>
        • Component Packages
          • avro
          • confluent-camus (deprecated)
          • confluent-control-center (Confluent Enterprise)
          • confluent-kafka-<scala_version>
          • confluent-kafka-connect-elasticsearch
          • confluent-kafka-connect-hdfs
          • confluent-kafka-connect-jdbc
          • confluent-kafka-connect-jms
          • confluent-kafka-connect-replicator (Confluent Enterprise)
          • confluent-kafka-connect-s3
          • confluent-kafka-dotnet
          • confluent-kafka-go
          • confluent-kafka-python
          • confluent-kafka-rest
          • confluent-ksql
          • confluent-libserdes
          • confluent-rebalancer (Confluent Enterprise)
          • confluent-schema-registry
          • confluent-support-metrics (Confluent Enterprise)
          • confluent-security
          • librdkafka
      • Installing using Docker
        • Docker Quick Start
          • Installing and Running Docker
          • Getting Started with Docker Compose
          • Getting Started with Docker Client
            • Docker Network
            • ZooKeeper
            • Kafka
            • Schema Registry
            • REST Proxy
            • Confluent Control Center
              • Stream Monitoring
              • Alerts
            • Kafka Connect
              • Getting Started
              • Monitoring in Control Center
            • Cleanup
        • Docker Configuration
          • Confluent Docker Images
          • Configuration Notes
          • Configuration Parameters
            • ZooKeeper
              • Required Settings
            • Confluent Kafka (cp-kafka)
              • Required Settings
            • Confluent Enterprise Kafka (cp-enterprise-kafka)
              • Required Settings
            • Schema Registry
              • Required Settings
            • Kafka REST Proxy
              • Required Settings
            • Kafka Connect
              • Required Settings
              • Optional Settings
            • Confluent Control Center
              • Docker Options
              • Required Settings
              • Optional Settings
            • Confluent Enterprise Replicator
              • Required Settings
              • Optional Settings
        • Docker Advanced Tutorials
          • Clustered Deployment
            • Installing and Running Docker
            • Docker Client: Setting Up a Three Node Kafka Cluster
            • Docker Compose: Setting Up a Three Node Kafka Cluster
          • Clustered Deployment Using SSL
            • Installing and Running Docker
            • Docker Client: Setting Up a Three Node Kafka Cluster
            • Docker Compose: Setting Up a Three Node Confluent Platform Cluster with SSL
          • Clustered Deployment Using SASL and SSL
            • Installing and Running Docker
            • Docker Client: Setting Up a Three Node Kafka Cluster
            • Docker Compose: Setting Up a Three Node Confluent Platform Cluster with SASL
          • Kafka Connect Tutorial
            • Installing and Running Docker
            • Starting Up Confluent Platform and Kafka Connect
          • Automatic Data Balancing
            • Installing and Running Docker
            • Docker Client: Setting Up a Three Node Kafka Cluster
          • Replicator Tutorial
            • Installing and Running Docker
            • Docker Client: Setting Up a Three Node Kafka Cluster
      • Deployment Startup Using Systemd
        • Usage
          • Enabling Automatic Startup During Boot
          • Starting Service Immediately
          • Checking Service Status
          • Stopping a Service
          • Disabling a Service
        • Logging
          • Following the Log
        • Modifying Configuration
          • Modifying Service Unit Configuration Settings
          • Modifying the Systemd Component Configuration
        • Additional Resources
      • Clients
        • Maven repository for JARs
        • C/C++
        • Python
        • Go
        • .NET
      • Migrating from Open Source to Enterprise
        • Migrating from Confluent Open Source
          • DEB Packages via apt or RPM Packages via yum
          • TAR or ZIP archives
        • Migrating from Apache Kafka
      • Upgrade
        • Preparation
        • Step-by-step Guide
          • Upgrade All Kafka Brokers
          • Upgrade Schema Registry
          • Upgrade Kafka REST Proxy
          • Upgrade Kafka Streams applications
          • Upgrade Kafka Connect
            • Upgrade Kafka Connect Standalone Mode
            • Upgrade Kafka Connect Distributed Mode
          • Upgrade Camus
          • Confluent Control Center
          • Upgrade Other Client Applications
      • Supported Versions and Interoperability
        • Confluent Platform Versions
        • Confluent Platform and Apache Kafka Compatibility
        • Operating Systems
        • Confluent Control Center
        • Java
        • KSQL
        • Scala
        • ZooKeeper
        • Component Security
          • Apache Kafka
          • Apache Kafka Connect clients
          • Apache Kafka Streams clients
          • Apache ZooKeeper
          • Confluent Control Center
          • REST Proxy
          • Schema Registry
        • Clients
        • Connectors
        • Cross-Component Compatibility
          • Apache Kafka Connect Workers
          • Apache Kafka Java Clients
          • Apache Kafka Streams Clients
          • Auto Data Balancing
          • Confluent C, C++, Python, Go and .NET
          • Schema Registry
          • REST Proxy
        • Docker and Orchestration Tools
          • DC/OS
          • Docker
    • Command Line Interface
      • Installing and Configuring the CLI
        • Installing the CLI
        • Configuring the CLI
      • Command Reference
        • confluent acl
          • Description
          • Usage
          • Options
          • Positional arguments
        • confluent config
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent current
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
        • confluent destroy
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
        • confluent help
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
        • confluent list
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent load
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
        • confluent log
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent start
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent status
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent stop
          • Description
          • Usage
          • Options
          • Positional arguments
          • Examples
        • confluent top
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
        • confluent unload
          • Description
          • Usage
          • Options
          • Positional arguments
          • Example
    • Tutorials and Resources
      • Monitoring Kafka streaming ETL deployments
        • Overview
        • Run demo
        • Playbook
          • Confluent Control Center
          • KSQL
          • Consumer rebalances
          • Slow consumers
          • Over consumption
          • Under consumption
          • Failed broker
          • Alerting
          • Replicator
          • Security
            • Encryption & Authentication
            • Authorization
          • Schema Registry and REST Proxy
        • Troubleshooting the demo
        • Teardown
      • Demos
      • Technical Blogs
      • Video Tutorials and Screencasts
      • White Papers
      • KSQL
    • Confluent Platform 4.1.0 Release Notes
      • Highlights
        • Enterprise Features
          • Control Center
          • Replicator
          • JMS Connectors
        • Open Source Features
          • Scalability
          • Reduced Producer Latency
          • Availability and Resilience
          • Operational Improvements
          • KSQL
          • Kafka Connect
          • Elasticsearch Connector
          • HDFS Connector
          • S3 Connector
          • JDBC Source Connector
          • JDBC Sink Connector
          • Confluent clients
            • Confluent CLI
          • Kafka Streams: New Test Utility Artifact
          • Kafka Streams: Embedded Admin Client Configuration
          • Apache Kafka 1.1.0-cp1
            • Data Balancing with JBOD
            • More Bug Fixes and Improvements
      • Previous releases
        • Confluent Platform 4.0.1 Release Notes
          • Highlights
            • JMS Client
              • Open Source Features
            • Apache Kafka 1.0.1-cp1
        • Confluent Platform 4.0.0 Release Notes
          • Highlights
            • Control Center: Backward Compatibility
            • Control Center: Improvements
              • Open Source Features
            • JBOD Disk Failure Handling
            • Metrics Improvements
            • Kafka Connect
            • Elasticsearch Connector
            • HDFS Connector
            • S3 Connector
            • JDBC Source Connector
            • Confluent CLI
            • Kafka Streams: Extended Punctuation Semantics
            • Kafka Streams: Deserialization Error Handling
            • Kafka Streams: State Restoration Listener
              • Apache Kafka 1.0.0-cp1
            • Security Failure Handling
            • Java 9 Support
            • Idempotent Producer Performance
            • More Bug Fixes and Improvements
        • Confluent Platform 3.3.1 Release Notes
          • Highlights
            • Control Center
              • Open Source Features
            • Elasticsearch Sink Connector
            • HDFS Sink Connector
            • S3 Sink Connector
            • JDBC Source Connector
            • librdkafka 0.11.1
            • Kafka Streams: Improved Operability due to Rebalancing Optimizations
            • Apache Kafka 0.11.0.1-cp1
        • Confluent Platform 3.3.0 Release Notes
          • Highlights
            • REST Proxy ACL Plug-In
            • Control Center Interceptors for Non-Java Clients
            • Control Center
            • Metrics Publisher
              • Open Source Features
            • Exactly-Once Semantics for Apache Kafka
            • Confluent CLI
            • Non-Java Clients
            • Kafka Streams: Confluent serializers/deserializers (serdes) for Avro
            • Kafka Streams: Capacity Planning Guide available
              • Apache Kafka 0.11.0
            • Admin client
            • Record headers
            • Leader epoch in replication protocol
            • Single-threaded controller
              • Classpath Isolation for Apache Kafka Connectors
              • Request rate quotas
              • Kafka Streams: rebalancing improvements
              • More bug fixes and improvements
        • Confluent Platform 3.2.2 Release Notes
          • Highlights
        • Confluent Platform 3.2.1 Release Notes
          • Highlights
            • Fixes
              • S3 Connector
            • Fixes
        • Confluent Platform 3.2.0 Release Notes
          • Highlights
            • JMS Client
            • Confluent Control Center
            • Automatic Data Balancing
              • Open Source Features
            • Kafka Streams: Backwards Compatible to Older Kafka Clusters
            • Kafka Streams: Compatibility Matrix
            • Kafka Streams: Session Windows
            • Kafka Streams: Global KTables
            • Kafka Streams: ZooKeeper Dependency Removed
            • Kafka Connect: Single Message Transform
            • S3 Connector
            • .Net Client
            • REST Proxy
            • Apache Kafka 0.10.2
        • Confluent Platform 3.1.2 Release Notes
          • Highlights
            • Fixes
        • Confluent Platform 3.1.1 Release Notes
          • Highlights
            • Automatic Data Balancing
            • Multi-Datacenter Replication
            • Confluent Control Center
              • Open Source Features
            • Kafka Streams: Interactive Queries
            • Kafka Streams: Application Reset Tool
            • Kafka Streams: Improved memory management
            • Kafka Connect
            • Go Client
              • Apache Kafka 0.10.1
            • New Feature
            • Improvement
            • Bug
            • Task
            • Wish
            • Test
            • Sub-task
        • Confluent Platform 3.1.0 Release Notes
          • Highlights
            • Automatic Data Balancing
            • Multi-Datacenter Replication
            • Confluent Control Center
              • Open Source Features
            • Kafka Streams
            • Kafka Connect
            • Go Client
          • Apache Kafka 0.10.1
        • Confluent Platform 3.0.1 Release Notes
          • Highlights
            • New Feature: Application Reset Tool
              • Apache Kafka 0.10.0.1
            • New Feature
            • Improvement
            • Bug
            • Test
            • Sub-task
        • Confluent Platform 3.0.0 Release Notes
          • Highlights
        • Confluent Platform 2.0.1 Release Notes
          • New Java consumer
          • Compatibility
          • Security
          • Performance/memory usage
          • Topic deletion
        • Confluent Platform 2.0.0 Release Notes
          • Security
          • Kafka Connect
          • User Defined Quotas
          • New Consumer
          • New Client Library - librdkafka
          • Proactive Support
          • Compatibility Notes
      • How to Download
      • Supported Versions and Interoperability
      • Questions?
  • Operations
    • Kafka Operations
      • Production Deployment
        • Hardware
          • Memory
          • CPUs
          • Disks
          • Network
          • Filesystem
          • General considerations
        • JVM
        • Important Configuration Options
          • Replication configs
        • File descriptors and mmap
        • Multi-node Configuration
      • Post Deployment
        • Changing configs dynamically
          • cleanup.policy
          • flush.messages
          • flush.ms
          • max.message.bytes
          • min.insync.replicas
          • retention.bytes
          • retention.ms
          • segment.bytes
          • segment.jitter.ms
          • segment.ms
          • unclean.leader.election.enable
        • Logging
        • Admin operations
        • Rolling Restart
        • Performance Tips
        • Software Updates
        • Backup and Restoration
      • Auto Data Balancing
        • Requirements
        • Quickstart
          • Start a Kafka cluster
          • Create topics and produce data
          • Execute the rebalancer
          • Check status and finish
          • Leader balance
          • Decommissioning brokers
          • Limiting Bandwidth Usage during Data Migration
        • Licensing
        • Configuration Options
          • Configuration Options for the rebalancer tool
      • Monitoring Kafka
        • Server Metrics
          • Broker Metrics
          • ZooKeeper Metrics
        • Producer Metrics
          • Global Request Metrics
          • Global Connection Metrics
          • Per-Broker Metrics
          • Per-Topic Metrics
        • New Consumer Metrics
          • Fetch Metrics
          • Topic-level Fetch Metrics
          • Consumer Group Metrics
          • Global Request Metrics
          • Global Connection Metrics
          • Per-Broker Metrics
        • Old Consumer Metrics
      • Confluent Metrics Reporter
        • Installation
        • Configuration
          • Message size
        • Configuration Options
          • Security
          • Authentication
          • Authorization
        • Verification
    • Confluent Control Center
      • Control Center Quick Start
      • Installing and Configuring Control Center
        • Installing Control Center
          • System Requirements
          • Partitions and Replication
          • Multi-Cluster Configuration
          • Dedicated Metric Data Cluster
          • Saturation Testing
            • Test Setup
            • Broker Monitoring
            • Streams Monitoring
          • Example Deployments
            • Broker Monitoring
            • Streams Monitoring
            • Next steps
        • Control Center System Requirements
          • Hardware
            • Memory
            • CPUs
            • Disks
            • Network
          • OS
          • JVM
          • Kafka
        • Control Center Parameter Reference
          • Base Settings
          • Logging
          • Optional Settings
            • General
            • Monitoring Settings
            • UI Authentication Settings
            • Email Settings
            • Kafka Encryption, Authentication, Authorization Settings
            • HTTPS Settings
            • Internal Kafka Streams Settings
            • Internal Command Settings
        • Confluent Monitoring Interceptors
          • Installing Interceptors
            • Java Clients
            • librdkafka-based Clients
          • Enabling Interceptors
            • Java Producers and Consumers
            • Kafka Streams
            • KSQL
            • librdkafka-based clients
            • Kafka Connect
            • Confluent Replicator
          • Configuring Interceptors
            • Defaults
            • General Options
            • Security
            • Logging
            • Example Configuration
          • Verification
          • Caveats
        • Installing Control Center on Apache Kafka
          • Software requirements
          • Hardware requirements
            • Install Control Center
          • Installation on Red Hat, CentOS, or Fedora
          • Installation on Debian Linux
            • Install and configure components
          • Installing Confluent Metrics Reporter with Kafka
          • Installing Kafka Connect
          • Installing Confluent Metrics Clients with Kafka Connect
            • Start Control Center
            • License Keys
            • Removing Control Center
          • TAR and ZIP Archives
          • Debian packages
          • Red Hat packages
        • Troubleshooting Control Center
          • Common issues
            • Installing and Setup
              • Bad security configuration
              • InvalidStateStoreException
              • Not enough brokers
              • Local store permissions
              • Multiple Control Center’s with the same ID
              • License expired
            • System health
              • Web interface that is blank or stuck loading
              • I see a rocket ship
              • Nothing is produced on the Metrics (_confluent-metrics) topic
              • Control Center is lagging behind Kafka
              • RecordTooLargeException
              • Parts of the broker or topic table have blank values
            • Streams Monitoring
              • Blank charts
              • Unexpected herringbone pattern
              • Missing consumers or consumer groups
            • Connect
              • I see rocket ship
          • Debugging
            • Check logs
            • Enable debug and trace logging
            • Check configurations
            • Review input topics
            • Size of clusters
            • System check
            • Frontend and REST API
            • Consumer offset lag
            • Enable GC logging
            • Thread dump
            • Data directory
        • Upgrading Control Center
          • Upgrading from version 3.1.x and later
          • Upgrading from version 3.0.x
      • Control Center User Interface
        • System Health
          • Navigation
          • UI Commonalities
          • Broker Aggregate Metrics
          • Produce and Fetch Charts
          • Broker Metrics Table
          • Topic Aggregate Metrics
          • Topic Metrics Table
        • Stream Monitoring
          • Chart Types
          • Page Layout
          • Navigation
          • Time Range and Cluster Selection
          • Summary statistics
          • Missing Metrics Data
          • Example Scenarios
            • Adding a new Consumer Group
              • Latency
              • Expected consumption
        • Kafka Connect
          • Creating new Connectors
          • Editing Sources and Sinks
        • Alerts
          • Concepts
          • User Interface
            • Overview page
            • Integration page
          • Trigger Management
            • New/Edit Trigger Form
            • Topic
            • Consumer Groups
            • Clusters
            • Brokers
          • Actions Management
            • New/Edit Action Form
          • Alert History
            • REST API
          • Control Center’s Status
            • Create alerts for Control Center Status
        • Clusters
          • Setting a Cluster Name
        • Topic Management
          • Create a topic
          • Detail view for a topic
          • Edit a topic
      • Security
        • Configuring SSL
          • Kafka Brokers
          • Control Center
          • Connect
        • Configuring SASL
          • ZooKeeper
          • Kafka Broker
          • Control Center Configuration
          • Schema Registry Configuration
          • Connect Configuration
        • UI Authentication
        • UI HTTPS
        • Authorization with Kafka ACLS
      • Changelog
        • Version 4.1.0
          • Confluent Control Center
        • Version 4.0.1
          • Confluent Control Center
        • Version 4.0.0
          • Confluent Control Center
        • Version 3.3.1
          • Confluent Control Center
        • Version 3.3.0
          • Confluent Control Center
        • Version 3.2.2
          • Confluent Control Center
        • Version 3.2.1
          • Confluent Control Center
        • Version 3.2.0
          • Confluent Control Center
        • Version 3.1.0
          • Confluent Control Center
        • Version 3.0.1
          • Confluent Control Center
    • Multi Data-Center Deployment
      • Replicator Quick Start
        • Start the destination cluster
        • Start the origin cluster
        • Create a topic
        • Configure and run Replicator
      • Installing and Configuring Replicator
        • Install and Configure Kafka Connect Cluster for Replicator
          • Configuring origin and destination brokers
          • Where to Install Connect Workers
          • Running Replicator on Existing Connect Cluster
          • Configuring Logging for Connect Cluster
          • License Key
        • Configure and run a Confluent Replicator on the Connect Cluster
      • Tuning and Monitoring Replicator
        • Sizing Replicator Cluster
        • Getting More Throughput From Replicator Tasks
          • Improving CPU Utilization of a Connect Task
          • Improving Network Utilization of a Connect Task
        • Monitoring Replicator
          • Monitoring Replicator Lag
          • Monitoring Producer and Consumer Metrics
            • Important Producer Metrics
            • Important Consumer Metrics
      • Installing and Running Replicator Executable
        • Configure producers and consumers with Replicator Executable
        • Configure replication and run Replicator Executable
        • Configuring Logging with Replicator Executable
        • License Key
        • Command Line Parameters of Replicator Executable
      • Apache Kafka’s MirrorMaker
        • Comparing Mirror Maker to Confluent Replicator
          • Greater control for reliable replication
          • Deploying, managing, and monitoring
          • Replication Patterns
        • Example of MirrorMaker Use
    • Schema Registry Operations
      • Production Deployment
        • Hardware
        • Memory
        • CPUs
        • Disks
        • Network
        • JVM
        • Important Configuration Options
        • listeners
          • Don’t Modify These Storage Settings
        • Kafka & ZooKeeper
        • Migration from ZooKeeper master election to Kafka master election
        • Downtime for Writes
        • Complete downtime
        • Backup and Restore
      • Monitoring
        • Global Metrics
        • Per-Endpoint Metrics
        • Endpoints
    • Kafka REST Proxy Operations
      • Production Deployment
        • Hardware
          • Memory
          • CPUs
          • Disks
          • Network
        • JVM
        • Deployment
        • Important Configuration Options
        • Don’t Touch These Settings!
        • Post Deployment
      • Monitoring
        • Global Metrics
        • Per-Endpoint Metrics
        • Endpoints
    • Docker Operations
      • Monitoring
        • Using JMX
          • Security on JMX
          • Kafka and ZooKeeper
            • Settings
            • Launching Kafka and ZooKeeper with JMX Enabled
      • Configuring logging
        • log4j Log Levels
        • Component Names
      • Mounting External Volumes
        • Data Volumes for Kafka and ZooKeeper
        • Security: Data Volumes for Configuring Secrets
        • Configuring Connect with External jars
    • Security
      • Security Tutorial
        • Overview
          • Prerequisites
        • Creating SSL Keys and Certificates
          • Generate the keys and certificates
          • Create your own Certificate Authority (CA)
          • Sign the certificate
          • Summary
        • ZooKeeper
        • Brokers
        • Clients
          • Common Configuration
          • Console Producer and Consumer
          • KSQL and Stream Processing Clients
        • Kafka Connect
        • Replicator
        • Confluent Control Center
        • Confluent Metrics Reporter
        • Confluent Monitoring Interceptors
        • Authorization and ACLs
        • Troubleshooting
        • Next Steps
      • Encryption with SSL
        • SSL Overview
        • Creating SSL Keys and Certificates
        • Brokers
          • Optional settings
        • Clients
          • Optional settings
        • ZooKeeper
        • Kafka Connect
        • Confluent Replicator
        • Confluent Control Center
        • Confluent Metrics Reporter
        • Confluent Monitoring Interceptors
          • Interceptors for General Clients
          • Interceptors for Kafka Connect
          • Interceptors for Replicator
        • Schema Registry
        • REST Proxy
        • SSL Logging
      • Encryption and Authentication with SSL
        • SSL Overview
        • Creating SSL Keys and Certificates
        • Brokers
          • Optional settings
        • Clients
          • Optional settings
        • ZooKeeper
        • Kafka Connect
        • Confluent Replicator
        • Confluent Control Center
        • Confluent Metrics Reporter
        • Confluent Monitoring Interceptors
          • Interceptors for General Clients
          • Interceptors for Kafka Connect
          • Interceptors for Replicator
        • Schema Registry
        • REST Proxy
        • SSL Logging
      • Authentication with SASL
        • SASL Overview
        • GSSAPI
          • Configuring GSSAPI
            • SASL/GSSAPI Overview
              • GSSAPI Logging
            • Brokers
              • JAAS
              • Configuration
              • Run
            • Clients
            • ZooKeeper
              • JAAS
              • Configuration
              • Run
            • Kafka Connect
            • Confluent Replicator
            • Confluent Control Center
            • Confluent Metrics Reporter
            • Confluent Monitoring Interceptors
              • Interceptors for General Clients
              • Interceptors for Kafka Connect
              • Interceptors for Replicator
            • Schema Registry
            • REST Proxy
        • SCRAM
          • Configuring SCRAM
            • SASL/SCRAM Overview
            • Brokers
              • JAAS
              • Configuration
              • Run
            • Clients
            • ZooKeeper
            • Kafka Connect
            • Confluent Replicator
            • Confluent Control Center
            • Confluent Metrics Reporter
            • Confluent Monitoring Interceptors
              • Interceptors for General Clients
              • Interceptors for Kafka Connect
              • Interceptors for Replicator
            • Schema Registry
            • REST Proxy
        • PLAIN
          • Configuring PLAIN
            • SASL/PLAIN Overview
            • Brokers
              • JAAS
              • Configuration
              • Run
            • Clients
            • ZooKeeper
            • Kafka Connect
            • Confluent Replicator
            • Confluent Control Center
            • Confluent Metrics Reporter
            • Confluent Monitoring Interceptors
              • Interceptors for General Clients
              • Interceptors for Kafka Connect
              • Interceptors for Replicator
            • Schema Registry
            • REST Proxy
        • JAAS configurations
          • JAAS and Confluent Control Center
        • Enabling multiple SASL mechanisms
        • Modifying SASL mechanisms in a Running Cluster
      • Authorization and ACLs
        • Overview
        • Broker Configuration
          • Authorizer
          • Super Users
          • User Names
        • Using ACLs
          • ACL Format
            • Common cases:
          • Adding ACLs
          • Removing ACLs
          • Listing ACLs
          • Adding or Removing a Principal as Producer or Consumer
        • Authorization in the REST Proxy and Schema Registry
        • Debugging
      • Adding Security to a Running Cluster
        • Migrating Brokers and Clients
        • Migrating ZooKeeper
        • Migrating the ZooKeeper ensemble
      • Streams Security
        • Required ACL setting for secure Kafka clusters
        • Security example
      • Confluent Security Plugins
        • Installation
          • Kafka REST Security Plugin
            • Kafka REST Security Plugin Quick Start
              • Installation
              • Authentication Mechanisms
              • Configuration
            • Principal Propagation
              • SSL
              • SASL
          • Schema Registry Security Plugin
            • Schema Registry Security Plugin Quick Start
              • Installation
              • Authentication Mechanisms
              • Configuration
            • Schema Registry Authorization
              • Schema Registry ACL Authorizer
                • Schema Registry ACL CLI
                  • Adding ACLs
                  • Removing ACLs
                  • List ACLs
              • Topic ACL Authorizer
      • Docker Security
      • Prefixes for Configuring Security
        • Configuration Parameters
        • Environment Variables for Configuring HTTPS
        • Addtional Environment Variables
    • Kafka Connect
      • Kafka Connect Quick Start
        • Goal
        • What we will do
        • Start the services
        • Read File Data with Connect
        • Write File Data with Connect
      • Kafka Connect Concepts
        • Connectors
        • Tasks
          • Task Rebalancing
        • Workers
          • Standalone Workers
          • Distributed Workers
        • Converters
        • Transforms
      • Installing and Configuring Kafka Connect
        • Getting Started
        • Planning for Installation
          • Prerequisites
          • Standalone vs. Distributed
          • Deployment Considerations
        • Installing Plugins
        • Running Workers
          • Standalone Mode
          • Distributed Mode
        • Configuring Workers
          • Common Worker Configs
          • Standalone Worker Configuration
          • Distributed Worker Configuration
          • Configuring Converters
          • Overriding Producer & Consumer Settings
        • Upgrading Kafka Connect Workers
      • Managing Connectors
        • Using Bundled Connectors
        • Configuring Connectors
          • Standalone Example
          • Distributed Example
        • Managing Running Connectors
          • Using the REST Interface
          • Connector and Task Status
          • Common REST Examples
        • Using Community Connectors
        • Upgrading a Connector Plugin
      • Bundled Connectors
        • Confluent JDBC Connector
          • JDBC Source Connector
            • Quick Start
              • Create SQLite Database and Load Data
              • Load the JDBC Source Connector
              • Add a Record to the Consumer
            • Features
            • Configuration
            • Schema Evolution
          • JDBC Source Configuration Options
            • Database Connection Security
            • Database
            • Mode
            • Connector
          • JDBC Sink Connector
            • Quick Start
              • Create SQLite Database and Load Data
              • Load the JDBC Sink Connector
              • Produce a Record in SQLite
            • Features
          • JDBC Sink Configuration Options
            • Database Connection Security
            • Connection
            • Writes
            • Data Mapping
            • DDL Support
            • Retries
          • Changelog
            • Version 4.1.0
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 4.0.1
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 4.0.0
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 3.3.1
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 3.3.0
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 3.2.2
            • Version 3.2.1
            • Version 3.2.0
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 3.1.1
            • Version 3.1.0
              • JDBC Source Connector
              • JDBC Sink Connector
            • Version 3.0.1
              • JDBC Source Connector
            • Version 3.0.0
              • JDBC Source Connector
        • Confluent HDFS Connector
          • HDFS Connector
            • Quickstart
            • Features
            • Configuration
              • Example
              • Format and Partitioner
              • Hive Integration
              • Secure HDFS and Hive Metastore
            • Schema Evolution
          • HDFS Connector Configuration Options
            • HDFS
            • Security
            • Connector
            • Storage
            • Partitioner
            • Hive
            • Schema
          • Changelog
            • Version 4.1.0
            • Version 4.0.1
            • Version 4.0.0
            • Version 3.3.1
            • Version 3.3.0
            • Version 3.2.2
            • Version 3.2.1
            • Version 3.2.0
            • Version 3.1.1
            • Version 3.1.0
            • Version 3.0.1
              • HDFS Connector
            • Version 3.0.0
              • HDFS Connector
        • Confluent S3 Connector
          • S3 Connector
            • Features
            • Exactly-once delivery on top of eventual consistency
            • Schema Evolution
            • Quickstart
            • Configuration
              • Basic Example
              • Write raw message values into S3
          • S3 Connector Configuration Options
            • Connector
            • S3
            • Storage
            • Partitioner
          • Changelog
            • Version 4.1.0
              • S3 Connector
              • S3 Connector
            • Version 4.0.0
              • S3 Connector
            • Version 3.3.1
              • S3 Connector
            • Version 3.3.0
              • S3 Connector
            • Version 3.2.2
              • S3 Connector
            • Version 3.2.1
              • S3 Connector
            • Version 3.2.0
              • S3 Connector
        • Confluent Elasticsearch Connector
          • Elasticsearch Connector
            • Quick Start
              • Add a Record to the Consumer
              • Load the Elasticsearch Connector
            • Features
            • Delivery Semantics
            • Mapping Management
            • Schema Evolution
            • Automatic Retries
            • Reindexing
            • Security
          • Configuration Options
            • Connector
            • Data Conversion
          • Changelog
            • Version 4.1.0
            • Version 4.0.1
            • Version 4.0.0
            • Version 3.3.1
            • Version 3.3.0
            • Version 3.2.2
            • Version 3.2.1
            • Version 3.2.0
        • Confluent Replicator
          • Confluent Replicator
            • Features
            • Requirements
            • Quick Start
              • Start the destination cluster
              • Start the source cluster
              • Configure and Run Replicator as an Executable
            • Configure and Run Replicator as a Connector (Advanced)
            • Test Your Replicator
            • Topic Renaming
            • Periodic Metadata Updates
            • Security
          • Changelog
            • Version 4.1.0
            • Version 4.0.1
            • Version 4.0.0
            • Version 3.3.1
            • Version 3.3.0
            • Version 3.2.2
            • Version 3.2.1
            • Version 3.2.0
          • Configuration Options
            • Confluent Platform
            • Source Topics
            • Source Data Conversion
            • Source Zookeeper
            • Source Kafka
            • Source Kafka: Security
            • Source Kafka: Consumer
            • Destination Topics
            • Destination Zookeeper
            • Destination Kafka
            • Destination Kafka: Security
            • Destination Data Conversion
              • Confluent Licensing
        • Kafka FileStream Connectors
          • Quickstart
          • FileSource Connector
          • FileSink Connector
      • Kafka Connect Transformations
        • Cast
          • Description
          • Properties
        • ExtractField
          • Description
          • Properties
        • Flatten
          • Description
          • Properties
        • HoistField
          • Description
          • Properties
        • InsertField
          • Description
          • Properties
        • MaskField
          • Description
          • Properties
        • RegexRouter
          • Description
          • Properties
        • ReplaceField
          • Description
          • Properties
        • SetSchemaMetadata
          • Description
          • Properties
        • TimestampConverter
          • Description
          • Properties
        • TimestampRouter
          • Description
          • Properties
        • ValueToKey
          • Description
          • Properties
      • Kafka Connect Security
        • Encryption
        • Authentication
        • Separate principals
        • ACL Considerations
          • Worker ACL Requirements
          • Connector ACL Requirements
        • Configuring the Connect REST API for HTTP or HTTPS
      • Kafka Connect Architecture
        • Motivation
        • Architecture
          • Internal Connect Offsets
      • Connector Developer Guide
        • Core Concepts and APIs
          • Connectors and Tasks
          • Partitions and Records
          • Dynamic Connectors
        • Developing a Simple Connector
          • Connector Example
          • Task Example - Source Task
          • Sink Tasks
          • Resuming from Previous Offsets
        • Dynamic Input/Output Partitions
        • Configuration Validation
        • Working with Schemas
        • Schema Evolution
        • Testing
        • Packaging
          • Creating an Archive
          • Creating an Uber JAR
      • Kafka Connect FAQ
        • How do I change the output data format of a SinkConnector?
        • Why does a connector configuration update trigger a task rebalance?
        • Why should I use distributed mode instead of standalone?
        • Do I need to write custom code to use Kafka Connect?
        • Is the Schema Registry a required service to run Kafka Connect?
        • How can I use plain JSON data with Connect?
        • Does source connector X support output format Y?
        • Why is CPU usage high for my Connect worker when no connectors have been deployed?
        • Can connect sink connectors read data written by other clients, e.g. a custom client?
        • After testing a connector in standalone mode, restarting it doesn’t write the data again?
        • Can I use a newer version of Connect with older brokers?
      • Kafka Connect Reference
        • Kafka Connect Javadocs
        • Kafka Connect REST Interface
          • Content Types
          • Statuses & Errors
          • Connectors
          • Tasks
          • Connector Plugins
        • Kafka Connect Worker Configs
          • Common Worker Configs
          • Standalone Worker Configuration
          • Distributed Worker Configuration
    • Camus Operations
      • Important Configuration Options
    • ZooKeeper Operations
      • ZooKeeper Production Deployment
        • Stable version
        • Hardware
          • Memory
          • CPU
          • Disks
        • JVM
        • Important Configuration Options
        • Monitoring
          • Operating System
          • “Four Letter Words”
          • JMX Monitoring
        • Multi-node Setup
      • Post Deployment
  • Development
    • Kafka Streams
      • Introduction
        • The Kafka Streams API in a Nutshell
        • Use Case Examples
        • A Closer Look
      • Requirements
        • Apache Kafka
        • Confluent
      • Kafka Streams Quick Start
        • Purpose
        • Start the Kafka cluster
        • Prepare the topics and the input data
        • Process the input data with Kafka Streams
        • Inspect the output data
        • Stop the Kafka cluster
        • Next steps
      • Kafka Streams Demo Application
        • Requirements
        • Running the Kafka Music demo application
        • Running further Confluent demo applications for the Kafka Streams API
        • Appendix
          • Inspecting the input topics of the Kafka Music application
          • Creating new topics
          • Listing available topics
      • Concepts
        • Kafka 101
        • Stream
        • Stream Processing Application
        • Processor Topology
        • Stream Processor
        • Stateful Stream Processing
        • Duality of Streams and Tables
        • KStream
        • KTable
        • GlobalKTable
        • Time
        • Aggregations
        • Joins
        • Windowing
        • Interactive Queries
        • Processing Guarantees
      • Architecture
        • Processor Topology
        • Parallelism Model
          • Stream Partitions and Tasks
          • Threading Model
          • Example
        • State
        • Memory management
          • Record caches
        • Fault Tolerance
        • Flow Control with Timestamps
        • Backpressure
      • Code Examples
        • Getting started examples
          • Java
          • Scala
        • Security examples
          • Java programming language
        • Interactive queries examples
          • Java
        • End-to-end application examples
          • Java
          • Scala
        • Event-Driven Microservice example
          • Java
      • Developer Guide
        • Writing a Streams Application
          • Libraries and Maven artifacts
          • Using Kafka Streams within your application code
        • Testing Streams Code
          • Importing the Test Utilities
          • Testing a Streams Application
            • Example
        • Configuring a Streams Application
          • Configuration parameter reference
            • Required configuration parameters
              • application.id
              • bootstrap.servers
            • Optional configuration parameters
              • default.deserialization.exception.handler
              • default.production.exception.handler
              • default.key.serde
              • default.value.serde
              • default.timestamp.extractor
              • num.standby.replicas
              • num.stream.threads
              • partition.grouper
              • processing.guarantee
              • replication.factor
              • state.dir
            • Kafka consumers, producer, and admin client configuration parameters
              • Naming
              • Default Values
              • enable.auto.commit
              • rocksdb.config.setter
            • Recommended configuration parameters for resiliency
              • acks
              • replication.factor
        • Streams DSL
          • Overview
          • Creating source streams from Kafka
          • Transform a stream
            • Stateless transformations
            • Stateful transformations
              • Aggregating
              • Joining
                • Join co-partitioning requirements
                • KStream-KStream Join
                • KTable-KTable Join
                • KStream-KTable Join
                • KStream-GlobalKTable Join
              • Windowing
                • Tumbling time windows
                • Hopping time windows
                • Sliding time windows
                • Session Windows
            • Applying processors and transformers (Processor API integration)
          • Writing streams back to Kafka
        • Processor API
          • Overview
          • Defining a Stream Processor
          • Accessing Processor Context
          • State Stores
            • Defining and creating a State Store
            • Fault-tolerant State Stores
            • Enable or Disable Fault Tolerance of State Stores (Store Changelogs)
            • Implementing Custom State Stores
          • Connecting Processors and State Stores
          • Describing a Topology
        • Data Types and Serialization
          • Configuring SerDes
          • Overriding default SerDes
          • Available SerDes
            • Primitive and basic types
            • Avro
            • JSON
            • Further serdes
          • Implementing custom SerDes
        • Interactive Queries
          • Querying local state stores for an app instance
            • Querying local key-value stores
            • Querying local window stores
            • Querying local custom state stores
          • Querying remote state stores for the entire app
            • Adding an RPC layer to your application
            • Exposing the RPC endpoints of your application
            • Discovering and accessing application instances and their local state stores
          • Demo applications
        • Memory Management
          • Record caches in the DSL
          • Record caches in the Processor API
          • Other memory usage
        • Running Streams Applications
          • Starting a Kafka Streams application
          • Elastic scaling of your application
            • Adding capacity to your application
            • Removing capacity from your application
            • State restoration during workload rebalance
            • Determining how many application instances to run
        • Managing Streams Application Topics
          • User topics
          • Internal topics
        • Streams Security
          • Required ACL setting for secure Kafka clusters
          • Security example
        • Application Reset Tool
          • Step 1: Run the application reset tool
          • Step 2: Reset the local environments of your application instances
            • Example
      • Operations
        • Capacity planning and sizing
          • Background and context
          • Stateless vs. stateful
            • Stateless applications
            • Stateful applications
          • Examples
          • Troubleshooting
        • Monitoring your application
          • Metrics
            • Accessing Metrics
              • Accessing Metrics via JMX and Reporters
              • Accessing Metrics Programmatically
            • Configuring Metrics Granularity
            • Built-in Metrics
              • Thread Metrics
              • Task Metrics
              • Processor Node Metrics
              • State Store Metrics
              • Record Cache Metrics
            • Adding Your Own Metrics
          • Run-time Status Information
            • Status of KafkaStreams instances
            • Monitoring the Restoration Progress of Fault-tolerant State Stores
          • Integration with Confluent Control Center
      • Upgrade Guide
        • Upgrading from CP 4.0.x (Kafka 1.0.x-cp1) to CP 4.1.0 (Kafka 1.1.0-cp1)
          • Compatibility
          • Upgrading your Kafka Streams applications to CP 4.1.0
          • API changes
            • Embedded Admin Client Configuration
            • New Functions in Window Store Interface
          • Full upgrade workflow
        • Upgrading older Kafka Streams applications to CP 4.1.0
          • API changes (from CP 3.3 to CP 4.0)
            • Building and running a topology
            • Describing topology and stream task metadata
            • Merging KStreams:
            • Punctuation functions
            • Streams Configuration
            • New classes for optional DSL parameters
          • API changes (from CP 3.2 to CP 3.3)
            • Streams Configuration
            • Local timestamp extractors
            • KTable Changes
          • API changes (from CP 3.1 to CP 3.2)
            • Handling Negative Timestamps and Timestamp Extractor Interface
            • Metrics
            • Scala
          • API changes (from CP 3.0 to CP 3.1)
            • Stream grouping and aggregation
            • Auto Repartitioning
            • TopologyBuilder
            • DSL: New parameters to specify state store names
            • Windowing
      • FAQ
        • General
          • Is Kafka Streams a project separate from Apache Kafka?
          • Is Kafka Streams a proprietary library of Confluent?
          • Do Kafka Streams applications run inside the Kafka brokers?
          • Why Does My Kafka Streams Application Use So Much Memory?
          • What are the system dependencies of Kafka Streams?
          • How do I migrate my older Kafka Streams applications to the latest Confluent Platform version?
          • Which versions of Kafka clusters are supported by Kafka Streams?
          • What programming languages are supported?
          • Why is my application re-processing data from the beginning?
        • Scalability
          • Maximum parallelism of my application? Maximum number of app instances I can run?
        • Processing
          • Accessing record metadata such as topic, partition, and offset information?
          • Difference between map, peek, foreach in the DSL?
          • How to avoid data repartitioning if you know it’s not required?
          • Serdes config method
          • How can I replace RocksDB with a different store?
        • Failure and exception handling
          • Handling corrupted records and deserialization errors (“poison pill records”)?
            • Option 1: Log the error and shut down the application
            • Option 2: Skip corrupted records and log the error
            • Option 3: Quarantine corrupted records (dead letter queue)
            • Option 4: Interpret corrupted records as a sentinel value
          • Sending corrupt records to a quarantine topic or dead letter queue?
        • Interactive Queries
          • Handling InvalidStateStoreException: “the state store may have migrated to another instance”?
        • Security
          • Application fails when running against a secured Kafka cluster?
        • Troubleshooting and debugging
          • Easier to interpret Java stacktraces?
          • Visualizing topologies?
            • Older releases:
          • Inspecting streams and tables?
          • Invalid Timestamp Exception
          • Why do I get an IllegalStateException when accessing record metadata?
          • Why is punctuate() not called?
          • Scala: compile error “no type parameter”, “Java-defined trait is invariant in type T”
          • How can I convert a KStream to a KTable without an aggregation step?
            • Option 1: Write KStream to Kafka, read back as KTable
            • Option 2: Perform a dummy aggregation
          • RocksDB behavior in 1-core environments
      • Javadocs
    • KSQL
      • What Is KSQL?
      • What Can I Do With KSQL?
      • What Are the Components?
      • Supported Versions and Interoperability
      • KSQL Documentation
        • KSQL Quick Start
        • Installation and Configuration
          • Installing KSQL
            • Supported Versions and Interoperability
            • Installation Instructions
            • Starting the KSQL Server
            • Starting the KSQL CLI
            • Configuring KSQL for Confluent Cloud
          • Check the Health of a KSQL Server
            • Check a KSQL Server Running in a Native Deployment
            • Check a KSQL Server by using the REST API
          • Configuring KSQL Server
            • Configuring Security for KSQL
              • Configuring KSQL for Confluent Cloud
              • Configuring KSQL for Secured Confluent Schema Registry
              • Configuring KSQL for Secured Apache Kafka clusters
                • Configuring Kafka Encrypted Communication
                • Configuring Kafka Authentication
                • Configuring Authorization of KSQL with Kafka ACLs
                  • Interactive KSQL clusters
                  • Non-Interactive (headless) KSQL clusters
                • Configuring Control Center Monitoring Interceptors
            • KSQL Configuration Parameter Reference
              • Kafka Streams and Kafka Client Settings
                • ksql.streams.auto.offset.reset
                • ksql.streams.bootstrap.servers
                • ksql.streams.commit.interval.ms
                • ksql.streams.cache.max.bytes.buffering
                • ksql.streams.num.stream.threads
              • KSQL Query Settings
                • ksql.fail.on.deserialization.error
                • ksql.schema.registry.url
                • ksql.service.id
                • ksql.sink.partitions
                • ksql.sink.replicas
              • KSQL Server Settings
                • ksql.queries.file
                • listeners
              • Confluent Control Center Settings
              • Recommended KSQL Production Settings
            • Avro and Schema Registry
              • Supported functionality
              • Configuring KSQL for Avro
              • Using Avro in KSQL
                • Example KSQL Statements with Avro
                  • Create a New Stream by Reading Avro-formatted Data
                  • Create a New Table by Reading Avro-formatted Data
                  • Create a New Stream with Selected Fields of Avro-formatted Data
                  • Convert a JSON Stream to an Avro Stream
            • Setting KSQL Server Parameters
              • KSQL Server Configuration File
              • KSQL_OPTS Environment Variable
            • JMX Metrics
            • Non-interactive (Headless) KSQL Usage
          • Configuring KSQL CLI
            • Configuring Per-session Properties
          • Upgrading KSQL
            • Upgrading from KSQL 0.x (Developer Preview) to KSQL 4.1
        • KSQL Operations
          • Local Development and Testing with Confluent CLI
          • Starting and Stopping KSQL Clusters
          • Healthchecks
          • Monitoring and Metrics
          • Capacity Planning
          • Troubleshooting
            • SELECT query hangs and doesn’t stop?
            • No results from SELECT * FROM table or stream?
            • Can’t create a stream from the output of windowed aggregate?
            • KSQL doesn’t clean up its internal topics?
            • KSQL CLI doesn’t connect to KSQL server?
              • Check the port that KSQL CLI is using
              • Check the KSQL server configuration
              • Check for a port conflict
            • Replicated topic with Avro schema causes errors?
            • Check KSQL server logs
        • KSQL Capacity Planning
          • Approach To Sizing
          • General Guidelines
            • KSQL
            • Kafka
              • Internal Topics
              • Consumption and Production
          • Important Sizing Factors
          • Recommendations and Best Practices
            • Interactive KSQL Servers vs Non-Interactive (“Headless”) Servers
            • Scaling KSQL
              • How to Know When to Scale
              • Mixed Workloads
          • Examples
            • Scenario 1: Project/Filter Only (Stateless Queries)
              • KSQL
              • Kafka
            • Scenario 2: Large Messages
            • Scenario 3: More Advanced Usage
              • KSQL
              • Kafka
        • KSQL Tutorials and Examples
          • Writing Streaming Queries Against Apache Kafka® Using KSQL (Docker)
            • Download the Tutorial and Start KSQL
            • Create a Stream and Table
            • Write Queries
            • Terminate and Exit
              • KSQL
              • Appendix
                • Produce more topic data
                • Verify your environment
            • Next steps
          • Writing Streaming Queries Against Apache Kafka® Using KSQL (Local)
            • Create Topics and Produce Data
            • Launch the KSQL CLI
            • Create a Stream and Table
            • Write Queries
            • Terminate and Exit
              • KSQL
              • Confluent Platform
          • Clickstream Data Analysis Pipeline Using KSQL (Docker)
            • Download the Tutorial
            • Configure and Start Elastic, Grafana, and Confluent Platform
            • Create the Clickstream Data
            • Load the Streaming Data to KSQL
              • Verify the data
            • Load and View the Clickstream Data in Grafana
              • Troubleshooting
          • Generate Custom Test Data by Using the ksql-datagen tool
            • Usage
              • Required Arguments
              • Optional Arguments
            • Generate Records From a Predefined Schema
              • Generate Example Order Records
              • Generate Example User Records
              • Generate Example User Page Views
            • Generate Records From an Avro Schema
              • Define a Custom Schema
              • Generate Test Data
              • Consume the Test Data Stream
          • KSQL Examples
            • Creating streams
            • Creating tables
            • Working with streams and tables
              • Transforming
              • Joining
              • Aggregating, windowing, and sessionization
              • Working with arrays and maps
            • Running KSQL Statements From the Command Line
          • KSQL Basics
          • Clickstream Data Analysis Pipeline
          • KSQL Examples
          • KSQL in a Kafka Streaming ETL
        • KSQL Syntax Reference
          • Terminology
            • Stream
            • Table
            • Time Units
          • KSQL CLI Commands
            • RUN SCRIPT
          • KSQL statements
            • CREATE STREAM
            • CREATE TABLE
            • CREATE STREAM AS SELECT
            • CREATE TABLE AS SELECT
            • DESCRIBE
            • EXPLAIN
            • DROP STREAM
            • DROP TABLE
            • PRINT
            • SELECT
              • CAST
              • LIKE
            • SHOW TOPICS
            • SHOW STREAMS
            • SHOW TABLES
            • SHOW QUERIES
            • SHOW PROPERTIES
            • TERMINATE
          • Scalar functions
          • Aggregate functions
          • Key Requirements
            • Message Keys
            • What To Do If Your Key Is Not Set or Is In A Different Format
            • Streams
            • Tables
        • KSQL REST API Reference
          • REST Endpoint
          • Content Types
          • Errors
          • Get the Status of a KSQL Server
          • Run a KSQL Statement
          • Run A Query And Stream Back The Output
          • Get the Status of a CREATE, DROP, or TERMINATE
        • Frequently Asked Questions
          • What are the benefits of KSQL?
          • What are the technical requirements of KSQL?
          • Is KSQL owned by the Apache Software Foundation?
          • How does KSQL compare to Apache Kafka’s Streams API?
          • Does KSQL work with vanilla Apache Kafka clusters, or does it require the Kafka version included in Confluent Platform?
          • Does KSQL support Kafka’s exactly-once processing semantics?
          • Can I use KSQL with my favorite data format (e.g. JSON, Avro)?
          • Is KSQL fully compliant to ANSI SQL?
          • How do I shut down a KSQL environment?
          • How do I configure the target Kafka cluster?
          • How do I add KSQL servers to an existing KSQL cluster?
          • How can I lock-down KSQL servers for production and prevent interactive client access?
          • How do I use Avro data and integrate with Confluent Schema Registry?
          • How can I scale out KSQL?
          • Can KSQL connect to an Apache Kafka cluster over SSL?
          • Can KSQL connect to an Apache Kafka cluster over SSL and authenticate using SASL?
          • Will KSQL work with Confluent Cloud?
          • Will KSQL work with a Apache Kafka cluster secured using Kafka ACLs?
          • Will KSQL work with a HTTPS Confluent Schema Registry?
          • Where are KSQL-related data and metadata stored?
          • Which KSQL queries read or write data to Kafka?
          • How do I check the health of a KSQL server?
          • What if automatic topic creation is turned off?
        • Troubleshoot KSQL issues
          • SELECT query hangs and doesn’t stop
          • SELECT query returns no results
            • Check the stream’s underlying Apache Kafka® topic
            • Check the Kafka topic for data
            • Read from the beginning of the topic
            • Check the query predicate
            • Check for deserialization errors
          • KSQL CLI doesn’t connect to KSQL server
            • Check the port that KSQL CLI is using
            • Check the KSQL server configuration
            • Check for a port conflict
          • View the message count for a KSQL query
          • Can’t create a stream from the output of a windowed aggregate
          • KSQL doesn’t clean up its internal topics
          • Replicated topic with Avro schema causes errors
          • Snappy encoded messages don’t decompress
          • Check the KSQL server logs
    • Kafka Clients
      • Kafka Consumers
        • Concepts
        • Configuration
        • Initialization
        • Basic Usage
          • Java Client
          • C/C++ Client (librdkafka)
          • Python, Go and .NET Clients
        • Detailed Examples
          • Basic Poll Loop
          • Shutdown with Wakeup
          • Synchronous Commits
          • Asynchronous Commits
        • Administration
          • List Groups
          • Describe Group
      • Kafka Producers
        • Concepts
        • Configuration
        • Examples
          • Initial Setup
          • Asynchronous Writes
          • Synchronous Writes
      • JMS Client
        • Overview
        • Example
        • Requirements
        • Installation
        • Development Guide
          • Configuration
            • Required Configuration Properties
            • Optional Configuration Properties
            • Standard Kafka Configuration Properties (Optional)
            • Confluent Specific Features (Optional)
            • Enabling TLS Encryption (Optional)
          • Developing JMS Applications
            • Creating a ConnectionFactory
            • Creating a Destination
            • Creating a Connection
            • Creating Sessions
            • Producing / Consuming Messages
            • JNDI Context
            • Threading
        • JMS 1.1 Compatibility
          • Features
          • Unsupported Features
            • Session
            • ConnectionFactory
            • Connection
        • Appendix 1 - Creating a Shaded Fat .jar
      • API Docs
    • Connector Developer Guide
      • Core Concepts and APIs
        • Connectors and Tasks
        • Partitions and Records
        • Dynamic Connectors
      • Developing a Simple Connector
        • Connector Example
        • Task Example - Source Task
        • Sink Tasks
        • Resuming from Previous Offsets
      • Dynamic Input/Output Partitions
      • Configuration Validation
      • Working with Schemas
      • Schema Evolution
      • Testing
      • Packaging
        • Creating an Archive
        • Creating an Uber JAR
    • Schema Registry
      • Introduction
        • Quickstart
        • Installation
        • Deployment
        • Development
        • Requirements
        • Contribute
        • License
      • Changelog
        • Version 4.1.0
        • Version 4.0.1
        • Version 4.0.0
          • Upgrade Notes
        • Version 3.3.0
        • Version 3.2.1
        • Version 3.2.0
        • Version 3.1.1
        • Version 3.1.0
        • Version 3.0.1
        • Version 3.0.0
        • Version 2.0.1
        • Version 2.0.0
      • API Reference
        • Overview
          • Compatibility
          • Content Types
          • Errors
        • Schemas
        • Subjects
        • Compatibility
        • Config
      • Schema Registry Configuration Options
        • kafkastore.connection.url
        • kafkastore.bootstrap.servers
        • listeners
        • avro.compatibility.level
        • host.name
        • kafkastore.ssl.key.password
        • kafkastore.ssl.keystore.location
        • kafkastore.ssl.keystore.password
        • kafkastore.ssl.truststore.location
        • kafkastore.ssl.truststore.password
        • kafkastore.topic
        • kafkastore.topic.replication.factor
        • response.mediatype.default
        • ssl.keystore.location
        • ssl.keystore.password
        • ssl.key.password
        • ssl.truststore.location
        • ssl.truststore.password
        • response.mediatype.preferred
        • zookeeper.set.acl
        • kafkastore.init.timeout.ms
        • kafkastore.security.protocol
        • kafkastore.ssl.enabled.protocols
        • kafkastore.ssl.keystore.type
        • kafkastore.ssl.protocol
        • kafkastore.ssl.provider
        • kafkastore.ssl.truststore.type
        • kafkastore.timeout.ms
        • master.eligibility
        • kafkastore.sasl.kerberos.service.name
        • kafkastore.sasl.mechanism
        • access.control.allow.methods
        • ssl.keystore.type
        • ssl.truststore.type
        • ssl.protocol
        • ssl.provider
        • ssl.client.auth
        • ssl.enabled.protocols
        • access.control.allow.origin
        • debug
        • kafkastore.ssl.cipher.suites
        • kafkastore.ssl.endpoint.identification.algorithm
        • kafkastore.ssl.keymanager.algorithm
        • kafkastore.ssl.trustmanager.algorithm
        • kafkastore.zk.session.timeout.ms
        • metric.reporters
        • metrics.jmx.prefix
        • metrics.num.samples
        • metrics.sample.window.ms
        • port
        • request.logger.name
        • schema.registry.inter.instance.protocol
        • schema.registry.resource.extension.class
        • schema.registry.zk.namespace
        • shutdown.graceful.ms
        • ssl.keymanager.algorithm
        • ssl.trustmanager.algorithm
        • ssl.cipher.suites
        • ssl.endpoint.identification.algorithm
        • kafkastore.sasl.kerberos.kinit.cmd
        • kafkastore.sasl.kerberos.min.time.before.relogin
        • kafkastore.sasl.kerberos.ticket.renew.jitter
        • kafkastore.sasl.kerberos.ticket.renew.window.factor
        • kafkastore.group.id
      • Design Overview
        • Schema ID Allocation
        • Kafka Backend
        • Single Master Architecture
          • Kafka Coordinator Master Election
          • ZooKeeper Master Election
      • Schema Registry Operations
        • Production Deployment
          • Hardware
          • Memory
          • CPUs
          • Disks
          • Network
          • JVM
          • Important Configuration Options
          • listeners
            • Don’t Modify These Storage Settings
          • Kafka & ZooKeeper
          • Migration from ZooKeeper master election to Kafka master election
          • Downtime for Writes
          • Complete downtime
          • Backup and Restore
        • Monitoring
          • Global Metrics
          • Per-Endpoint Metrics
          • Endpoints
      • Multi-DC Setup
        • Overview
        • Kafka Election
          • Recommended Deployment
          • Important Settings
          • Setup
          • Run Book
        • ZooKeeper Election
          • Recommended Deployment
          • Important Settings
          • Setup
          • Run Book
      • Security Overview
        • Kafka Store
        • ZooKeeper
        • Configuring the REST API for HTTP or HTTPS
          • Additional configurations for HTTPS
          • Migrating from HTTP to HTTPS
        • Authorizing Access to the Schemas Topic
      • Serializer and Formatter
        • Serializer
          • Subject Name Strategy
          • Basic Auth Security
        • Formatter
        • Wire Format
          • Compatibility Guarantees
      • Schema Deletion Guidelines
      • Maven Plugin
        • schema-registry:download
        • schema-registry:test-compatibility
        • schema-registry:register
      • Kafka Connect
        • Example Configuration
        • Configuration Options
    • Docker Developer Guide
      • Image Design Overview
        • The Bootup Process
        • Configuration
        • Preflight Checks
        • Launching the Process
        • Development Guidelines
      • Setup
        • Building the Images
        • Running Tests
        • Make Targets
      • Extending the Docker Images
        • Prerequisites
        • Adding Connectors to the Kafka Connect Image
        • Examples
      • Utility Scripts
        • Docker Utility Belt (dub)
        • Confluent Platform Utility Belt (cub)
        • Client Properties
      • References
    • Data Serialization and Evolution
      • Avro
        • Defining an Avro Schema
        • Schema Evolution
        • Backward Compatibility
        • Forward Compatibility
        • Full Compatibility
        • Confluent Schema Registry
    • Kafka REST Proxy
      • Kafka REST Proxy
        • Quickstart
          • Produce and Consume JSON Messages
          • Produce and Consume Avro Messages
          • Produce and Consume Binary Messages
          • Inspect Topic Metadata
        • Features
        • Installation
        • Deployment
        • Development
        • Requirements
        • Contribute
        • License
      • Changelog
        • Version 3.3.0
        • Version 3.2.2
        • Version 3.2.1
        • Version 3.2.0
        • Version 3.1.2
        • Version 3.1.1
        • Version 3.1.0
        • Version 3.0.0
        • Version 2.0.1
        • Version 2.0.0
      • API Reference
        • Overview
          • Content Types
          • Errors
        • API v2
          • Topics
          • Partitions
          • Consumers
          • Brokers
        • API v1
          • Topics
          • Partitions
          • Consumers
          • Brokers
      • REST Proxy Configuration Options
        • Security Configuration Options
      • Configuration Options for HTTPS
      • Configuration Options for SSL Encryption between REST Proxy and Apache Kafka Brokers
      • Configuration Options for SASL Authentication between REST Proxy and Apache Kafka Brokers
        • Interceptor Configuration Options
      • Kafka REST Operations
        • Production Deployment
          • Hardware
            • Memory
            • CPUs
            • Disks
            • Network
          • JVM
          • Deployment
          • Important Configuration Options
          • Don’t Touch These Settings!
          • Post Deployment
        • Monitoring
          • Global Metrics
          • Per-Endpoint Metrics
          • Endpoints
      • Security Overview
    • Application Development
      • kafkacat Utility
        • Consumer Mode
        • Producer Mode
        • Metadata Listing Mode
      • Native Clients with Serializers
        • Java
        • C/C++
      • REST Proxy
    • Confluent Hub
      • Overview
        • Terminology
      • Confluent Hub Client
        • Installing Confluent Hub Client
          • Archive
        • Installing Components With Confluent Hub Client
        • Command Reference
          • confluent-hub help
            • Description
            • Usage
            • Options
            • Positional arguments
            • Example
          • confluent-hub install
            • Description
            • Usage
            • Options
            • Positional arguments
            • Example
            • Dry Run
            • Scripting
        • Error Codes
      • Contributing Components
        • Legal Requirements for Community Submissions
        • Technical Requirements for Submissions
        • Verifying your Submission’s Authenticity
        • Submitting a Component
      • Component Archive Specification
        • Component Archive Format
          • Component Archive Name
          • Component Archive Structure
        • Component Manifests
          • A sample manifest
          • Manifest Field Descriptions
          • Logo Files
          • Verification Level
        • Maven Packaging Plugin
    • Camus
      • Camus
        • Key Features
        • Quickstart
        • Installation
        • Deployment
        • Development
        • Requirements
        • Contribute
        • License
      • Changelog
        • Version 4.0.0
        • Version 3.3.1
        • Version 3.3.0
        • Version 3.2.2
        • Version 3.2.1
        • Version 3.2.0
        • Version 3.1.2
        • Version 3.1.1
        • Version 3.1.0
        • Version 3.0.0
        • Version 2.0.1
        • Version 2.0.0
      • Camus Configuration Options
        • Schema Registry Configuration
        • Camus Job Configuration
        • Kafka Configuration
        • Example Configuration
      • Design
      • Operations
        • Important Configuration Options
  • Docs
  • Development »
  • Camus
  • View page source

Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Camus¶

Contents:

  • Camus
    • Key Features
    • Quickstart
    • Installation
    • Deployment
    • Development
    • Requirements
    • Contribute
    • License
  • Changelog
    • Version 4.0.0
    • Version 3.3.1
    • Version 3.3.0
    • Version 3.2.2
    • Version 3.2.1
    • Version 3.2.0
    • Version 3.1.2
    • Version 3.1.1
    • Version 3.1.0
    • Version 3.0.0
    • Version 2.0.1
    • Version 2.0.0
  • Camus Configuration Options
    • Schema Registry Configuration
    • Camus Job Configuration
    • Kafka Configuration
    • Example Configuration
  • Design
  • Operations
    • Important Configuration Options

© Copyright , Confluent, Inc. Privacy Policy | Terms & Conditions. Apache, Apache Kafka, Kafka and the Kafka logo are trademarks of the Apache Software Foundation. All other trademarks, servicemarks, and copyrights are the property of their respective owners.

Please report any inaccuracies on this page or suggest an edit.

Next Previous

Last updated on Jul 25, 2021.

Built with Sphinx using a theme provided by Read the Docs.
    Expand Content v.--