Kafka Streams Code Examples for Confluent Platform

If you’re new to Kafka Streams, here is a curated list of resources to get you started.

Getting started examples

Collections

The kafka-streams-examples GitHub repo is a curated repo with examples that demonstrate the use of Kafka Streams DSL, the low-level Processor API, Java 8 lambda expressions, reading and writing Avro data, and implementing unit tests with TopologyTestDriver and end-to-end integration tests using embedded Kafka clusters.

There are also numerous Kafka Streams examples in Kafka Tutorials that provide full code examples with step–by-step instructions.

Java

With lambda expressions for Java 8+:

• WordCountLambdaExample
• AnomalyDetectionLambdaExample
• GlobalKTablesExample
• MapFunctionLambdaExample
• PageViewRegionLambdaExample
• UserRegionLambdaExample
• WikipediaFeedAvroLambdaExample
• ApplicationResetExample
• SessionWindowsExample
• SumLambdaExample
• TopArticlesLambdaExample

Without lambda expressions for Java 7+:

• PageViewRegionExample
• WikipediaFeedAvroExample

Scala

• MapFunctionScalaExample

Security examples

Java programming language

Without lambda expressions for Java 7:

• SecureKafkaStreamsExample

Interactive Queries examples

Since Confluent Platform 3.1+ and Kafka 0.10.1+, it is possible to query state stores created via the Kafka Streams DSL and the Processor API. For further information, see Kafka Streams Interactive Queries for Confluent Platform.

Java

With lambda expressions for Java 8+:

• WordCountInteractiveQueriesExample
• KafkaMusicExample

End-to-end application examples

These demo applications use embedded instances of Kafka and Confluent Schema Registry. They are implemented as integration tests.

See also

To see end-to-end examples of Kafka Streams applications deployed in an event streaming platform with all the services in Confluent Platform and interconnecting other end systems, refer to confluentinc/examples.

Java

With lambda expressions for Java 8+:

• WordCountLambdaIntegrationTest
• FanoutLambdaIntegrationTest
• GenericAvroIntegrationTest
• GlobalKTablesExampleTest
• HandlingCorruptedInputRecordsIntegrationTest
• MapFunctionLambdaIntegrationTest
• MixAndMatchLambdaIntegrationTest – how to mix the DSL and the Processor API
• SpecificAvroIntegrationTest
• StateStoresInTheDSLIntegrationTest – how to use state stores in the DSL
• StreamToStreamJoinIntegrationTest
• StreamToTableJoinIntegrationTest
• TableToTableJoinIntegrationTest
• UserCountsPerRegionLambdaIntegrationTest
• ApplicationResetIntegrationTest
• EventDeduplicationLambdaIntegrationTest
• SumLambdaIntegrationTest
• TopArticlesLambdaExampleTest

Without lambda expressions for Java 7:

• PassThroughIntegrationTest

Scala

• StreamToTableJoinScalaIntegrationTest
• ProbabilisticCountingScalaIntegrationTest - demonstrates how to probabilistically count items in an input stream by implementing a custom state store that is backed by a Count-Min Sketch data structure
• GenericAvroScalaIntegrationTest
• SpecificAvroScalaIntegrationTest
• WordCountScalaIntegrationTest

Event-Driven Microservice example

Java

The Event-Driven Microservice example implements an Order Service that provides a REST interface to POST and GET orders. Posting an order creates an event in Kafka, which is picked up by three different validation engines: a Fraud Service, an Inventory Service, and an Order Details Service. These services validate the order in parallel, emitting a PASS or FAIL based on whether each validation succeeds.

• Microservices example source code
• Microservices example tests
• Self-paced Kafka Streams tutorial based on the microservices example above: deployed in an event streaming platform with all the services in Confluent Platform and interconnecting other end systems

Note

This website includes content developed at the Apache Software Foundation under the terms of the Apache License v2.