Kafka Streams is a new library and is not yet recommended for production use.
We are looking for feedback on APIs, operators, documentation, and really anything that will make the end user
experience better. Feel free to provide your feedback via email to
Is Kafka Streams a project separate from Apache Kafka?¶
No, it is not. Kafka Streams is a new component of the Apache Kafka open source project, and thus included in Apache Kafka 0.10+ releases.
Is Kafka Streams a proprietary library of Confluent?¶
No, it is not. Kafka Streams is a new component of the Apache Kafka open source project.
What are the system dependencies of Kafka Streams?¶
Kafka Streams has no external dependencies on systems other than Apache Kafka.
The Kafka Streams implementation included in Apache Kafka 0.10.0.0 has an additional dependency on Apache ZooKeeper.
This explains why you are currently required to configure the
zookeeper.connect setting in your application (see
Configuring a Kafka Streams application in the Developer Guide).
This dependency on ZooKeeper is only temporary though and will be removed once KIP-4 is included in Apache Kafka, and the decision was made to not duplicate parts of the KIP-4 efforts just for the sake of Kafka Streams.
In the meantime, this additional system dependency should not matter much in practice because ZooKeeper is required
for running Apache Kafka anyways.
How do I migrate my Kafka Streams Tech Preview applications to CP 3.0.0 / Kafka 0.10.0.0?¶
We provide instructions in our Migration Guide.
Can I use Kafka Streams with Kafka clusters running 0.9, 0.8, or 0.7?¶
Unfortunately, no. Kafka Streams relies on Kafka 0.10+ functionality. This means that applications using Kafka Streams 0.10.0.0-cp1 only work against Kafka clusters running 0.10.0.0-cp1. They are not compatible with any prior version, including the Kafka version that shipped in the Confluent Tech Preview of Kafka Streams.
Users of the Kafka Streams Tech Preview can migrate their applications with the help of our Migration Guide.
Troubleshooting and debugging¶
Get more meaningful Java stacktraces?¶
When you write Kafka Streams applications, you often create chains of method calls such as the following:
// Code all on one line. Unfortunately, this is bad practice when it comes to stacktraces. myStream.map(...).filter(...).countByKey(...);
Now if your code happens to trigger a runtime error, the Java stacktrace may not be very helpful because the JVM provides only line-number information about where an error occurred (“NullPointerException at line 123”). So, to pick the example above, you may deduce that some operation in the map/filter/countByKey chain failed, but the stacktrace will not tell you where exactly in the affected line.
A simple trick is to split your method chains across multiple lines, so that the row number returned by the stacktrace will more easily identify the actual culprit:
// Split the same code across multiple lines to benefit from more actionable stacktraces. myStream .map(...) .filter(...) .countByKey(...);