Kafka Basics on Confluent Cloud¶

Apache Kafka® is an open-source, distributed, event streaming platform capable of handling large volumes of real-time data. You use Kafka to build real-time streaming applications. Confluent is a commercial, global corporation that specializes in providing businesses with real-time access to data. Confluent was founded by the creators of Kafka, and its product line includes proprietary products based on open-source Kafka. This topic describes Kafka use cases, the relationship between Confluent and Kafka, and key differences between the Confluent products.

How Kafka relates to Confluent¶

Confluent products are built on the open-source software framework of Kafka to provide customers with reliable ways to stream data in real time. Confluent provides the features and know-how that enhance your ability to reliably stream data. If you’re already using Kafka, that means Confluent products support any producer or consumer code you’ve already written with the Kafka Java libraries. Whether you’re already using Kafka or just getting started with streaming data, Confluent provides features not found in Kafka. This includes non-Java libraries for client development and server processes that help you stream data more efficiently in a production environment, like Confluent Schema Registry, ksqlDB, and Confluent Hub. Confluent offers Confluent Cloud, a data-streaming service, and Confluent Platform, software you download and manage yourself.

Kafka use cases¶

Consider an application that uses Kafka topics as a backend to store and retrieve posts, likes, and comments from a popular social media site. The application incorporates producers and consumers that subscribe to those Kafka topics. When a user of the application publishes a post, likes something, or comments, the Kafka producer code in the application sends that data to the associated topic. When the user navigates to a particular page in the application, a Kafka consumer reads from the associated backend topic and the application renders data on the user’s device. For more information, see Use cases in the Apache Kafka Docs hosted by Confluent.

Confluent Platform¶

Confluent Platform is software you download and manage yourself. Any Kafka use cases are also Confluent Platform use cases. Confluent Platform is a specialized distribution of Kafka that includes additional features and APIs. Many of the commercial Confluent Platform features are built into the brokers as a function of Confluent Server.

The fundamental capabilities, concepts, design ethos, and ways of working that you already know from using Kafka, also apply to Confluent Platform. By definition, Confluent Platform ships with all of the basic Kafka command utilities and APIs used in development, along with several additional CLIs to support Confluent specific features. To learn more about Confluent Platform, see What is Confluent Platform?.

Confluent Platform releases include the latest stable version of Apache Kafka, so when you install Confluent Platform you are also installing Kafka. To view a mapping of Confluent Platform release to Kafka versions, see Supported Versions and Interoperability for Confluent Platform.

Ready to get started?

Download Confluent Platform, the self managed, enterprise-grade distribution of Apache Kafka and get started using the Confluent Platform quick start.

Confluent Cloud¶

Confluent Cloud provides Kafka as a cloud service, so that means you no longer need to install, upgrade or patch Kafka server components. You also get access to a cloud-native design, which offers Infinite Storage, elastic scaling and an uptime guarantee. If you’re coming to Confluent Cloud from open source Kafka, you can use data-streaming features only available from Confluent, including non-Java client libraries and proxies for Kafka producers and consumers, tools for monitoring and observability, an intuitive browser-based user interface, enterprise-grade security and data governance features.

Confluent Cloud includes different types of server processes for steaming data in a production environment. In addition to brokers and topics, Confluent Cloud provides implementations of Kafka Connect, Schema Registry, and ksqlDB.