# Confluent Platform Overview Confluent Platform is a full-scale [streaming platform](../_glossary.md#term-event-streaming-platform) that enables you to easily access, store, and manage data as continuous, real-time streams. Built by the original creator/co-creator of Apache Kafka®, Confluent Platform is an enterprise-ready platform that completes Kafka with advanced capabilities designed to help accelerate application development and connectivity. Confluent Platform enables transformations through stream processing, simplifies enterprise operations at scale, and meets stringent architectural requirements. Confluent Platform is software you download and manage yourself. Any Kafka use cases are also Confluent Platform use cases. Confluent Platform is a specialized distribution of Kafka that includes additional features and APIs. Many of the commercial Confluent Platform features are built into Kafka brokers as a function of Confluent Server. By integrating historical and real-time data into a single, central source of truth, Confluent makes it easy to build an entirely new category of modern, event-driven applications, gain a universal data pipeline, and unlock powerful new use cases with full scalability, performance, and reliability. ## Kafka use cases Kafka is used for a wide array of use cases across numerous industries, such as: - Financial services - Omnichannel retail - Autonomous cars - Fraud detection services - Microservices - IoT You can use Kafka to collect user activity data, system logs, application metrics, stock ticker data, and device instrumentation signals. Regardless of the use case, Confluent Platform lets you focus on how to derive business value from your data rather than worrying about the underlying mechanics, such as how data is being transported or integrated between disparate systems. Specifically, Confluent Platform simplifies connecting data sources to Kafka, building streaming applications, as well as securing, monitoring, and managing your Kafka infrastructure. ## Confluent Platform features At the core of Confluent Platform is Kafka, the most popular open source distributed streaming platform. Kafka enables you to: - Publish and subscribe to streams of records - Store streams of records in a fault tolerant way - Process streams of records Each Confluent Platform release includes the latest release of Kafka and additional tools and services that make it easier to build and manage an event streaming platform. Confluent Platform provides community and commercially licensed features such as [Schema Registry](/platform/current/schema-registry/index.html), [Cluster Linking](../multi-dc-deployments/cluster-linking/index.md#cluster-linking), a [REST Proxy](../kafka-rest/index.md#kafkarest-intro), [Confluent Platform for Apache Flink](/cp-flink/current/overview.html), and [100+ pre-built Kafka connectors](https://docs.confluent.io/connectors/index.html), and [ksqlDB](../ksqldb/overview.md#ksql-home). For more information about Confluent components and the license that applies to them, see [Confluent Licenses](../installation/license.md#cp-license-overview).  ### Kafka capabilities Confluent Platform provides all of Kafka’s open-source features plus additional proprietary components. Following is a summary of Kafka features. For an overview of Kafka use cases, features and terminology, see [Kafka Introduction](/kafka/introduction.html). - At the core of Kafka is the [Kafka broker](../_glossary.md#term-Kafka-broker). A broker stores data in a durable way from clients in one or more topics that can be consumed by one or more clients. Kafka also provides several [command-line tools](../tools/cli-reference.md#cp-all-cli) that enable you to start and stop Kafka, create topics and more. - Kafka provides security features such as [data encryption](../security/protect-data/encrypt-tls.md#kafka-ssl-encryption) between producers and consumers and brokers using SSL / TLS. [Authentication](../security/authentication/overview.md#authentication-overview) using SSL or SASL and authorization using ACLs. These security features are disabled by default. - Additionally, Kafka provides the following [Java APIs](/kafka/kafka-apis.html). - The Producer API that enables an application to send messages to Kafka. To learn more, see [Producer](../clients/producer.md#kafka-producer). - The Consumer API that enables an application to subscribe to one or more topics and process the stream of records produced to them. To learn more, see [Consumer](../clients/consumer.md#kafka-consumer). - [Kafka Connect for Confluent Platform](../connect/index.md#kafka-connect), a component that you can use to stream data between Kafka and other data systems in a scalable and reliable way. It makes it simple to configure connectors to move data into and out of Kafka. Kafka Connect can ingest entire databases or collect metrics from all your application servers into Kafka topics, making the data available for stream processing. Connectors can also deliver data from Kafka topics into secondary indexes like Elasticsearch or into batch systems such as Hadoop for offline analysis. - The [Streams API](../streams/introduction.md#streams-intro) that enables applications to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams. It has a very low barrier to entry, easy operationalization, and a high-level DSL for writing stream processing applications. As such it is the most convenient yet scalable option to process and analyze data that is backed by Kafka. - The [Admin API](/kafka/kafka-apis.html#admin-client-api) that provides the capability to create, inspect, delete, and manage topics, brokers, ACLs, and other Kafka objects. To learn more, see [Confluent REST Proxy for Apache Kafka on Confluent Platform](../kafka-rest/index.md#kafkarest-intro), which leverages the Admin API. ### Development and connectivity features To supplement Kafka’s Java APIs and to help you connect all of your systems to Kafka and build full streaming applications, Confluent Platform provides the following features: - Connectors, which leverage the Kafka Connect APIs to connect Kafka to other systems such as databases, key-value stores, search indexes, and file systems. Confluent Marketplace has downloadable connectors for the most popular data sources and sinks. These include fully tested and supported versions of these connectors with Confluent Platform. See the following documentation for more information: - [Get started with Kafka Connect for Confluent Platform](../connect/userguide.md#connect-userguide) - [Supported Self-Managed Connectors for Confluent Platform](../connect/supported.md#connect-bundled-connectors) - [Preview Self-Managed Connectors for Confluent Platform](../connect/preview.md#connect-preview-connectors) Confluent provides both commercial and community licensed connectors. For details, and to download connectors , see [Confluent Marketplace](https://www.confluent.io/hub/). - [Confluent Platform for Apache Flink](/cp-flink/current/get-started/overview.html), which brings support for Apache Flink® to Confluent Platform. Confluent Platform for Apache Flink is fully compatible with Apache Flink, and integrates nicely with Confluent Platform, including support for Kafka topics as sources and sinks. Flink applications are deployed in Kubernetes with [Confluent Management Framework (CMF)](/cp-flink/current/get-started/get-started-application.html), which is a central management component that enables users to securely manage a fleet of Flink applications across multiple environments. - [Non-java clients](../clients/overview.md#kafka-clients) including [C/C++](/kafka-clients/librdkafka/current/overview.html), [Python](/kafka-clients/python/current/overview.html), [Go](/kafka-clients/go/current/overview.html), [.NET client](/kafka-clients/dotnet/current/overview.html), and [Javascript](/kafka-clients/javascript/current/overview.html) libraries in addition to the Java client. These clients are full-featured and performant. For more information, see [Build Streaming Applications on Confluent Platform](../clients/overview.md#kafka-clients). - A [REST Proxy](../kafka-rest/index.md#kafkarest-intro), which leverages the Admin API and makes it easy to work with Kafka from any language by providing a RESTful HTTP service for interacting with Kafka clusters. The REST Proxy supports all the admin core functionality: sending messages to Kafka, reading messages, both individually and as part of a consumer group, and inspecting cluster metadata, such as the list of topics and their settings. You get the full benefits of the high quality, officially maintained Java clients from any language. The REST Proxy also integrates with Schema Registry. Because it automatically translates JSON data to and from Avro, you can get all the benefits of centralized schema management from any language using only HTTP and JSON. - All of the Kafka command-line tools and other [CLI Tools Bundled With Confluent Platform](../tools/cli-reference.md#cp-all-cli). In addition, Confluent provides the [Confluent CLI](https://docs.confluent.io/confluent-cli/current/overview.html) and a [tool to help with KRaft migrations](../tools/kraft-migration-tool.md#kraft-migration-tool). - [Schema Registry](/platform/current/schema-registry/index.html), which provides a centralized repository for managing and validating schemas for topic message data, and for serialization and deserialization of data over a network. With a messaging service like Kafka, services that interact with each other must agree on a common format, called a schema, for messages. Schema Registry helps enable safe, zero-downtime evolution of schemas by centralizing schema management. It provides a RESTful interface for storing and retrieving Avro®, JSON Schema, and Protobuf schemas. Schema Registry tracks all versions of schemas and enables the evolution of schemas according to user-defined compatibility settings. Schema Registry also includes plugins for Kafka clients that handle schema storage and retrieval for Kafka messages that are sent in the Avro format. For more information, see the [Schema Registry Documentation](/platform/current/schema-registry/index.html). For a hands-on introduction to working with schemas, see the [On-Premises Schema Registry Tutorial](/platform/current/schema-registry/schema_registry_onprem_tutorial.html). For a deep dive into supported serialization and deserialization formats, see [Formats, Serializers, and Deserializers](/platform/current/schema-registry/fundamentals/serdes-develop/index.html). - [ksqlDB](../ksqldb/overview.md#ksql-home), a streaming SQL engine for Kafka. It provides an interactive SQL interface for stream processing on Kafka, without the need to write code in a programming language such as Java or Python. ksqlDB is scalable, elastic, fault-tolerant, and real-time. It supports a wide range of streaming operations, including data filtering, transformations, aggregations, joins, windowing, and sessionization. For more information, see the [ksqlDB Documentation](../ksqldb/overview.md#ksql-home), or the [ksqlDB Quick Start](../ksqldb/quickstart.md#ksqldb-quick-start). ### Management and monitoring features Confluent Platform provides several features to supplement Kafka’s Admin API, and built-in JMX monitoring. - [Confluent Control Center](/control-center/current/overview.html), which is a web-based system for managing and monitoring Kafka. It allows you to easily manage Kafka Connect, to create, edit, and manage connections to other systems. Control Center also enables you to monitor data streams from producer to consumer, assuring that every message is delivered, and measuring how long it takes to deliver messages. Using Control Center, you can build a production data pipeline based on Kafka without writing a line of code. - [Unified Stream Manager](../usm/overview.md#usm-overview) (USM), a unified interface within the web-based Confluent Cloud Console so that you can monitor, manage, and govern both on-premises Confluent Platform and fully-managed Confluent Cloud clusters from a single location. - [Health+](../health-plus/index.md#health-plus), also a web-based tool to help ensure the health of your clusters and minimize business disruption with intelligent alerts, monitoring, and proactive support. - [Metrics reporter](../monitor/metrics-reporter.md#metrics-reporter) for collecting various metrics from a Kafka cluster. The metrics are produced to a topic in a Kafka cluster. ### Performance and scalability features Confluent offers a number of features to scale effectively and get the maximum performance for your investment. - To help save money, you can use the [Tiered Storage in Confluent Platform](../clusters/tiered-storage.md#tiered-storage) feature, which automatically tiers data to cost-effective object storage, and scale brokers only when you need more compute resources. - [Manage Self-Balancing Kafka Clusters in Confluent Platform](../clusters/sbc/index.md#sbc) provides automated load balancing, failure detection and self-healing for your clusters. It provides support for adding or decommissioning brokers as needed, with no manual tuning. Self-Balancing Clusters are the next iteration of Auto Data Balancer in that Self-Balancing auto-monitors clusters for imbalances, and automatically triggers rebalances based on your configurations. Partition reassignment plans and execution are taken care of for you. - To deploy Confluent Platform at scale, you can deploy with [Ansible Playbooks](https://docs.confluent.io/ansible/current/overview.html) or [Confluent for Kubernetes](https://docs.confluent.io/operator/current/). - [Confluent for Kubernetes](https://docs.confluent.io/operator/current/) is a Kubernetes operator. Kubernetes operators extend the orchestration capabilities of Kubernetes by providing the unique features and requirements for a specific platform application. For Confluent Platform, this includes greatly simplifying the deployment process of Kafka on Kubernetes and automating typical infrastructure lifecycle tasks. - [Ansible Playbooks](https://docs.confluent.io/ansible/current/overview.html) enable you to automatically configure, provision, and deploy Confluent Platform clusters using YAML files. ### Security and resilience features Confluent Platform also offers a number of features that build on Kafka’s security features to help ensure your deployment stays secure and resilient. - You can set authorization by role with Confluent’s [Role-based Access Control (RBAC)](../security/authorization/rbac/overview.md#rbac-overview) feature. - If you use Control Center, you can set up [Single Sign On (SSO)](../security/authentication/sso-for-c3/overview.md#sso-for-c3) that integrates with a supported OIDC identity provider, and enable additional security measures such as multi-factor authentication. - The [REST Proxy Security Plugins in Confluent Platform](../confluent-security-plugins/kafka-rest.md#kafka-rest-security-plugins-install) and [Schema Registry Security Plugin for Confluent Platform](../confluent-security-plugins/schema-registry/introduction.md#confluentsecurityplugins-schema-registry-security-plugin) add security capabilities to the Confluent Platform REST Proxy and Schema Registry. The Confluent REST Proxy Security Plugin helps in authenticating the incoming requests and propagating the authenticated principal to requests to Kafka. This enables Confluent REST Proxy clients to utilize the multi-tenant security features of the Kafka broker. The Schema Registry Security Plugin supports authorization for both role-based access control (RBAC) and ACLs. - [Audit logs](../security/compliance/audit-logs/audit-logs-concepts.md#audit-logs-concepts) provide the ability to capture, protect, and preserve authorization activity into topics in Kafka clusters on Confluent Platform using [Confluent Server Authorizer](../security/csa-introduction.md#confluent-server-authorizer). - The [Cluster Linking](../multi-dc-deployments/cluster-linking/index.md#cluster-linking) feature enables you to directly connect clusters and mirror topics from one cluster to another. This makes it easier to build multi-datacenter, multi-region and hybrid cloud deployments. - [Confluent Replicator](../multi-dc-deployments/replicator/index.md#replicator-detail) makes it easier to maintain multiple Kafka clusters in multiple data centers. Managing replication of data and topic configuration between data centers enables use-cases such as active geo-localized deployments, centralized analytics and cloud migration. You can use Replicator to configure and manage replication for all these scenarios from either Control Center or command-line tools. To get started, see the [Replicator documentation](../multi-dc-deployments/replicator/index.md#replicator-detail), including the [Replicator Quick Start](../multi-dc-deployments/replicator/replicator-quickstart.md#replicator-quickstart). ## Install Confluent Platform You can install Confluent Platform and its components manually using [ZIP or TAR installation](../installation/installing_cp/zip-tar.md#prod-kafka-cli-install) packages, using package managers on [CentOS, RHEL, Rocky Linux](../installation/installing_cp/rhel-centos.md#systemd-rhel-centos-install), [Ubuntu, Debian,](../installation/installing_cp/deb-ubuntu.md#systemd-ubuntu-debian-install) or [Docker images](../installation/docker/installation.md#cpdocker-intro). The [Quick Start for Confluent Platform](platform-quickstart.md#quickstart) uses Docker Compose to get you up and running quickly. Docker images of Confluent Platform are available on [Docker Hub](https://hub.docker.com/u/confluentinc). To learn more about the packages, see [Docker Image Reference for Confluent Platform](../installation/docker/image-reference.md#image-reference). Some of these images contain proprietary components that require a Confluent Enterprise license. You can also install using an orchestrator like [Ansible Playbooks](https://docs.confluent.io/ansible/current/overview.html) or [Confluent for Kubernetes](https://docs.confluent.io/operator/current/). For more information about all of the installation options, see [Install Confluent Platform On-Premises](../installation/overview.md#installation). If you would rather take advantage of all of Confluent Platform’s features in a managed cloud environment, you can use [Confluent Cloud](https://www.confluent.io/confluent-cloud/) and get started for free using the [Cloud quick start](/cloud/current/get-started/index.html).