.. _kafka_connect:

|kconnect-long|
===============

|kconnect-long|, an open source component of |ak-tm|, is a framework for connecting
|ak| with external systems such as databases, key-value stores, search indexes, and file systems.

Using |kconnect-long| you can use existing connector implementations for common data sources and sinks to move data into and
out of |ak|.

Source Connector
   A source connector ingests entire databases and streams table updates to |ak| topics. It can also collect
   metrics from all of your application servers into |ak| topics, making the data available for stream processing with low
   latency.

Sink Connector
   A sink connector delivers data from |ak| topics into secondary indexes such as Elasticsearch or batch systems
   such as Hadoop for offline analysis.

|kconnect-long| is focused on streaming data to and from |ak|, making it simpler for you to write high quality, reliable, and
high performance connector plugins. It also enables the framework to make guarantees that are difficult to achieve using
other frameworks. |kconnect-long| is an integral component of an ETL pipeline when combined with |ak| and a stream processing
framework.

|kconnect-long| can run either as a standalone process for running jobs on a single machine (e.g., log collection), or as a
distributed, scalable, fault tolerant service supporting an entire organization. This allows it to scale down to development,
testing, and small production deployments with a low barrier to entry and low operational overhead, and to scale up to support
a large organization's data pipeline.

The main benefits of using |kconnect-long| are:

* **Data Centric Pipeline** -- use meaningful data abstractions to pull or push data to |ak|.
* **Flexibility and Scalability** -- run with streaming and batch-oriented systems on a single node or scaled to an organization-wide service.
* **Reusability and Extensibility** -- leverage existing connectors or extend them to tailor to your needs and lower time to production.

Contents:

.. toctree::
   :maxdepth: 1

   userguide
   Connecting Clients to Confluent Cloud <https://docs.confluent.io/cloud/current/cp-component/clients-cloud-config.html>
   devguide
   quickstart
   concepts
   references/index
   transforms/index
   security
   design
   faq