.. _connect_filestreamconnector:

Kafka FileStream Connectors
===========================

The FileStream Connector examples are intended to show how a simple connector runs for 
those first getting started with Kafka Connect as either a user or developer. It is not 
recommended for production use. Instead, we encourage users to use them to learn in a 
local environment. The examples include both a file source and a file sink to demonstrate
an end-to-end data flow implemented through Kafka Connect. The FileStream Connector examples
are also detailed in the :ref:`developer guide <connect_developing_simple_connector>` as 
a demonstration of how a custom connector can be implemented.

Quickstart
----------

A simple demo of how to use the FileStreamSourceConnector and FileStreamSinkConnector is 
available in the :ref:`Kafka Connect quick start guide <connect_quickstart>`.

FileSource Connector
--------------------

The FileSource Connector reads data from a file and sends it to Kafka. Beyond the :ref:`configurations common 
to all connectors <connect_managing_config_connectors>` it takes only an input ``file`` and output ``topic`` as properties. 
Here is an example configuration:

.. sourcecode:: bash

   name=local-file-source
   connector.class=FileStreamSource
   tasks.max=1
   file=/tmp/test.txt
   topic=connect-test

This connector will read only one file and send the data within that file to Kafka. It will then watch the file for appended 
updates only. Any modification of file lines already sent to Kafka will not be reprocessed.

FileSink Connector
------------------

The FileSink Connector reads data from Kafka and outputs it to a local file. Multiple topics may be specified as with any other
sink connector. The FileSink Connector takes only a ``file`` property in addition to the :ref:`configurations common
to all connectors <connect_managing_config_connectors>`.  Here is an example configuration:

.. sourcecode:: bash

   name=local-file-sink
   connector.class=FileStreamSink
   tasks.max=1
   file=/tmp/test.sink.txt
   topics=connect-test

As messages are added to the topics specified in the configuration, they are produced to a local file as specified in the configuration.