.. _ce-quickstart:

|cpe| Quick Start
=================

This quick start shows you how to get up and running with |cpe| and its main components. This quick start will show you the
basic and most powerful capabilities of |cpe|, including using |c3-short| for topic management and stream processing by
using KSQL. In this quick start you will create Kafka topics and streaming queries on these topics by using KSQL. You will
then go to |c3-short| to monitor and analyze the streaming queries!

You can also run an `automated version of this quick start <https://github.com/confluentinc/examples/tree/master/pageviews>`_ designed for |cp| local installs.

.. important::
                .. include:: ../includes/java-reqs.rst
                    :start-after: java_snippet

.. _download-start-cp:

Step 1: Download and Start |cp|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

#. Go to the `downloads page <https://www.confluent.io/download/>`_ and choose **Confluent Platform Enterprise**.

#. Provide your name and email and select **Download**.

#. Decompress the file. You should have these directories:

   .. include:: ../includes/tarball-contents.rst

#. Create a folder named ``<path-to-confluent>/ui/`` and download the
   `experimental KSQL web interface file <https://s3.amazonaws.com/ksql-experimental-ui/ksql-experimental-ui-0.1.war>`_
   to this location. This quick start uses the |ksql-ui| to run KSQL commands for stream processing.

   .. important:: The |ksql-ui| is meant for development purposes in non-production environments. The final version of the interface
                  will be available in a future version of :ref:`control_center`.

   .. code:: bash

        $ mkdir <path-to-confluent>/ui/ && cd <path-to-confluent>/ui/
        $ curl -O https://s3.amazonaws.com/ksql-experimental-ui/ksql-experimental-ui-0.1.war

#.  Start |cp| using the :ref:`Confluent CLI <cli>`. This command will start all of the |cp| components, including Kafka,
    |zk|, |sr|, HTTP REST Proxy for Apache Kafka, Kafka Connect, KSQL, and |c3-short|.

    .. tip:: You can add the ``bin`` directory to your PATH by running: ``export PATH=<path-to-confluent>/bin:$PATH``.

    .. sourcecode:: bash

      $ <path-to-confluent>/bin/confluent start

    Your output should resemble:

    .. include:: ../includes/cli.rst
        :start-after: CE_CLI_startup_output
        :end-before: COS_CP_CLI_startup_output

Step 2: Create Kafka Topics
^^^^^^^^^^^^^^^^^^^^^^^^^^^

In this step Kafka topics are created by using the |c3|. :ref:`control_center` provides the functionality for building
and monitoring production data pipelines and streaming applications.

#.  Navigate to the |c3-short| web interface at `http://localhost:9021/ <http://localhost:9021/>`_.

    .. image:: ../images/c3-landing-page.png
        :width: 600px

#.  Select **Management -> Topics** and click **Create** on the upper-right corner.

    .. image:: ../images/c3-create-topic.png
        :width: 600px

#.  Create a topic named ``pageviews`` and click **Create with defaults**.

    .. image:: ../images/c3-create-topic-name.png
        :width: 600px

#.  Create a topic named ``users`` and click **Create with defaults**.

    .. image:: ../images/c3-create-topic-name-2.png
        :width: 600px

.. include:: includes/quickstart.rst
   :start-after: qs_step_3
   :end-before: qs_step_3_produce_1_start

#.  .. include:: includes/quickstart.rst
        :start-after: qs_step_3_produce_1_start
        :end-before: qs_step_3_produce_1_end

    .. code:: bash

           $ <path-to-confluent>/bin/ksql-datagen quickstart=pageviews format=delimited topic=pageviews maxInterval=100 \
             propertiesFile=<path-to-confluent>/etc/ksql/datagen.properties

#.  .. include:: includes/quickstart.rst
        :start-after: qs_step_3_produce_2_start
        :end-before: qs_step_3_produce_2_end

    .. code:: bash

           $ <path-to-confluent>/bin/ksql-datagen quickstart=users format=json topic=users maxInterval=1000 \
             propertiesFile=<path-to-confluent>/etc/ksql/datagen.properties

Step 4: Create and Write to a Stream and Table using KSQL
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In this step, KSQL queries are run on the ``pageviews`` and ``users`` topics that were created in the previous step. The
KSQL commands are run using the |ksql-ui| at `http://localhost:8088/index.html <http://localhost:8088/index.html>`__. Enter
these commands in the **Query** field and click **RUN**. Each command must be entered separately.

.. important::

    - |cp| and the |ksql-ui| must be :ref:`installed and running <download-start-cp>`.
    - You can also run these commands using the :ref:`KSQL CLI <install_ksql>` from your terminal
      with this command: ``<path-to-confluent>/bin/ksql http://localhost:8088``.
    - All KSQL commands must end with a closing semicolon (``;``).

.. image:: ../images/ksql-interface-create-stream.png
    :width: 600px

^^^^^^^^^^^^^^^^^^^^^^^^^
Create Streams and Tables
^^^^^^^^^^^^^^^^^^^^^^^^^

In this step, KSQL is used to create streams and tables for the ``pageviews`` and ``users`` topics.

#. .. include:: includes/quickstart.rst
        :start-after: create_streams_tables1_start
        :end-before: create_streams_tables1_end

   .. code:: bash

        CREATE STREAM pageviews (viewtime BIGINT, userid VARCHAR, pageid VARCHAR) WITH (KAFKA_TOPIC='pageviews', VALUE_FORMAT='DELIMITED');

   **Tip:** Click the **SHOW STREAMS** predefined query to view your streams.

   .. image:: ../images/ksql-interface-show-streams.png
        :width: 400px

#. .. include:: includes/quickstart.rst
        :start-after: create_streams_tables1_end
        :end-before: create_streams_tables2_end

   .. code:: bash

        CREATE TABLE users (registertime BIGINT, gender VARCHAR, regionid VARCHAR, userid VARCHAR, interests array<VARCHAR>, contact_info map<VARCHAR, VARCHAR>) WITH (KAFKA_TOPIC='users', VALUE_FORMAT='JSON', KEY = 'userid');

   **Tip:** Click the **SHOW TABLES** predefined query to view your tables.

   .. image:: ../images/ksql-interface-show-tables.png
        :width: 400px

^^^^^^^^^^^^^
Write Queries
^^^^^^^^^^^^^

These examples write queries using KSQL.

#. .. include:: includes/quickstart.rst
        :start-after: custom_query_property_start
        :end-before: custom_query_property_end

   .. image:: ../images/ksql-interface-query-property.png
        :width: 400px

#. .. include:: includes/quickstart.rst
        :start-after: create_query_stream3_start
        :end-before: create_query_stream3_end

   .. code:: bash

       SELECT pageid FROM pageviews LIMIT 3;

#. .. include:: includes/quickstart.rst
        :start-after: create_query_stream3_end
        :end-before: create_persist_end

   .. code:: bash

       CREATE STREAM pageviews_female AS SELECT users.userid AS userid, pageid, regionid, gender FROM pageviews LEFT JOIN users ON pageviews.userid = users.userid WHERE gender = 'FEMALE';

#. .. include:: includes/quickstart.rst
        :start-after: create_persist_end
        :end-before: create_persist_regionid_end

   .. code:: bash

       CREATE STREAM pageviews_female_like_89 WITH (kafka_topic='pageviews_enriched_r8_r9', value_format='DELIMITED') AS SELECT * FROM pageviews_female WHERE regionid LIKE '%_8' OR regionid LIKE '%_9';

   **Tip:** Click the **SHOW STREAMS** predefined query to view your streams.

   .. image:: ../images/ksql-interface-show-streams.png
        :width: 400px

#. .. include:: includes/quickstart.rst
        :start-after: create_persist_regionid_end
        :end-before: custom_query_property_start

   .. code:: bash

       CREATE TABLE pageviews_regions AS SELECT gender, regionid , COUNT(*) AS numusers FROM pageviews_female WINDOW TUMBLING (size 30 second) GROUP BY gender, regionid HAVING COUNT(*) > 1;

   **Tip:** Click the **SHOW TABLES** predefined query to view your streams.

   .. image:: ../images/ksql-interface-show-tables.png
        :width: 400px

Step 5: View Your Stream in |c3-short|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

From the |c3-short| interface you can view all of your streaming KSQL queries.

Navigate to the |c3-short| web interface **Monitoring -> Data streams** tab at `http://localhost:9021/monitoring/streams <http://localhost:9021/monitoring/streams>`_.
The monitoring page shows the total number of messages produced and consumed on the cluster. You can scroll down to see
more details on the consumer groups for your queries.

.. tip:: Depending on your machine, these charts may take a few minutes to populate and you might need to refresh your browser.

.. image:: ../images/c3-monitor-ksql.png
    :width: 75%

Here are some cool things to try:

View the consumers that have been created by KSQL
    Click the **View Details** button for the ``ksql_query_CSAS_PAGEVIEWS_FEMALE_LIKE_89`` stream. This graph shows the messages being
    consumed by the stream query.

    .. image:: ../images/ksql-interface-monitor-pgvw-f.png
       :width: 600px


For more information about |c3-short|, see :ref:`control_center`.


Next Steps
^^^^^^^^^^

.. include:: includes/quickstart.rst
    :start-after: next_steps_start
    :end-before: next_steps_end