Confluent Cloud Quick Start

Step 1: Log into the Confluent Cloud web interface

  1. Log into Confluent Cloud at https://confluent.cloud.

  2. Click Create cluster.

    _images/cloud-create-topic.png

  3. Specify a cluster name, accept the billing terms, and click Pay and launch. Optionally, you can specify read and write throughput, storage, region, and durability.

    _images/cloud-pay-launch.png

  4. Copy your API key and API secret, accept the acknowledgement, and click Continue. You will use this information to link your Confluent Cloud web interface and Confluent Cloud CLI.

    Important

    The API secret access key may not be retrievable later.

    _images/cloud-confirmation.png

    The Client configuration tab is shown:

    _images/cloud-summary.png

Step 2: Install Confluent Cloud CLI

After you have a working Kafka cluster in Confluent Cloud, you can use the Confluent Cloud command line tool to interact with your cluster from your laptop.

MacOS

Install using these brew commands:

  1. Tap the Confluent repository for Confluent Cloud.

    $ brew tap confluentinc/ccloud

  2. Install Confluent Cloud.

    $ brew install ccloud

  1. Optional: Verify installation by typing ccloud in your terminal.

    $ ccloud

    You should see Confluent Cloud CLI usage information:

    usage: ccloud [(--config-dir <configDir> | -c <configDir>)] [(--verbose | -v)]
        <command> [<args>]

The most commonly used ccloud commands are:
    consume   Consume from a topic.
    help      Display help information
    init      Initialize the CLI.
    produce   Produce to a topic.
    topic     Manage topics.
    version   Print the version number.

See 'ccloud help <command>' for more information on a specific command.

Linux

Download and unzip the Confluent Cloud tarball.

  1. Download and unzip from here. Add the contents of the bin directory to your PATH environment variable so that which ccloud finds the Confluent Cloud CLI command.

  1. Optional: Verify installation by typing ccloud in your terminal.

    $ ccloud

    You should see Confluent Cloud CLI usage information:

    usage: ccloud [(--config-dir <configDir> | -c <configDir>)] [(--verbose | -v)]
        <command> [<args>]

The most commonly used ccloud commands are:
    consume   Consume from a topic.
    help      Display help information
    init      Initialize the CLI.
    produce   Produce to a topic.
    topic     Manage topics.
    version   Print the version number.

See 'ccloud help <command>' for more information on a specific command.

Windows

Download and unzip the Confluent Cloud ZIP file.

  1. Download and unzip from here. Use the bin/ccloud.ps1 PowerShell script to run Confluent Cloud CLI.

  2. Optional: Verify installation by typing ccloud in your terminal.

    $ ccloud

    You should see Confluent Cloud CLI usage information:

    usage: ccloud [(--config-dir <configDir> | -c <configDir>)] [(--verbose | -v)]
        <command> [<args>]

The most commonly used ccloud commands are:
    consume   Consume from a topic.
    help      Display help information
    init      Initialize the CLI.
    produce   Produce to a topic.
    topic     Manage topics.
    version   Print the version number.

See 'ccloud help <command>' for more information on a specific command.

Step 3: Configure Confluent Cloud CLI

This quick start describes how to configure Confluent Cloud for Java clients. You can also use Confluent Cloud with librdkafka-based clients.

  1. Configure Confluent Cloud CLI by running the init command. This command creates a configuration file located at ~/.ccloud/config.

    $ ccloud init

  2. Specify the cluster configuration in the ~/.ccloud/config file based on your client. This cluster information is shown on the Client Configuration tab of Confluent Cloud web interface.

    Bootstrap broker list: <broker-list>
API Key: <api-key-id>
API Secret: <secret-access-key>

Step 4: Create Topics and Produce and Consume to Kafka

  1. Create a topic named test with default options.

    $ ccloud topic create test

    Tip

    • By default the Confluent Cloud CLI creates topics with a replication factor of 3.

  2. Optional: describe the test topic.

    $ ccloud topic describe test

    Your output should resemble:

    Topic:test      PartitionCount:12       ReplicationFactor:3     Configs:min.insync.replicas=2
    Topic: test Partition: 0    Leader: 4       Replicas: 4,5,6 Isr: 4,5,6
    Topic: test Partition: 1    Leader: 5       Replicas: 5,6,7 Isr: 5,6,7
    Topic: test Partition: 2    Leader: 6       Replicas: 6,7,8 Isr: 6,7,8
    Topic: test Partition: 3    Leader: 7       Replicas: 7,8,9 Isr: 7,8,9
    Topic: test Partition: 4    Leader: 8       Replicas: 8,9,10        Isr: 8,9,10
    Topic: test Partition: 5    Leader: 9       Replicas: 9,10,11       Isr: 9,10,11
    Topic: test Partition: 6    Leader: 10      Replicas: 10,11,12      Isr: 10,11,12
    Topic: test Partition: 7    Leader: 11      Replicas: 11,12,13      Isr: 11,12,13
    Topic: test Partition: 8    Leader: 12      Replicas: 12,13,14      Isr: 12,13,14
    Topic: test Partition: 9    Leader: 13      Replicas: 13,14,15      Isr: 13,14,15
    Topic: test Partition: 10   Leader: 14      Replicas: 14,15,16      Isr: 14,15,16
    Topic: test Partition: 11   Leader: 15      Replicas: 15,16,17      Isr: 15,16,17

  3. Modify the test topic to have a retention period of 259200000 milliseconds.

    $ ccloud topic alter test --config="retention.ms=259200000"

    Your output should resemble:

    $ Topic configuration for "test" altered.

  4. Produce items into the test topic.

    1. Run this command.

      ccloud produce -t test

    2. Enter text in your terminal and press Ctrl + C to exit. For example:

      foo
bar
baz
^C

  5. Consume items from the test topic. Press Ctrl + C to exit.

    ccloud consume -b -t test

    Your output should show the items that you entered in the production step:

    baz
foo
bar
^C
Processed a total of 3 messages.

    The order of the consumed messages does not match the order that they were produced. This is because the producer spread them over the 10 partitions in the test topic and the consumer reads from all 10 partitions in parallel.

  6. Delete your test topic.

    Caution

    Use this command carefully as data loss can occur.

    $​ ccloud topic delete

    Your output should resemble:

    Topic "test" marked for deletion.

Step 5: Run Java Examples

In this step you clone the Examples repository from GitHub and run Confluent Cloud Java examples. The examples repository contains demo applications and code examples for Confluent Platform and Apache Kafka.

  1. Clone the Confluent Cloud examples repository from GitHub.

    $ git clone git@github.com:confluentinc/examples.git

  2. Navigate to the /examples/ccloud/java-clients directory.

  3. Build the client examples.

    $ mvn clean package

  4. Run the producer

    $ mvn exec:java -Dexec.mainClass="io.confluent.examples.clients.ProducerExample" \
  -Dexec.args="$HOME/.ccloud/config page_visits 10"

  5. Run the Kafka consumer application to read the records that were just published to the Kafka cluster, and to display the records in the console.

    1. Run the client examples.

      $ mvn clean package

    2. Run the consumer.

      $ mvn exec:java -Dexec.mainClass="io.confluent.examples.clients.ConsumerExample" \
  -Dexec.args="$HOME/.ccloud/config page_visits

      Hit Ctrl+C to stop.

Supported Clients

In addition to Java, Confluent Cloud also supports these clients.

librdkafka

Confluent’s official Python, Golang, and .NET clients for Apache Kafka are all based on librdkafka, as are other community-supported clients such as node-rdkafka.

  1. Configure Confluent Cloud CLI by running the init command.

    $ ccloud init

  2. Specify the librdkafka cluster config information in the ~/.ccloud/config file. The bootstrap.servers, sasl.username, and sasl.password information is shown on the Client Configuration tab of Confluent Cloud web interface. The api.version.request, broker.version.fallback, and api.version.fallback.ms options specify that librdkafka should use the latest protocol version that it can and not fallback to an older version.

    bootstrap.servers=<broker-list>
broker.version.fallback=0.10.0.0
api.version.fallback.ms=0
sasl.mechanisms=PLAIN
security.protocol=SASL_SSL
ssl.ca.location=/usr/local/etc/openssl/cert.pem
sasl.username=<api-key-id>
sasl.password=<secret-access-key>

    For more information about librdkafka and Apache Kafka version compatibility, see the documentation. For a complete list of the librdkafka configuration options, see the documentation.

Next Steps

For more information about Confluent Cloud, see the Confluent Cloud documentation.