Tutorial: User Management in Confluent Cloud

This tutorial provides an end-to-end workflow for Confluent Cloud user and service account management. In this example, a Confluent Cloud user account is created. The user then creates topics, a service account for applications, and then access control lists (ACLs) to authorize access.

Step 1: Invite User

A Confluent Cloud administrator invites a user via email address (e.g. susan@myemail.com).

../../_images/cloud-susan-invite.png

Step 2: Login to Confluent Cloud Web Browser and CLI

  1. User accepts the invitation from email and logs in via a web browser.

    ../../_images/cloud-susan-accept.png
  2. Install the Confluent Cloud CLI.

  3. Log in to the Confluent Cloud CLI using the ccloud login command with the cluster URL specified.

    ccloud login
    

    Specify your credentials.

    Enter your Confluent Cloud credentials:
    Email: susan@myemail.com
    Password: ************
    

Step 3: Configure CLI and Connect to Cluster

  1. List the available clusters using the ccloud kafka cluster list command.

    ccloud kafka cluster list
    

    The output should resemble:

         ID     |          NAME          | PROVIDER |   REGION    | DURABILITY | STATUS
    +-----------+------------------------+----------+-------------+------------+---------+
      lkc-43npm | Cluster1               | aws      | us-west-2   | LOW        | UP
      lkc-lq8dd | Cluster2               | aws      | us-west-2   | LOW        | UP
      lkc-43nkw | Cluster3               | aws      | us-west-2   | LOW        | DELETED
      lkc-4xrp1 | Cluster4               | gcp      | us-central1 | LOW        | UP
    
  2. Connect to Cluster4 (lkc-4xrp1) using the ccloud kafka cluster use command.

    ccloud kafka cluster use lkc-4xrp1
    

Step 4: Configure Access to Kafka

  1. Log in to your cluster using the ccloud login command with the cluster URL specified.

    ccloud login
    
    Enter your Confluent Cloud credentials:
    Email: susan@myemail.com
    Password:
    
  2. Set the Confluent Cloud environment.

    1. Get the environment ID.

      ccloud environment list
      

      Your output should resemble:

           Id    |      Name
      +----------+----------------+
        * a-542  | dev
          a-4985 | prod
          a-2345 | jdoe-gcp-env
          a-9012 | jdoe-aws-env
      
    2. Set the environment using the ID (<env-id>).

      ccloud environment use <env-id>
      

      Your output should resemble:

      Now using a-4985 as the default (active) environment.
      
  3. Set the cluster to use.

    1. Get the cluster ID.

      ccloud kafka cluster list
      

      Your output should resemble:

            Id      |       Name        | Provider |   Region    | Durability | Status
      +-------------+-------------------+----------+-------------+------------+--------+
          ekg-rr8v7 | dev-aws-oregon    | aws      | us-west-2   | LOW        | UP
          ekg-q2j96 | prod              | gcp      | us-central1 | LOW        | UP
      
    2. Set the cluster using the ID (<cluster-id>). This is the cluster where the commands are run.

      ccloud kafka cluster use <cluster-id>
      
  4. Create the API key/secret with the resource ID (<resource-id>) specified and save the output. You can find the Kafka resource ID by using the ccloud kafka cluster list command. You can find the Schema Registry resource ID by using the ccloud schema-registry cluster describe command.

    ccloud api-key create --resource <resource-id>
    

    Your output should resemble:

    Save the API key and secret. The key/secret is not retrievable later.
    +---------+------------------------------------------------------------------+
    | API Key | KIELS5LZKXCBOT9L                                                 |
    | Secret  | XVLE434R43R532RFSASDeaatawefafeazzzeeeeeelllll4354t5345452432x   |
    +---------+------------------------------------------------------------------+
    

    Tip

    To use an existing API key/secret, run this command with the resource ID (<resource-id>), API key (<api-key>), and API secret (<api-secret>) specified. This command registers an API key/secret created by another process and stores it locally.

    ccloud api-key store <api-key> <api-secret> --resource <resource-id>
    
  5. Associate the Kafka API key/secret with this cluster, the API key (<api-key>) must be specified. This step is not necessary for Schema Registry resources.

    ccloud api-key use <api-key>
    

Step 5: Create and Manage Topics

  1. Create a topic with all the default values using the ccloud kafka topic create command.

    ccloud kafka topic create myTopic1
    
  2. Create a topic with six partitions and a replication factor of three.

    ccloud kafka topic create myTopic2 --partitions 6 --replication-factor 3
    
  3. List topics using the ccloud kafka topic list command.

    ccloud kafka topic list
    

    The output should resemble:

        NAME
    +----------+
      myTopic1
      myTopic2
    
  4. Delete a topic named myTopic1 using the ccloud kafka topic delete command.

    ccloud kafka topic delete myTopic1
    
  5. Describe a topic using the ccloud kafka topic describe command.

    ccloud kafka topic describe myTopic2
    

    The output should resemble:

    Topic: myTopic2 PartitionCount: 6 ReplicationFactor: 3
       TOPIC   | PARTITION | LEADER | REPLICAS |   ISR
    +----------+-----------+--------+----------+---------+
      myTopic2 |         0 |      2 | [2 1 3]  | [2 1 3]
      myTopic2 |         1 |      3 | [3 2 0]  | [3 2 0]
      myTopic2 |         2 |      0 | [0 3 1]  | [0 3 1]
      myTopic2 |         3 |      1 | [1 0 2]  | [1 0 2]
      myTopic2 |         4 |      2 | [2 3 0]  | [2 3 0]
      myTopic2 |         5 |      3 | [3 0 1]  | [3 0 1]
    
  6. Modify the myTopic2 configuration to set cleanup.policy using the ccloud kafka topic update command.

    ccloud kafka topic update myTopic2 --config cleanup.policy=compact
    

Step 6: Produce and consume

  1. Produce messages to a topic using the ccloud kafka topic produce command. Press CTRL-C when you are done.

    ccloud kafka topic produce myTopic2
    
  2. Consume messages from a topic using the ccloud kafka topic consume command. Press CTRL-C when you are done.

    ccloud kafka topic consume myTopic2
    

Step 7: Create Service Accounts and API Key/Secret Pairs

  1. Create a service account named dev-apps using the ccloud service-account create command.

    ccloud service-account create "dev-apps" \
    --description "Service account for dev apps"
    

    The output should resemble:

    +----------------+--------------------------------+
    | Id             |                           1629 |
    | Name           | dev-apps                       |
    | Description    | Service account for dev apps   |
    | OrganizationId |                            857 |
    +----------------+--------------------------------+
    

    Note the Id associated to this service account, in this case 1629.

  2. Create an API key/secret pair for this service account using the ccloud api-key create command. It also needs the cluster ID, which is available from the output of ccloud kafka cluster list.

    ccloud api-key create --service-account-id 1629 --resource lkc-4xrp1
    
  3. Take note of the API key and secret, this is the only time you will be able to see it.

  4. Client applications that will connect to this cluster will need to configure at least these three identifying parameters:

    • API key: available when you create the API key/secret pair the first time
    • API secret: available when you create the API key/secret pair the first time
    • bootstrap.servers: set to the Endpoint in the output of ccloud kafka cluster describe

Step 8: Manage Access with ACLs

  1. Grant the dev-apps service account the ability to produce to a particular topic using the ccloud kafka acl create command.

    ccloud kafka acl create --allow --service-account-id 1629 --operation WRITE --topic myTopic2
    
  2. If the service also needs to create topics, grant the dev-apps service account the ability to create new topics.

    ccloud kafka acl create --allow --service-account-id 1629 --operation CREATE --topic myTopic2
    
  3. Grant the dev-apps service account the ability to consume from a particular topic using the ccloud kafka acl create command. Note that it requires two commands: one to specify the topic and one to specify the consumer group.

    ccloud kafka acl create --allow --service-account-id 1629 --operation READ --topic myTopic2
    ccloud kafka acl create --allow --service-account-id 1629 --operation READ --consumer-group java_example_group_1
    
  4. List all ACLs for the dev-apps service account using the ccloud kafka acl list command.

    ccloud kafka acl list --service-account-id 1629
    

    The output should resemble:

      SERVICEACCOUNTID | PERMISSION | OPERATION | RESOURCE |        NAME
    +------------------+------------+-----------+----------+----------------------+
      User:1629        | ALLOW      | WRITE     | TOPIC    | myTopic2
      User:1629        | ALLOW      | CREATE    | TOPIC    | myTopic2
      User:1629        | ALLOW      | READ      | TOPIC    | myTopic2
      User:1629        | ALLOW      | READ      | GROUP    | java_example_group_1
    
  5. You can add ACLs on prefixed resource patterns. For example, you can add an ACL for any topic whose name starts with demo

    ccloud kafka acl create --allow --service-account-id 1629 --operation WRITE --topic demo --prefix
    
  6. You can add ACLs using a wildcard which matches any name for that resource. For example, you can add an ACL to allow a topic of any name.

    ccloud kafka acl create --allow --service-account-id 1629 --operation WRITE --topic '*'
    
  7. Remove an ACL from the dev-apps service account using the ccloud kafka acl delete command.

    ccloud kafka acl delete --allow --service-account-id 1629 --operation WRITE --topic myTopic2
    

Step 9: Logout

Log out using the ccloud logout command.

ccloud logout