# Tutorial: Access Management on Confluent Cloud This tutorial provides an end-to-end workflow for Confluent Cloud user and service account management. The steps are: - [Step 1: Invite User](#ccloud-invite-user) - [Step 2: Configure the CLI, Cluster, and Access to Kafka](#ccloud-config-cli-cluster-kafka) - [Step 3: Create and Manage Topics](#ccloud-create-manage-topics) - [Step 4: Produce and consume](#ccloud-user-produce-consume) - [Step 5: Create service accounts and API keys](#sa-api-key-secret) - [Step 6: Manage access with ACLs](#cloud-user-access-acls) - [Step 7: Log out](#cloud-user-log-out) ## Step 1: Invite User Refer to [Create a local user account](../authenticate/user-identities/user-accounts/manage-local-user-accounts.md#local-user-username-password). ## Step 2: Configure the CLI, Cluster, and Access to Kafka 1. Accept the invitation from email and log in using a web browser. 2. [Install the Confluent CLI](https://docs.confluent.io/confluent-cli/current/install.html). 3. Log in to the Confluent CLI using the [confluent login](https://docs.confluent.io/confluent-cli/current/command-reference/confluent_login.html) command. ```text confluent login ``` Specify your credentials. ```text Enter your Confluent Cloud credentials: Email: jane.smith@big-data.com Password: ************ ``` The output should resemble: ```text Logged in as "jane.smith@big-data.com". Using environment "a-42619" ("default"). ``` You are logged in to the default environment for your organization. If work in a different environment than the default, set the environment using the ID (``): > ```text > confluent environment use > ``` > Your output should resemble: > ```text > Now using a-4985 as the default (active) environment. > ``` 4. List the available clusters using the [confluent kafka cluster list](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/cluster/confluent_kafka_cluster_list.html) command. ```text confluent kafka cluster list ``` The output should resemble: ```text Id | Name | Type | Cloud | Region | Availability | Status +--------------+---------------+----------+----------+-------------+--------------+----------+ lkc-j5zrrm | dev-test-01 | STANDARD | gcp | us-central1 | single-zone | UP lkc-382g7m | dev-test-02 | BASIC | aws | us-west-2 | single-zone | UP lkc-43npm | prod-test-01 | BASIC | aws | us-west-2 | single-zone | UP lkc-lq8dd | stage-test-01 | BASIC | aws | us-west-2 | single-zone | DELETED ``` 5. Connect to cluster dev-test-01 (`lkc-j5zrrm`) using the [confluent kafka cluster use](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/cluster/confluent_kafka_cluster_use.html) command. This is the cluster where the commands are run. Be sure to replace the cluster ID shown in the example with your own. ```text confluent kafka cluster use lkc-j5zrrm ``` The output should look like: ```text Set Kafka cluster "lkc-j5zrrm" as the active cluster for environment "a-42619". ``` 6. Create an API key and secret, and save them. You must complete this step to produce or consume to your topics. In this step you create an API Key for your user account, which has full permissions. See steps 5 and 6 for guidance on how to create API Keys for service accounts and grant them permissions with ACLs. You can generate the API key using either the Confluent Cloud Console or the Confluent CLI. Be sure to save the API key and secret. - If using the web UI, click the **API access** tab and click **+ Add key**. Save the key and secret, then click the checkbox next to **I have saved my API key and secret and am ready to continue**. ![image](images/cloud-api-key-confirm.png) - If using the Confluent CLI, type the following command (be sure to replace the cluster ID shown in the example with your own): ```text confluent api-key create --resource lkc-j5zrrm ``` The output should look like: ```text It may take a couple of minutes for the API key to be ready. Save the API key and secret. The secret is not retrievable later. +-------------+------------------------------------------------------------------+ | API Key | ABCD1EFGHIJK2LMN | | API Secret | abC1dEf23G4H567IJKLmn8O1PqrST1UvW0XyZAbcdefGHIjK23LMNOpQRSTUv4WX | +-------------+------------------------------------------------------------------+ ``` 7. Optional: Add the API secret using the `confluent api-key store` command. command. When you create an API key with the CLI, it is automatically stored locally. However, when you create an API key using the Confluent Cloud Console or with the CLI on another machine, the secret is not available for use with the Confluent CLI until you store it. API secrets cannot be retrieved after creation. ```text confluent api-key store --resource ``` 8. Set the API key to use for Confluent CLI commands: ```text confluent api-key use --resource ``` ## Step 3: Create and Manage Topics 1. Create topics with all the default values using the [confluent kafka topic create](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_create.html) command. ```text confluent kafka topic create myTopic1 ``` The output should look like: ```text Created topic "myTopic1". ``` Add another topic: ```text confluent kafka topic create myTopic2 ``` 2. Create a topic with six partitions. ```text confluent kafka topic create myTopic3 --partitions 6 ``` The output should look like: ```text Created topic "myTopic3". ``` 3. List topics using the [confluent kafka topic list](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_list.html) command. ```text confluent kafka topic list ``` The output should resemble: ```text Name +----------+ myTopic1 myTopic2 myTopic3 ``` 4. Delete a topic named `myTopic1` using the [confluent kafka topic delete](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_delete.html) command. ```text confluent kafka topic delete myTopic1 ``` The output should look like: ```text Deleted topic "myTopic1". ``` 5. Describe a topic using the [confluent kafka topic configuration list](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_describe.html) command. ```text confluent kafka topic configuration list myTopic2 ``` The output should resemble: ```text Name | Value ------------------------------------------+---------------------- cleanup.policy | delete compression.type | producer delete.retention.ms | 86400000 file.delete.delay.ms | 60000 flush.messages | 9223372036854775807 flush.ms | 9223372036854775807 follower.replication.throttled.replicas | index.interval.bytes | 4096 leader.replication.throttled.replicas | max.compaction.lag.ms | 9223372036854775807 max.message.bytes | 2097164 message.downconversion.enable | true message.format.version | 3.0-IV1 message.timestamp.difference.max.ms | 9223372036854775807 message.timestamp.type | CreateTime min.cleanable.dirty.ratio | 0.5 min.compaction.lag.ms | 0 min.insync.replicas | 2 num.partitions | 6 preallocate | false retention.bytes | -1 retention.ms | 604800000 segment.bytes | 104857600 segment.index.bytes | 10485760 segment.jitter.ms | 0 segment.ms | 604800000 unclean.leader.election.enable | false ``` 6. Modify the `myTopic2` configuration to set `num.partitions` using the [confluent kafka topic update](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_update.html) command. ```text confluent kafka topic update myTopic2 --config num.partitions=8 ``` The output should resemble: ```text Updated the following configuration values for topic "myTopic2": Name | Value | -----------------+-------- num.partitions | 6 | ``` ## Step 4: Produce and consume 1. Produce messages to a topic using the [confluent kafka topic produce](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_produce.html) command. ```text confluent kafka topic produce myTopic3 ``` Type your messages at the prompt, and press Return after each one. Your command window should resemble the following: ```text confluent kafka topic produce myTopic3 Starting Kafka Producer. ^C or ^D to exit hello cool topic did you get this message? first second third yes! I love this cool topic ``` If you want to stay on a single screen, type `^C` to exit the producer, and then run the consumer with `-b` (or `--from-beginning`) as shown in the next step. 2. Consume messages from a topic using the [confluent kafka topic consume](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/topic/confluent_kafka_topic_consume.html) command. ```text confluent kafka topic consume myTopic3 --from-beginning ``` Your output should resemble the following: ```text confluent kafka topic consume myTopic3 --from-beginning Starting Kafka Consumer. ^C or ^D to exit second did you get this message? first third cool topic hello yes! I love this cool topic ``` ## Step 5: Create service accounts and API keys In [Step 2](#ccloud-config-cli-cluster-kafka) you added an API key for your [user account](../authenticate/user-identities/user-accounts/manage-local-user-accounts.md#local-user-username-password), which has `super.user` privileges. This step describes how to create a [service account](../authenticate/workload-identities/service-accounts/overview.md#service-accounts) so that you can grant application access with limited permissions. 1. Create a service account named `dev-apps` using the [confluent iam service-account create](https://docs.confluent.io/confluent-cli/current/command-reference/iam/service-account/confluent_iam_service-account_create.html) command. ```text confluent iam service-account create "dev-apps" \ --description "Service account for dev apps" ``` The output should resemble: ```text +----------------+----------------------------------+ | ID | sa-1a2b3c | | Name | dev-apps | | Description | Service account for dev apps | +----------------+----------------------------------+ ``` Note the `ID` associated to this service account, in this case `sa-1a2b3c`. 2. Create an API key and API secret for this service account using the [confluent api-key create](https://docs.confluent.io/confluent-cli/current/command-reference/api-key/confluent_api-key_create.html) command. For the `resource` value, use the cluster ID, which is available from the output of `confluent kafka cluster list`. ```text confluent api-key create --service-account sa-1a2b3c --resource lkc-4xrp1 ``` 3. Take note of the API key and API secret — this is the only time you will be able to see the API secret. 4. Client applications that will connect to this cluster will need to configure at least these three identifying parameters: - API key: available when you initially create the API key pair - API secret: available when you initially create the API key pair - `bootstrap.servers`: set to the `Endpoint` in the output of `confluent kafka cluster describe` ## Step 6: Manage access with ACLs 1. Grant the `dev-apps` service account the ability to produce to topics using the [confluent kafka acl create](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/acl/confluent_kafka_acl_create.html) command. ```text confluent kafka acl create --allow --service-account sa-1a2b3c --operations write --topic myTopic2 Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+---------------+--------------- User:sa-1a2b3c | ALLOW | WRITE | TOPIC | myTopic2 | LITERAL ``` 2. If the service also needs to create topics, grant the `dev-apps` service account the ability to create new topics. ```text confluent kafka acl create --allow --service-account sa-1a2b3c --operations create --topic "*" Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+---------------+--------------- User:sa-1a2b3c | ALLOW | CREATE | TOPIC | * | LITERAL ``` 3. Grant the `dev-apps` service account the ability to consume from a particular topic using the [confluent kafka acl create](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/acl/confluent_kafka_acl_create.html) command. Note that it requires two commands: one to specify the topic and one to specify the consumer group. ```text confluent kafka acl create --allow --service-account sa-1a2b3c --operations read --topic myTopic2 Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+---------------+--------------- User:sa-1a2b3c | ALLOW | READ | TOPIC | myTopic2 | LITERAL confluent kafka acl create --allow --service-account sa-1a2b3c --operations read --consumer-group java_example_group_1 Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+----------------------+--------------- User:sa-1a2b3c | ALLOW | READ | GROUP | java_example_group_1 | LITERAL ``` 4. List all ACLs for the `dev-apps` service account using the [confluent kafka acl list](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/acl/confluent_kafka_acl_list.html) command. ```text confluent kafka acl list --service-account sa-1a2b3c ``` The output should resemble: ```text Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+----------------------+--------------- User:sa-1a2b3c | ALLOW | READ | TOPIC | * | LITERAL User:sa-1a2b3c | ALLOW | READ | TOPIC | myTopic2 | LITERAL User:sa-1a2b3c | ALLOW | WRITE | TOPIC | myTopic2 | LITERAL User:sa-1a2b3c | ALLOW | READ | GROUP | java_example_group_1 | LITERAL ``` 5. You can add ACLs on prefixed resource patterns. For example, you can add an ACL for any topic whose name starts with `demo`. ```text confluent kafka acl create --allow --service-account sa-1a2b3c --operations write --topic demo --prefix Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+---------------+--------------- User:sa-1a2b3c | ALLOW | WRITE | TOPIC | demo | PREFIXED ``` 6. You can add ACLs using a wildcard which matches any name for that resource. For example, you can add an ACL to allow a topic of any name. ```text confluent kafka acl create --allow --service-account sa-1a2b3c --operations write --topic "*" Principal | Permission | Operation | Resource Type | Resource Name | Pattern Type -----------------+------------+-----------+---------------+---------------+--------------- User:sa-1a2b3c | ALLOW | WRITE | TOPIC | * | LITERAL ``` Linux and macOS users can use either double or single quotes. Windows users must use double quotes around the wildcard character. 7. Remove an ACL from the `dev-apps` service account using the [confluent kafka acl delete](https://docs.confluent.io/confluent-cli/current/command-reference/kafka/acl/confluent_kafka_acl_delete.html) command. ```text confluent kafka acl delete --allow --service-account sa-1a2b3c --operations write --topic myTopic2 Deleted 1 ACLs. ``` ## Step 7: Log out Log out using the [confluent logout](https://docs.confluent.io/confluent-cli/current/command-reference/confluent_logout.html) command. ```text confluent logout You are now logged out. ```