Consume Stream Shares from Confluent Cloud¶

Accessing data that is shared with you through Stream Sharing requires that you have or create a Confluent Cloud account.

View stream shares¶

You can view the data that has been shared with you in the Cloud Console or you can retrieve a list of data shared with you using the Confluent CLI or by making a REST API call.

You can view unredeemed shares from multiple organizations, but after you redeem a share from a particular organization, you can only view that share when you are logged into the organization from which the stream was redeemed from.

To access the Stream shares page:

Log in to your Confluent Cloud account.
From the navigation menu, select Stream shares.

The Stream shares page opens.
View the Data shared with you section. This section contains tiles for each topic that has been shared with you, and if not yet redeemed, how many days left to redeem the invitation for the shared data. Each tile provides the organization name and account for the provider that shared the data and how many days are left to access the shared data. Click through a tile to see an overview of the topic metadata.

Note

To use the CLI for Stream Sharing, you must have CLI v3. For more information, see: Migrate to Confluent CLI v3

You can retrieve a list of data shared with you using the Confluent CLI.

confluent stream-share consumer share list
Copy

The response will resemble:

    ID    | Cloud Name     | Status | Shared Resource ID |       Invite Expires At
----------+----------------+--------+--------------------+---------------------------------
 ss-123x4 | Jane Doe       | ACTIVE |    sr-12abc        | 2022-09-05 22:28:19.924174
          |                |        |                    | +0000 UTC
Copy

You can retrieve a list of data that has been shared with you by making a REST API call.

Make a GET call to the consumer-shared-resources endpoint.

GET /cdx/v1/consumer-shared-resources
Copy

The response will resemble:

{
   "api_version": "v1",
   "data": [
      {
         "api_version": "v1",
         "cloud": "GCP",
         "display_name": "Users",
         "id": "sr-12abc",
         "kind": "ConsumerSharedResource",
         "labels": [
            "our users"
         ],
         "logo_url": "/api/cdx/v1/consumer-shared-resources/sr-12abc/images/logo",
         "network_connection_types": [],
         "organization_contact": "test@confluent.io",
         "organization_details": "Confluent Test Org",
         "organization_name": "Confluent",
         "schemas": null
      }
   ],
   "kind": "ConsumerSharedResourceList",
   "metadata": {}
}
Copy

Access your stream shares¶

You should receive an email if data has been shared with you.

Considerations:

Check your junk or spam folders if you know you should have received an email for shared data.
If the shared topic is on a cluster that uses a private endpoint, you must set up network connectivity to the Confluent Cloud network of the cluster before you can access the topic. For more information, see Use Google Cloud Private Service Connect with Confluent Cloud, Use AWS PrivateLink with Confluent Cloud, or Use Azure Private Link with Confluent Cloud.

Tip

To find the network details required to setup network connectivity within Cloud Console, click Access from the stream share email.
By default, you have one week to access the link to shared data or redeem the access token, although the provider can invalidate the link at any time. If you cannot find the email, or if you experience issues while trying to access shared data, you should contact the data provider.
You can only redeem a stream share once. After you receive the credentials for the stream share, you cannot redeem that same stream share again. Once redeemed, the stream share becomes active and remains so indefinitely until you delete it or the provider revokes access.
You will receive an email if a topic that was shared with you is deleted or if the share is removed.

Prerequisites

You must already have, or you must create, a Confluent Cloud account associated with the email address that the data was shared to.
If the provider of the stream is using a private endpoint, you must have the ID of your cloud provider account to access the stream.

To access the shared data:

Click the Access button in the email. The Confluent Cloud Console will open in a browser.
Using the same email address where the data was shared, log in to your Confluent Cloud account.
In the Read and process shared data box, click the Access button. If the provider of the stream is using a private endpoint, you must enter the ID of your cloud provider account and click Next.
Select the programming language you will use to create an Kafka client and click Next.
Click Generate & download credentials.

Your credentials are generated and added to the configuration file shown in the browser.
Click Copy.

Save the configuration file.

The file will resemble the following, containing the bootstrap server and username and password required to access the shared data.

# Topic: Users
# Consumer group ID requires a prefix of stream-share.ss-3vpvd
# Example: group.id = stream-share.ss-3vpvd.[Add optional consumer group suffix]

Required connection configs for Kafka producer, consumer, and admin
bootstrap.servers=SASL_SSL://pkc-ABC123.us-central1.gcp.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='ABCDEFGHIJKLMNOP' password='ABCD+hzdHX2aTWVT5+OcD0CPhzMPl+ScjpjBdPN8b1RZu2h9efHeZULkLACpVmko';
sasl.mechanism=PLAIN
# Required for correctness in Apache Kafka clients prior to 2.6
client.dns.lookup=use_all_dns_ips

# Best practice for higher availability in Apache Kafka clients prior to 3.0
session.timeout.ms=45000

# Best practice for Kafka producer to prevent data loss
acks=all

# Required connection configs for Confluent Cloud Schema Registry
schema.registry.url=https://psrc-1a2xyz.us-central1.gcp.confluent.cloud
basic.auth.credentials.source=USER_INFO
basic.auth.user.info=EXAMPLE6THZZR5GY:g13tSFeALOM2nPpaF+V368MG+JNPxOoUbDKlFTPB2kRabTgf3vO9jtvClexample
Copy

Prerequisite:

To accept an invitation to shared data with the Confluent CLI, you will need the access token provided in the emailed invitation.

Use the token with one of the following commands. If the stream provider uses a private endpoint, include the ID of your cloud provider account in the request. Otherwise, use the public internet.

confluent stream-share consumer redeem <token-provided-in-email>
Copy

confluent stream-share consumer redeem <token-provided-in-email> --aws-account <AWS-ID>
Copy

confluent stream-share consumer redeem <token-provided-in-email> --azure-subscription <Azure-ID>
Copy

confluent stream-share consumer redeem <token-provided-in-email> --gcp-project <GCP-ID>
Copy

The response will resemble the following, containing the bootstrap server and username and password required to access the shared data.

+----------------------------+-------------------------------------------------------------------+
| ID                         | ss-3vpvd                                                          |
| API Key                    | ABCDEFGHIJKLMNOP                                                  |
| Secret                     | ABCD+hzdHX2aTWVT5+OcD0CPhzMPl+ScjpjBdPN8b1RZu2h9efHeZULkLACpVmko  |
| Kafka Bootstrap URL        | SASL_SSL://pkc-abc123.us-central1.gcp.confluent.cloud:9092        |
| Resources                  | [Topic:Users                                                      |
|                            | Group:stream-share-ss-3vpvd]                                      |
+----------------------------+-------------------------------------------------------------------+
Copy

Prerequisite:

To accept an invitation to shared data with the REST APIs, you will need the access token provided in the emailed invitation.

Once you have the token, make a POST call to the https://api.confluent.cloud/cdx/v1/shared-tokens URI specifying the token. If the stream provider uses a private endpoint, include the ID of your cloud provider account in the request. Otherwise, use the public internet.

POST /cdx/v1/shared-tokens:redeem

{
   "token": "<token-provided-in-email>"
}
Copy

POST /cdx/v1/shared-tokens:redeem

{
   "token": "<token-provided-in-email>",
   "aws_account": "000000000000"
}
Copy

POST /cdx/v1/shared-tokens:redeem

{
   "token": "<token-provided-in-email>",
   "azure_subscription": "00000000-0000-0000-0000-000000000000"
}
Copy

POST /cdx/v1/shared-tokens:redeem

{
   "token": "<token-provided-in-email>",
   "gcp_project": "string"
}
Copy

The response will resemble the following, containing the bootstrap server and API key and secret required to access the shared data.

{
   "api_key": "ABCDEFGHIJKLMNOP",
   "api_version": "v1",
   "id": "ss-ex123",
   "kafka_bootstrap_url": "SASL://pkc-xxxxx.us-west-2.aws.confluent.cloud:9092",
   "kind": "RedeemedShare",
   "resources": [
      {
         "kind": "Topic",
         "topic": "user"
      },
      {
         "group_prefix": "stream-share.ss-ex123",
         "kind": "Group"
      }
   ],
   "schema_registry_api_key": "abcdefghijklmnop",
   "schema_registry_api_secret": "abcd+examplehzdHX2aTWVT5+OcD0CPhzMPl+ScjpjBdPN8b1RZu2h9efHeZULkLA",
   "schema_registry_url": "https://psrc-xxxxx.us-west-2.aws.confluent.cloud",
   "secret": "ABCD+hzdHX2aTWVT5+OcD0CPhzMPl+ScjpjBdPN8b1RZu2h9efHeZULkLACpVmko"
}
Copy

Create a Kafka client¶

After you have the configuration file, you can write a Kafka client application to connect to the shared data. For more information, see Configure clients from the Confluent CLI.