Generate an API Key for Access in Confluent Cloud for Apache Flink¶
To manage Flink workloads programmatically in Confluent Cloud for Apache Flink®, you need an API key that’s specific to Flink. You can use the Confluent CLI, the Confluent Cloud APIs, the Confluent Terraform Provider, or the Cloud Console to create API keys.
Before you create an API key for Flink access, decide whether you want to create long-running statements. If you need long-running statements, you should use a service account and create an API key for it. If you only need to run interactive queries or run statements for a short time while developing queries, you can create an API key for your user account.
A Flink API key is scoped to an environment and region pair, for example,
env-abc123.aws.us-east-2
. The key enables creating, reading, updating, and
deleting Flink SQL statements.
To create an API key for Flink access by using the Confluent Cloud APIs or the Confluent Terraform Provider, you must first create a Cloud API key. This step is done automatically if you use the Confluent CLI to create an API key for Flink access.
Create a service account (optional)¶
If you need to create long-running Flink SQL statements, create a service account principal before you create a Flink API key.
Create a service account by using the Cloud Console or the CLI.
Assign the OrganizationAdmin role to the service account by following the steps in Add a role binding to a principal.
Store the service account ID in a convenient location, for example, in an environment variable:
export PRINCIPAL_ID="<service-account-id>"
Generate an API Key¶
You can use the Confluent Cloud APIs, the Confluent Terraform Provider, the Confluent CLI, or the Cloud Console to create an API key for Flink access. For more information, see Manage API Keys.
You can use the Cloud Console to generate an API key for Flink access.
Log in to the Confluent Cloud Console and navigate to the environment that hosts your data and compute pools.
Click Flink and in the Flink overview page, click API keys.
Click Add API Key to open the Create API key page.
Select either the My account tile to create an API key for your user account or the Service account tile to create an API key for a service account.
For production Flink deployments, select the Service account option, and click either Existing account or New account to assign the service account principal.
Click Next to open the Resource scope page.
Select the cloud provider and region for the API key. Ensure that you choose the same provider and region where your data and compute pools are located.
Click Next to open the API key detail page.
Enter a name and a description for the new API key. This step is optional.
Click Create API key. The API key download page opens.
Click Download API key and save the key to a secure location on your local machine.
Click Complete.
You can use the Confluent CLI to generate an API key for Flink access. For more information, see confluent api-key create .
Log in to Confluent Cloud:
confluent login
To see the available regions for Flink, run the following command:
confluent flink region list
Your output should resemble:
Current | Name | Cloud | Region ----------+--------------------------+-------+--------------- | Frankfurt (eu-central-1) | aws | eu-central-1 | Ireland (eu-west-1) | aws | eu-west-1 * | N. Virginia (us-east-1) | aws | us-east-1 | Ohio (us-east-2) | aws | us-east-2
Run the following command to create an API key. Enure that the environment variables are set to your values.
# Example values for environment variables. export CLOUD_PROVIDER=aws export CLOUD_REGION=us-east-1 export ENV_ID=env-a12b34 # Generate the API key and secret. confluent api-key create \ --resource flink \ --cloud ${CLOUD_PROVIDER} \ --region ${CLOUD_REGION} \ --environment ${ENV_ID}
Your output should resemble:
It may take a couple of minutes for the API key to be ready. Save the API key and secret. The secret is not retrievable later. +------------+------------------------------------------------------------------+ | API Key | ABC1DDN2BNASQVRU | | API Secret | B0b+xCoSPY2pSNETeuyrziWmsPmou0WP9rH0Nxed4y4/msnESzjj7kBrRWGOMu1a | +------------+------------------------------------------------------------------+
- If the environment, cloud, and region flags are set globally, you
can create an API key by running
confluent api-key create --resource flink
. For more information, see Manage API Keys in Confluent Cloud. - To create an API key for an existing service account, provide the
--service-account <sa-a1b2c3>
option. This enables submitting long-running Flink SQL statements.
- If the environment, cloud, and region flags are set globally, you
can create an API key by running
To create an API key for Flink access by using the Confluent Cloud APIs, you must first create a Cloud API key.
To generate the Flink key, you send your Cloud API key and secret in the request header, encoded as a base64 string.
Create a Cloud API key for the principal, which is either a service account or your user account. For more information, see Add an API key.
Assign the Cloud API key and secret to environment variables that you use in your REST API requests.
export CLOUD_API_KEY="<cloud-api-key>" export CLOUD_API_SECRET="<cloud-api-secret>" export PRINCIPAL_ID="<service-account-id>" # or "<user-account-id>" export ENV_REGION_ID="<environment-id>.<cloud-region>" # example: "env-z3y2x1.aws.us-east-1"
The ENV_REGION_ID variable is a concatenation of your environment ID and the cloud provider region of your Kafka cluster, separated by a
.
character. To see the available regions, run theconfluent flink region list
command.Run the following command to send a POST request to the
api-keys
endpoint. The REST API uses basic authentication, which means that you provide a base64-encoded string made from your Cloud API key and secret in the request header.curl --request POST \ --url 'https://api.confluent.cloud/iam/v2/api-keys' \ --header "Authorization: Basic $(echo -n "${CLOUD_API_KEY}:${CLOUD_API_SECRET}" | base64 -w 0)" \ --header 'content-type: application/json' \ --data "{"spec":{"display_name":"flinkapikey","owner":{"id":"${PRINCIPAL_ID}"},"resource":{"api_version":"fcpm/v2","id":"${ENV_REGION_ID}"}}}"
Your output should resemble:
{ "api_version": "iam/v2", "id": "KJDYFDMBOBDNQEIU", "kind": "ApiKey", "metadata": { "created_at": "2023-12-15T23:10:20.406556Z", "resource_name": "crn://api.confluent.cloud/organization=b0b21724-4586-4a07-b787-d0bb5aacbf87/user=u-lq1dr3/api-key=KJDYFDMBOBDNQEIU", "self": "https://api.confluent.cloud/iam/v2/api-keys/KJDYFDMBOBDNQEIU", "updated_at": "2023-12-15T23:10:20.406556Z" }, "spec": { "description": "", "display_name": "flinkapikey", "owner": { "api_version": "iam/v2", "id": "u-lq1dr3", "kind": "User", "related": "https://api.confluent.cloud/iam/v2/users/u-lq2dr7", "resource_name": "crn://api.confluent.cloud/organization=b0b21724-4586-4a07-b787-d0bb5aacbf87/user=u-lq2dr7" }, "resource": { "api_version": "fcpm/v2", "id": "env-z3q9rd.aws.us-east-1", "kind": "Region", "related": "https://api.confluent.cloud/fcpm/v2/regions?cloud=aws", "resource_name": "crn://api.confluent.cloud/organization=b0b21724-4586-4a07-b787-d0bb5aacbf87/environment=env-z3q9rd/flink-region=aws.us-east-1" }, "secret": "B0BYFzyd0bb5Q58ZZJJYV52mbwDDHnZx21f0gOTz2k6Qv2V9I4KraVztwFOlQx6z" } }
You can use the Confluent Terraform Provider to generate an API key for Flink access.
Follow the steps in Sample Project for Confluent Terraform Provider and use the configuration shown in Example Flink API Key.
When your API key and secret are generated, save them in environment variables for later use.
export FLINK_API_KEY="<flink-api-key>"
export FLINK_API_SECRET="<flink-api-secret>"
You can manage the API key by using the Confluent CLI commands. For more information, see confluent api-key . Also, you can use the REST API and Cloud Console.