# Manage Stream Governance Packages in Confluent Cloud Stream Governance offers two package types in Confluent Cloud, Essentials and Advanced, that determine the features, limits, and billing for an environment. This page describes each package and how to upgrade or delete them. ## Overview Confluent offers two types of governance packages in Confluent Cloud that are applied to your cloud environments: Essentials and Advanced. The governance package type you choose determines the features, capabilities, limits, and billing for the governance package. Use the information in this topic to find the governance package with the features and capabilities that’s right for you: - Use the **Essentials** package to help you get started with the governance fundamentals. - Use the **Advanced** package for enterprise grade data governance in production workloads. The table below offers a high-level comparison of features across the governance package types. | Feature | Essentials | Advanced | |---------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|------------------------| | [Schema Registry](../sr/index.md#schemaregistry-intro) | ✔ (Yes) | ✔ (Yes) | | Schema Registry SLA | 99.5% | 99.99% | | Schema Registry calls per second | Read: 75
Write: 25 | Read: 75
Write: 25 | | [Data rules](../sr/fundamentals/data-contracts.md#sr-data-contracts) | No | ✔ (Yes) | | Number of schemas included [1](#f1) | 100 [2](#f2) | 20,000 | | Number of [exporters](../sr/schema-linking.md#schema-exporters) included | 10 | 100 | | [Stream catalog tags](stream-catalog.md#data-discovery-tagging) | ✔ (Yes) | ✔ (Yes) | | [Stream catalog business metadata](stream-catalog.md#data-discovery-business-metadata) | (No) | ✔ (Yes) | | [Stream catalog REST API](stream-catalog-rest-apis.md#stream-catalog-rest-apis) | ✔ (Yes) | ✔ (Yes) | | [Stream catalog GraphQL API](graphql.md#catalog-graphql) | (No) | ✔ (Yes) | | [Data portal](data-portal.md#stream-gov-data-portal) (powered by the Stream Catalog) | ✔ (Yes) | ✔ (Yes) | | [Stream lineage](stream-lineage.md#cloud-stream-lineage) ([point in time](stream-lineage.md#stream-lineage-point-in-time): last 10 minutes) | ✔ (Yes) | ✔ (Yes) | | [Stream lineage](stream-lineage.md#cloud-stream-lineage) ([point in time](stream-lineage.md#stream-lineage-point-in-time): last 7 days) | No | ✔ (Yes) | | [AsyncAPI specification export and import](async-api.md#stream-gov-async-api) | ✔ (Yes) | ✔ (Yes) | * **[1]** Number of [data contracts](/cloud/current/sr/fundamentals/data-contracts.html) include all schema versions created by users and Confluent components like Kafka Connect and Apache Flink®. If a schema version is [soft-deleted](../sr/schemas-delete-cc.md#soft-delete-schema) but not [hard-deleted](../sr/schemas-delete-cc.md#hard-delete-schema), it counts towards the number of data contracts. * **[2]** Schemas (data contracts) created above this free limit are charged. For information on pricing, see [Stream Governance packages pricing](https://www.confluent.io/confluent-cloud/pricing/?platform=stream-governance). ## Create an environment with a package The Essentials package is automatically attached to your environment by default. The Stream Governance package is available in all Confluent Cloud regions. The Schema Registry cluster is instantiated in the same region when you create the first cluster in the environment. The Schema Registry cluster is resilient against zonal failures. You cannot change the Schema Registry region once it has been created. If you choose Essentials, you can upgrade to Advanced later, if needed. ### Confluent Cloud Console To create an environment with the Cloud Console: 1. Log in to Confluent Cloud at [https://confluent.cloud](https://confluent.cloud). 2. Select **Environments** on the left panel, choose **Add cloud environment**, provide an environment name in the dialog, and click **Create**. 3. Choose a [Stream Governance package](#stream-gov-packages) to enable Schema Registry, Stream Catalog, and Stream Lineage, either upgrade to Advanced or accept the Essentials package: - Upgrade to Stream Governance Advanced starting at $1/hour - Now, continue with Stream Governance Essentials for free ![Screenshot of choosing a Stream Governance package](images/dg-packages-choose.png) (Note that the full workflow for the Cloud Console is also provided in [Add a cloud environment](../get-started/schema-registry.md#cloud-sr-enable-zones) in the [Quick Start for Schema Management on Confluent Cloud](../get-started/schema-registry.md#cloud-sr-config).) ### Confluent CLI To create an environment with the Confluent CLI, use the [confluent environment create command](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_create.html). For example, to create an environment called “production” with the “ESSENTIALS” governance package: ```bash confluent environment create production ``` Your output should resemble: ```bash +---------------------------+----------------+ | Current | false | | ID | env-vnnpnz | | Name | production | | Stream Governance Package | ESSENTIALS | +---------------------------+----------------+ ``` Since “ESSENTIALS” is the default, it is not necessary to specify the package type. If you wanted to create the environment using the “ADVANCED” package, you would specify the package type: ```bash confluent environment create production --governance-package ADVANCED ``` **Note:** If you are using Confluent CLI v3 or lower, the governance package will not be created automatically. Confluent CLI v3 or lower will be removed starting March 1, 2025. You should [upgrade to Confluent CLI v4](https://docs.confluent.io/confluent-cli/current/migrate.html) at your earliest convenience. To learn more and read upgrade guidelines, see [Upgrade to SRCM v3 clusters and regions APIs (Deprecation of SRCM v2)](#update-srcm-api-to-v3). ### Confluent Cloud APIs To create an environment with the Confluent Cloud APIs, use the [Create an environment](https://docs.confluent.io/cloud/current/ccloud/list-org-v-2-environments/) API call, and indicate the environment name and governance package you want (ESSENTIALS or ADVANCED). You will need a [Confluent Cloud API key and secret](../security/authenticate/workload-identities/service-accounts/api-keys/manage-api-keys.md#create-cloud-api-key) to make these calls. In the examples, these have been stored shell environment variables. (Note that since you are working at the Confluent Cloud environment level, you do not need [Schema Registry API keys and secrets](../get-started/schema-registry.md#ccloud-sr-manage-api-keys-per-env) for these particular calls.) For example, to create an environment called “production” with the “ADVANCED” governance package: ```bash me$ curl --silent -u $APIKEY_CLOUD:$APISECRET_CLOUD --url https://api.confluent.cloud/org/v2/environments --header 'content-type: application/json' --data '{"display_name":"production","stream_governance_config"}' | jq . ``` Your output should resemble: ```bash { "api_version": "org/v2", "display_name": "production", "id": "env-2jjwwq", "kind": "Environment", "metadata": { "created_at": "2024-05-01T22:25:26.42437Z", "resource_name": "crn://confluent.cloud/organization=7ec51216-6104-4493-bf5b-50b11910f748/environment=env-2jjwwq", "self": "https://api.confluent.cloud/org/v2/environments/env-2jjwwq", "updated_at": "2024-05-01T22:25:26.42437Z" }, "stream_governance_config": { "package": "ESSENTIALS" . . . ``` Since “ESSENTIALS” is the default, it is not necessary to specify the package type. If you wanted to create the environment using the “ADVANCED” package, you would specify the package type: ```bash me$ curl --silent -u $APIKEY_CLOUD:$APISECRET_CLOUD --url https://api.confluent.cloud/org/v2/environments --header 'content-type: application/json' --data '{"display_name":"production","stream_governance_config":{"package":"ADVANCED"}}' | jq . ``` **Note:** If you are using srcm/v2, the governance package will not be created automatically. srcm/v2 is a deprecated API which will be removed starting March 1, 2025. You should upgrade to srcm/v3 at your earliest convenience. To learn more and read upgrade guidelines, see [Upgrade to SRCM v3 clusters and regions APIs (Deprecation of SRCM v2)](#update-srcm-api-to-v3). ### Terraform You can use the [Confluent Terraform provider](https://registry.terraform.io/providers/confluentinc/confluent/latest) to deploy and manage your Confluent Cloud infrastructure. The Confluent Terraform provider automates the workflow managing environments, Apache Kafka® clusters, Kafka topics, and other resources in Confluent Cloud. The specifics of setting up environments and clusters with Terraform require following the Terraform guides and examples to ensure all resources and dependencies are properly accounted for: - [Confluent Terraform Provider for Confluent Cloud](../clusters/terraform-provider.md#confluent-terraform-provider) - [Tutorials and sample projects](../clusters/terraform-provider.md#confluent-terraform-provider-tutorial) - [Sample Project for Confluent Terraform Provider](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/guides/sample-project) - [Using the Confluent Terraform provider](../clusters/terraform-provider.md#confluent-terraform-provider-using) **Note:** If you are using Terraform provider v1, the governance package will not be created automatically. Terraform v1 will be removed starting March 1, 2025. To learn more and read upgrade guidelines, see [Upgrade to SRCM v3 clusters and regions APIs (Deprecation of SRCM v2)](#update-srcm-api-to-v3). ### Cloud providers and region support Stream Governance Essentials and Advanced packages are available in all [Confluent Cloud regions](../get-started/regions.md#providers-regions). When you create your first Kafka cluster in the new Confluent Cloud environment, the Schema Registry cluster is instantiated in the same region. This Schema Registry region does not change, it persists regardless of clusters created in other regions or deleted. The Schema Registry region determines where Schema Registry and Stream Catalog services will run, and corresponding metadata is stored. ### Resilience in Schema Registry and Stream Governance - The Schema Registry cluster is a multi-zone cluster where all nodes are set to leader eligible. - All Schema Registry nodes are behind a Confluent Cloud Load Balancer. - The storage used by the Schema Registry cluster is also a multi-zone store, protecting schema data from zonal failures. ### Limitations - You can upgrade to an Advanced governance package, but it is not possible to downgrade from Advanced to Essentials. The only option is to delete the environment, and start over. - There are rate limits on API requests to search the Catalog API, which is not designed for continuous requests at upper rate limits. To learn more about alternative strategies to list subjects, schemas, and topics, see [Catalog API usage limitations and best practices](stream-catalog-rest-apis.md#catalog-api-usage-limits). ## View a cloud environment governance package ### Confluent Cloud Console To view the governance package associated with an environment on the [Confluent Cloud Console](https://confluent.cloud/), navigate to the environment, and view the package indicated under “Stream Governance package” on the right panel. ![Screenshot of choosing a Stream Governance package](images/dg-packages-sr-on-cloud-ui.png) ### Confluent CLI To view the governance package associated with an environment using the Confluent CLI, use either of the commands, [confluent environment list](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_list.html) or [confluent environment describe](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_describe.html): - Use [confluent environment list](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_list.html): ```bash me$ confluent environment list ``` Your output should resemble: ```bash Current | ID | Name | Stream Governance Package ----------+------------+---------------------+---------------------------- | env-1wrq56 | new-destination-env | ESSENTIALS | env-3w2p50 | new-source-env | ESSENTIALS * | env-97wz20 | my-stuff | ADVANCED ``` - Use [confluent environment describe](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_describe.html): ```bash me$ confluent environment describe env-97wz20 ``` Your output should resemble: ```bash +---------------------------+------------+ | Current | true | | ID | env-97wz20 | | Name | my-stuff | | Stream Governance Package | ADVANCED | +---------------------------+------------+ ``` ### Confluent Cloud APIs To view the governance package associated with an environment with the Confluent Cloud REST APIs, see [List environments](https://docs.confluent.io/cloud/current/ccloud/list-org-v-2-environments/) or [Read an environment](https://docs.confluent.io/cloud/current/ccloud/get-org-v-2-environment/) in the Confluent Cloud API reference. You will need a [Confluent Cloud API key and secret](../security/authenticate/workload-identities/service-accounts/api-keys/manage-api-keys.md#create-cloud-api-key) to make these calls. In the examples, these have been stored shell environment variables. (Note that since you are working at the Confluent Cloud environment level, you do not need [Schema Registry API keys and secrets](../get-started/schema-registry.md#ccloud-sr-manage-api-keys-per-env) for these particular calls.) For example, use [List environments](https://docs.confluent.io/cloud/current/ccloud/list-org-v-2-environments/): ```bash curl --silent -u $APIKEY_CLOUD:$APISECRET_CLOUD --request GET --url https://api.confluent.cloud/org/v2/environments | jq . ``` Your output should resemble the following. Note that listing environments shows all environment IDs: ```bash "api_version": "org/v2", "data": [ { "api_version": "org/v2", "display_name": "my-stuff", "id": "env-97wz20", "kind": "Environment", "metadata": { "created_at": "2023-02-21T21:03:04.884544Z", "resource_name": "crn://confluent.cloud/organization=7ec51216-6104-4493-bf5b-50b11910f748/environment=env-97wz20", "self": "https://api.confluent.cloud/org/v2/environments/env-97wz20", "updated_at": "2024-04-29T18:35:54.69138Z" }, "stream_governance_config": { "package": "ADVANCED" } }, { "api_version": "org/v2", "display_name": "new-source-env", "id": "env-3w2p50", "kind": "Environment", "metadata": { "created_at": "2024-03-31T13:30:12.042014Z", "resource_name": "crn://confluent.cloud/organization=7ec51216-6104-4493-bf5b-50b11910f748/environment=env-3w2p50", "self": "https://api.confluent.cloud/org/v2/environments/env-3w2p50", "updated_at": "2024-03-31T13:30:12.042014Z" }, "stream_governance_config": { "package": "ES SENTIALS" } ... ``` Or, use [Read an environment](https://docs.confluent.io/cloud/current/ccloud/get-org-v-2-environment/) and provide the environment ID: ```bash curl --silent -u $APIKEY_CLOUD:$APISECRET_CLOUD --request GET --url https://api.confluent.cloud/org/v2/environments/env-97wz20 | jq . ``` Your output should resemble: ```bash "api_version": "org/v2", "display_name": "my-stuff", "id": "env-97wz20", "kind": "Environment", "metadata": { "created_at": "2023-02-21T21:03:04.884544Z", "resource_name": "crn://confluent.cloud/organization=7ec51216-6104-4493-bf5b-50b11910f748/environment=env-97wz20", "self": "https://api.confluent.cloud/org/v2/environments/env-97wz20", "updated_at": "2024-04-29T18:35:54.69138Z" }, "stream_governance_config": { "package": "ADVANCED" ``` ### Terraform > Once you have deployed your Confluent Cloud environments, governance packages, and clusters with the [Confluent Cloud Terraform](../clusters/terraform-provider.md#confluent-terraform-provider), you can view and manage them using Terraform tools. > To learn more, see [Schema management and Stream Governance](../clusters/terraform-provider.md#confluent-terraform-provider-resources-schema) under [Resources you can manage](../clusters/terraform-provider.md#confluent-terraform-provider-resources) in the Terraform documentation. ## Understand the total number of schemas in your environment An environment’s schema count is the total number of schema versions, including [soft-deleted](../sr/schemas-delete-cc.md#soft-delete-schema) schemas. For example, if the subject `customer-value` has four schema versions and two are soft-deleted, the schema count is still four. Keep the following in mind: - To permanently remove a schema, [hard delete](../sr/schemas-delete-cc.md#hard-delete-schema) it so that it isn’t included in the total schema count. - Schema versions created by Flink SQL, Connect, or ksqlDB count towards the schema count. - Internal schemas created by ksqlDB whose subject names start with `_confluent-ksql` don’t count towards the schema count, schema limit, or billing. To learn more about deleting schemas and schema limits, see [Delete Schemas and Manage Storage Space on Confluent Cloud](../sr/schemas-delete-cc.md#schemaregistry-deletion). ## Schema limits, included counts, and overrides Three distinct numbers govern how many schemas a Stream Governance environment can hold. These numbers affect billing, capacity planning, and whether enforcement occurs at the public threshold. | Term | What it means | |-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Included count | The number of schemas covered by the package price. Essentials
includes 100 schemas. Advanced includes 20,000. See the
[Manage Stream Governance Packages in Confluent Cloud](#stream-gov-packages) table for the current values. | | Default cap | The default enforcement threshold for all environments on a
package. For Advanced, the default cap matches the included count
of 20,000. Essentials doesn’t enforce a cap. Schemas above the
Essentials included count are billed per schema per hour instead. | | Effective limit | The enforced limit on a specific Schema Registry cluster. If Confluent
approves a higher limit for the environment, the effective limit
replaces the default cap for that cluster. | When an effective limit is in place, schema registration succeeds until the environment reaches the effective limit, even though the public default cap is lower. As a result, you might not see an enforcement error near the published 20,000 threshold. Above the effective limit, schema registration returns an `HTTP 422` error with the message `Schema Limit Exceeded`. Reduce the schema count by [hard deleting unused schemas](../sr/schemas-delete-cc.md#hard-delete-schema), or contact Confluent to request a higher limit. If your environment is approaching the Advanced included count of 20,000 schemas and needs more, contact [Confluent Support](https://support.confluent.io/hc/en-us) or your account team to request a higher limit. Provide the environment ID, the logical Schema Registry cluster ID (`lsrc-xxxxx`), the requested limit, and the expected growth timeline. ## Upgrade a cloud environment from Essentials to Advanced If you have started with the Essentials package, you can upgrade to the Advanced package anytime. If you started with the Advanced package, you cannot downgrade. Options are to start over with a new Confluent Cloud environment, or contact [Confluent Support](https://support.confluent.io/hc/en-us). ### Confluent Cloud Console To upgrade an environment from Essentials to Advanced on the [Confluent Cloud Console](https://confluent.cloud/), navigate to the environment, and choose **Upgrade now** under “Stream Governance package” on the right panel. Features that specifically require the Advanced Stream Governance package provide an upgrade option when you hover over the feature on the right panel. (For example, Business metadata requires the Advanced package.) ![image](images/dg-packages-upgrade.png) ### Confluent CLI To upgrade to an environment from Essentials to Advanced, use [confluent environment update](https://docs.confluent.io/confluent-cli/current/command-reference/environment/confluent_environment_update.html). For example, this command upgrades the “production” environment from the previous Confluent CLI example (environment ID `env-vnnpnz`) from Essentials to the Advanced package: ```bash confluent environment update env-vnnpnz --governance-package advanced ``` Your output should resemble: ```bash +---------------------------+----------------+ | Current | false | | ID | env-vnnpnz | | Name | production | | Stream Governance Package | ADVANCED | +---------------------------+----------------+ ``` ### Confluent Cloud APIs You will need a [Confluent Cloud API key and secret](../security/authenticate/workload-identities/service-accounts/api-keys/manage-api-keys.md#create-cloud-api-key) to make these calls. In the examples, these have been stored shell environment variables. (Note that since you are working at the Confluent Cloud environment level, you do not need [Schema Registry API keys and secrets](../get-started/schema-registry.md#ccloud-sr-manage-api-keys-per-env) for these particular calls.) For example, to upgrade the environment “production” from ESSENTIALS to ADVANCED: ```bash curl --silent -u $APIKEY_CLOUD:$APISECRET_CLOUD --request PATCH \ > --url 'https://api.confluent.cloud/org/v2/environments/{env-2jjwwq}' \ > --header 'content-type: application/json' \ > --data '{"display_name":"production","stream_governance_config":{"package":"ADVANCED"}}' ``` Your output should resemble the following, showing the associated governance package as upgraded to ADVANCED: ```bash { "api_version": "org/v2", "display_name": "production", "id": "env-2jjwwq", "kind": "Environment", "metadata": { "created_at": "2024-05-01T22:25:26.42437Z", "resource_name": "crn://confluent.cloud/organization=7ec51216-6104-4493-bf5b-50b11910f748/environment=env-2jjwwq", "self": "https://api.confluent.cloud/org/v2/environments/env-2jjwwq", "updated_at": "2024-05-01T22:45:17.119634Z" }, "stream_governance_config": { "package": "ADVANCED" } } ``` ### Terraform > Once you have deployed your Confluent Cloud environments, governance packages, and clusters with the [Confluent Cloud Terraform](../clusters/terraform-provider.md#confluent-terraform-provider), you can view and manage them using Terraform tools. > To learn more, see [Schema management and Stream Governance](../clusters/terraform-provider.md#confluent-terraform-provider-resources-schema) under [Resources you can manage](../clusters/terraform-provider.md#confluent-terraform-provider-resources) in the Terraform documentation. ## Remove an environment with a package You cannot delete a Stream Governance package nor the Schema Registry cluster within an environment. If you want to remove a governance package or Schema Registry cluster, the only way to do this is to delete the environment. If you would like to recreate the Schema Registry in another region within an environment, contact Confluent Support. You cannot delete a Schema Registry cluster directly. The Schema Registry cluster is auto-deleted if you delete an environment. ## Upgrade to SRCM v3 clusters and regions APIs (Deprecation of SRCM v2) Support for Schema Registry cluster management (SRCM) v2 regions API and v2 clusters API will end on May 3, 2025. Similarly, any related Confluent CLI commands and Confluent Terraform Provider resources and data sources that rely on this API will no longer be supported. You should upgrade from the SRCM v2 to v3 clusters API by May 3, 2025. With the new APIs, Schema Registry is automatically provisioned in the same region as the first Kafka cluster provisioned in the environment. To learn more about region support, see [Cloud providers and region support](#providers-regions-sr). - To upgrade the Confluent CLI, see [Update to Confluent CLI v4](https://docs.confluent.io/confluent-cli/current/migrate.html). - To upgrade Confluent Terraform, see the [Confluent Provider 2.0.0: Upgrade Guide at](https://registry.terraform.io/providers/confluentinc/confluent/latest/docs) (**Guides** > **Confluent Provider 2.0.0: Upgrade Guide**). - For `Get` and `List` API operations, upgrade to srcm/v3: - [Read a Cluster (Get)](https://docs.confluent.io/cloud/current/ccloud/get-srcm-v-3-cluster/) - [List of Clusters (List)](https://docs.confluent.io/cloud/current/ccloud/list-srcm-v-3-clusters/) - For `Create` and `Delete` operations, Schema Registry will automatically be provisioned after the first Kafka cluster in the environment, and deleted during environment deletion. These operations are no longer supported. - For `Update` operations, update the Stream Governance package using the [org/v2 Environments API](https://docs.confluent.io/cloud/current/ccloud/environments-org-v-2/) For more information, log into the Confluent Support Portal to refer to the article on [Removal of Schema Registry Cluster Management (srcm) version v2 API](https://support.confluent.io/hc/en-us/articles/31147729558676-Removal-of-Schema-Registry-Cluster-Management-srcm-version-v2-API). ## Related content - To learn more about this workflow, including choosing a Stream Governance package for an environment and creating a cluster, see [Quick Start for Schema Management on Confluent Cloud](../get-started/schema-registry.md#cloud-sr-config). - [Confluent Cloud pricing for Stream Governance](https://www.confluent.io/confluent-cloud/pricing/#stream-governance) gives details on pricing and limits.