.. _schemaregistry_api: |sr| API Reference ================== Overview -------- This section provides a detailed reference for the |sr| API. The best way to test these is to use `curl `__. For examples of using ``curl`` to test these APIs, see :ref:`schemaregistry_using`. You can also see curl API calls used in the :ref:`Schema Registry Tutorial ` and to test drive different schema formats in the deep dive sections on :ref:`serializer_and_formatter`. Compatibility ^^^^^^^^^^^^^ The |sr| server can enforce certain compatibility rules when new schemas are registered in a subject. These are the compatibility types: .. include:: ../includes/compatibility_list.rst We recommend keeping the default backward compatibility since it's common to have all data loaded into Hadoop. For more details on schema resolution, see :ref:`schema_evolution_and_compatibility`. Content Types ^^^^^^^^^^^^^ The |sr| REST server uses content types for both requests and responses to indicate the serialization format of the data as well as the version of the API being used. Currently, the only serialization format supported is JSON and the only version of the API is ``v1``. However, to remain compatible with future versions, you *should* specify preferred content types in requests and check the content types of responses. The preferred format for content types is ``application/vnd.schemaregistry.v1+json``, where ``v1`` is the API version and ``json`` is the serialization format. However, other less specific content types are permitted, including ``application/vnd.schemaregistry+json`` to indicate no specific API version should be used (the most recent stable version will be used), ``application/json``, and ``application/octet-stream``. The latter two are only supported for compatibility and ease of use. Your requests *should* specify the most specific format and version information possible via the HTTP ``Accept`` header:: Accept: application/vnd.schemaregistry.v1+json The server also supports content negotiation, so you may include multiple, weighted preferences:: Accept: application/vnd.schemaregistry.v1+json; q=0.9, application/json; q=0.5 which can be useful when, for example, a new version of the API is preferred but you cannot be certain it is available yet. Errors ^^^^^^ All API endpoints use a standard error message format for any requests that return an HTTP status indicating an error (any 400 or 500 statuses). For example, a request entity that omits a required field may generate the following response: .. sourcecode:: http HTTP/1.1 422 Unprocessable Entity Content-Type: application/vnd.schemaregistry.v1+json { "error_code": 422, "message": "schema may not be empty" } Although it is good practice to check the status code, you may safely parse the response of any non-DELETE API calls and check for the presence of an ``error_code`` field to detect errors. Example Requests Format and Valid JSON ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The example requests below are formatted for readability. Therefore, multi-line example requests include returns (newlines). To use a multi-line example as a copy-paste test of an API, remove the newlines before running it so as not to trigger errors for invalid JSON. For example, a direct copy-and-paste of this request will produce errors because it includes newlines to help with readability: .. sourcecode:: http { "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" } This is the same request as that above with the newlines removed. It should run properly as a copy-and-paste example. .. sourcecode:: http {"schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" } Schemas ------- .. http:get:: /schemas/ids/{int: id} Get the schema string identified by the input ID. :param int id: the globally unique identifier of the schema :>json string schema: Schema string identified by the ID :statuscode 404: * Error code 40403 -- Schema not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /schemas/ids/1 HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { "schema": "{\"type\": \"string\"}" } .. http:get:: /schemas/types/ Get the schema types that are registered with |sr|. :>json string schema: Schema types currently available on |sr|. :statuscode 404: * Error code 40403 -- Schema not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /schemas/types HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { ["JSON", "PROTOBUF", "AVRO"] } .. http:get:: /schemas/ids/{int: id}/versions Get the subject-version pairs identified by the input ID. :param int id: the globally unique identifier of the schema :>jsonarr string subject: Name of the subject :>jsonarr int version: Version of the returned schema :statuscode 404: * Error code 40403 -- Schema not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /schemas/ids/1/versions HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json [{"subject":"test-subject1","version":1}] .. _schemaregistry-api-subjects: Subjects -------- The subjects resource provides a list of all registered subjects in your |sr|. A subject refers to the name under which the schema is registered. If you are using |sr| for Kafka, then a subject refers to either a "-key" or "-value" depending on whether you are registering the key schema for that topic or the value schema. .. tip:: Currently, there is no built-in lookup API to find subjects for a given ID. See :ref:`list-subjects-for-given-id` for a two-step process to use instead. .. http:get:: /subjects Get a list of registered subjects. :>jsonarr string name: Subject :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /subjects HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json ["subject1", "subject2"] .. http:get:: /subjects/(string: subject)/versions Get a list of versions registered under the specified subject. :param string subject: the name of the subject :>jsonarr int version: version of the schema registered under this subject :statuscode 404: * Error code 40401 -- Subject not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /subjects/test/versions HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json [ 1, 2, 3, 4 ] .. http:delete:: /subjects/(string: subject) Deletes the specified subject and its associated compatibility level if registered. It is recommended to use this API only when a topic needs to be recycled or in a development environment. See also, :ref:`schemaregistry_deletion`. :param string subject: the name of the subject :param boolean permanent: Add ``?permanent=true`` at the end of this request to specify a hard delete of the subject, which removes all associated metadata including the schema ID. The default is ``false``. If the flag is not included, a soft delete is performed. You must perform a soft delete first, then the hard delete. This flag is now supported on both |cp| and |ccloud|, which sets limits on the number of schema versions supported in the registry. A hard delete frees up space in a way that a soft delete does not. See also, :ref:`hard-delete-schema` in :ref:`schemaregistry_deletion`, and :ref:`sr-in-cloud-manage-space`. :>jsonarr int version: version of the schema deleted under this subject :statuscode 404: * Error code 40401 -- Subject not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http DELETE /subjects/test HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json [ 1, 2, 3, 4 ] .. http:get:: /subjects/(string: subject)/versions/(versionId: version) Get a specific version of the schema registered under this subject :param string subject: Name of the subject :param versionId version: Version of the schema to be returned. Valid values for versionId are between [1,2^31-1] or the string "latest", which returns the last registered schema under the specified subject. Note that there may be a new latest schema that gets registered right after this request is served. :>json string subject: Name of the subject that this schema is registered under :>json int id: Globally unique identifier of the schema :>json int version: Version of the returned schema :>json string schema: The schema string :statuscode 404: * Error code 40401 -- Subject not found * Error code 40402 -- Version not found :statuscode 422: * Error code 42202 -- Invalid version :statuscode 500: * Error code 50001 -- Error in the backend data store **Example request**: .. sourcecode:: http GET /subjects/test/versions/1 HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { "name": "test", "version": 1, "schema": "{\"type\": \"string\"}" } .. http:get:: /subjects/(string: subject)/versions/(versionId: version)/schema Get the schema for the specified version of this subject. The unescaped schema only is returned. :param string subject: Name of the subject :param versionId version: Version of the schema to be returned. Valid values for versionId are between [1,2^31-1] or the string "latest", which returns the last registered schema under the specified subject. Note that there may be a new latest schema that gets registered right after this request is served. :>json string schema: The schema string (unescaped) :statuscode 404: * Error code 40401 -- Subject not found * Error code 40402 -- Version not found :statuscode 422: * Error code 42202 -- Invalid version :statuscode 500: * Error code 50001 -- Error in the backend data store **Example request**: .. sourcecode:: http GET /subjects/test/versions/1/schema HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json {"type": "string"} .. _api-example-avro-schema-references: .. http:post:: /subjects/(string: subject)/versions Register a new schema under the specified subject. (Essentially, create a new schema.) If successfully registered, this returns the unique identifier of this schema in the registry. The returned identifier should be used to retrieve this schema from the schemas resource and is different from the schema's version which is associated with the subject. If the same schema is registered under a different subject, the same identifier will be returned. However, the version of the schema may be different under different subjects. A schema should be compatible with the previously registered schema or schemas (if there are any) as per the configured compatibility level. If a compatibility level has been set on a subject, the configured compatibility level can be obtained by issuing a ``GET http:get:: /config/(string: subject)``. If this returns an error, it means that no `subject-specific` compatibility level is set for the subject. In that case, use ``GET http:get:: /config`` to find the `global` compatibility level, which applies to all subjects. (Subject-specific compatibility, when configured, overrides global.) For examples, see :ref:`set-compatibility-on-subject` and :ref:`get-compatibility-on-subject`. When there are multiple instances of |sr| running in the same cluster, the schema registration request will be forwarded to one of the instances designated as the primary. If the primary is not available, the client will get an error code indicating that the forwarding has failed. If no ``schemaType`` is supplied, ``schemaType`` is assumed to be AVRO. :param string subject: Subject under which the schema will be registered :reqjson schema: The schema string :reqjson schemaType: Defines the schema format: AVRO (default), PROTOBUF, JSON (Optional) :reqjson references: Specifies the names of referenced schemas (Optional). To learn more, see :ref:`referenced-schemas`. :>json string subject: Name of the subject that this schema is registered under :>json int id: Globally unique identifier of the schema :>json int version: Version of the returned schema :>json string schema: The schema string :statuscode 409: Incompatible schema :statuscode 422: * Error code 42201 -- Invalid schema :statuscode 500: * Error code 50001 -- Error in the backend data store * Error code 50002 -- Operation timed out * Error code 50003 -- Error while forwarding the request to the primary **Example request**: .. sourcecode:: http POST /subjects/test/versions HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json { "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"com.acme.Referenced\" } ] }", "schemaType": "AVRO", "references": [ { "name": "com.acme.Referenced", "subject": "childSubject", "version": 1 } ] } **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json {"id":1} .. http:post:: /subjects/(string: subject) Check if a schema has already been registered under the specified subject. If so, this returns the schema string along with its globally unique identifier, its version under this subject and the subject name. :param string subject: Subject under which the schema will be registered :reqjson schema: The schema string :reqjson schemaType: Defines the schema format: AVRO (default), PROTOBUF, JSONSCHEMA (Optional) :reqjson references: Specifies the names of referenced schemas (Optional). To learn more, see :ref:`referenced-schemas`. :>json string subject: Name of the subject that this schema is registered under :>json int id: Globally unique identifier of the schema :>json int version: Version of the returned schema :>json string schema: The schema string :statuscode 404: * Error code 40401 -- Subject not found * Error code 40403 -- Schema not found :statuscode 500: Internal server error **Example request**: .. sourcecode:: http POST /subjects/test HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json { "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" } **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { "subject": "test", "id": 1 "version": 3 "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" } .. http:delete:: /subjects/(string: subject)/versions/(versionId: version) Deletes a specific version of the schema registered under this subject. Unless you perform a hard delete (with ``?permanent=true`` as noted below), this only deletes the version, leaving the schema ID intact and making it still possible to decode data using the schema ID. This API is recommended to be used only in development environments or under extreme circumstances where-in, its required to delete a previously registered schema for compatibility purposes or re-register previously registered schema. :param string subject: Name of the subject :param versionId version: Version of the schema to be deleted. Valid values for versionId are between [1,2^31-1] or the string "latest", which deletes the last registered schema under the specified subject. Note that there may be a new latest schema that gets registered right after this request is served. For hard-delete of a schema version, you must provide a version number as input; `not` ``version:latest``, which results in a soft delete only, even with the ``?permanent=true`` flag appended. See also, :ref:`schemaregistry_deletion`. :param boolean permanent: Add ``?permanent=true`` at the end of this request to specify a hard delete for a specific version of the subject, which removes all associated metadata including the schema ID. The default is ``false``. If the flag is not included, a soft delete is performed. You must perform a soft delete first, then the hard delete. This flag is now supported on both |cp| and |ccloud|, which sets limits on the number of schema versions supported in the registry. A hard delete frees up space in a way that a soft delete does not. See also, :ref:`hard-delete-schema` in :ref:`schemaregistry_deletion`, and :ref:`sr-in-cloud-manage-space`. :>json int: Version of the deleted schema :statuscode 404: * Error code 40401 -- Subject not found * Error code 40402 -- Version not found :statuscode 422: * Error code 42202 -- Invalid version :statuscode 500: * Error code 50001 -- Error in the backend data store **Example request**: .. sourcecode:: http DELETE /subjects/test/versions/1 HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json 1 .. http:get:: /subjects/(string: subject)/versions/{versionId: version}/referencedby Get a list of IDs of schemas that reference the schema with the given subject and version. :param string subject: Name of the subject :param versionId version: Version of the schema to be returned. Valid values for versionId are between [1,2^31-1] or the string "latest", which returns the last registered schema under the specified subject. Note that there may be a new latest schema that gets registered right after this request is served. :>jsonarr int id: Globally unique identifier of the schema :statuscode 404: * Error code 40401 -- Subject not found :statuscode 500: * Error code 50001 -- Error in the backend datastore **Example request**: .. sourcecode:: http GET /subjects/test/versions/1/referencedby HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json [ 1, 2, 3, 4 ] Mode ------ You can use the mode resource to set the mode on |sr| at a global level or to set the mode on a specific subject. The following modes are available: - IMPORT - READONLY - READWRITE .. tip:: To enable mode changes on a |sr| cluster, you must also set ``mode.mutability=true`` in the |sr| properties file before starting |sr|. Examples of setting this property and changing the mode on |sr| at a global level are shown as a part of the procedure to :ref:`schemaregistry_migrate`. .. http:get:: /mode Get the current mode for |sr| at a global level. :json boolean is_compatible: True, if compatible. False otherwise :statuscode 404: * Error code 40401 -- Subject not found * Error code 40402 -- Version not found :statuscode 422: * Error code 42201 -- Invalid schema * Error code 42202 -- Invalid version :statuscode 500: * Error code 50001 -- Error in the backend data store **Example request**: .. sourcecode:: http POST /compatibility/subjects/test/versions/latest HTTP/1.1 Host: schemaregistry.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json { "schema": "{ \"type\": \"record\", \"name\": \"test\", \"fields\": [ { \"type\": \"string\", \"name\": \"field1\" }, { \"type\": \"int\", \"name\": \"field2\" } ] }" } **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { "is_compatible": true } Config ------ The config resource allows you to inspect the cluster-level configuration values as well as subject overrides. .. http:put:: /config Update global compatibility level. When there are multiple instances of |sr| running in the same cluster, the update request will be forwarded to one of the instances designated as the primary. If the primary is not available, the client will get an error code indicating that the forwarding has failed. :`. Must be one of BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE :statuscode 422: * Error code 42203 -- Invalid compatibility level :statuscode 500: * Error code 50001 -- Error in the backend data store * Error code 50003 -- Error while forwarding the request to the primary .. sourcecode:: http PUT /config HTTP/1.1 Host: kafkaproxy.example.com Accept: application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json { "compatibility": "FULL", } **Example response**: .. sourcecode:: http HTTP/1.1 200 OK Content-Type: application/vnd.schemaregistry.v1+json { "compatibility": "FULL", } .. http:get:: /config Get global compatibility level. :`__, `standard clusters `__, and `dedicated clusters `__ as described in `Confluent Cloud Cluster Types `__. You can view usage and available schemas on the |ccloud| web UI as described in `View Allowed Usage for Schemas on Confluent Cloud `__. When the maximum limit is reached and the registry is full, any requests to register new schemas will generate the following HTTP response: .. sourcecode:: http HTTP/1.1 422 Unprocessable Entity Content-Type: application/vnd.schemaregistry.v1+json { "error_code": 403, "message": "Schema Limit Exceeded" } See the topic below for how you can :ref:`free up storage space for new schemas ` if needed. To learn more about |ccloud| features, see `Confluent Cloud `__. .. _sr-in-cloud-free-up-space: Free Up Storage Space in the Registry for New Schemas ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Simply deleting schemas will not free up space in the registry because this will always result in a `soft delete` and schema IDs are not reusable. The schema count tracks IDs, so the used number will increase as new schemas are added, regardless of whether you soft deleted schemas. |ccloud| now supports "hard delete" of a schema as a two-step process (soft delete, followed by hard delete) with the use of the query string, ``?permanent=true`` on the second delete. If your |sr| reaches the :ref:`maximum schemas limit`, you can free up space for additional schemas by following the procedure described in :ref:`hard-delete-schema` in :ref:`schemaregistry_deletion`. Suggested Reading ----------------- * :ref:`schemaregistry_using` * :ref:`schema_registry_tutorial`