.. _schemaregistry_using:

|sr| API Usage Examples for |cp|
================================

This section provides examples of calls to the :ref:`Schema Registry API <schemaregistry_api>` using `curl <https://curl.haxx.se/>`__ commands.

You can also see a few more examples of using curl to interact with these APIs in the :ref:`Schema Registry Tutorial <tutorial-use-curl-with-schema-registry>`.  
  
For your API testing, you may want to use `jq <https://stedolan.github.io/jq/>`_ along with ``--silent`` flag for `curl <https://curl.haxx.se/>`__
to get nicely formatted output for the given commands. For example, the command: ``curl -X GET http://localhost:8081/subjects`` results in:

.. code:: bash 

    ["my-cool-topic-value","my-other-cool-topic-value"]

Whereas, the same command using curl in silent mode and piped through jq: ``curl --silent -X GET http://localhost:8081/subjects | jq`` results in:

.. code:: bash

    "my-cool-topic-value",
    "my-other-cool-topic-value"

Starting |sr|
~~~~~~~~~~~~~
Start |sr| and its dependent services |zk| and Kafka. Each service reads its configuration from its property files under
``etc``.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Development or Test Environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: ../../includes/install-cli-prereqs.rst

You can use the |confluent-cli| :confluent-cli:`confluent local|command-reference/local/index.html` commands to start |sr| and its dependent services with this command:

.. codewithvars:: bash

     |confluent_start_schema_registry|

.. include:: ../../includes/cli.rst
      :start-after: cli_limitations_start
      :end-before: cli_limitations_end

~~~~~~~~~~~~~~~~~~~~~~
Production Environment
~~~~~~~~~~~~~~~~~~~~~~

Start each |cp| service in its own terminal using this order of operations:

#. Start |zk|. Run this command in its own terminal.

   .. sourcecode:: bash

      bin/zookeeper-server-start ./etc/kafka/zookeeper.properties

#. Start Kafka. Run this command in its own terminal.

   .. sourcecode:: bash

      bin/kafka-server-start ./etc/kafka/server.properties

#. Start |sr|. Run this command in its own terminal.

   .. sourcecode:: bash

      bin/schema-registry-start ./etc/schema-registry/schema-registry.properties

.. ifconfig:: platform_docs

   See the :ref:`installation` for a more detailed explanation of how
   to get these services up and running.

Common |sr| API Usage Examples
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. seealso:: 
  
   - `Formats, Serializers, and Deserializers <https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html>`__
   - :ref:`schemaregistry_api`
   - For a detailed example that uses |sr| configured with security, see the :ref:`Confluent Platform demo <cp-demo>`.

These examples use `curl <https://curl.haxx.se/docs/manual.html>`__ commands to interact with the |sr| :ref:`Schema Registry API <schemaregistry_api>`.

Commands and results are shown separately to make it easy to copy-paste the commands into a shell.

For |sr| on |ccloud|, pass the API key and secret with the ``--user`` (or ``-u``) flag on the curl command. For example, to view all subjects in the registry:

.. codewithvars:: bash

   curl --user <schema-registry-api-key>:<schema-registry-api-secret> \
   <schema-registry-url>/subjects

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register a new version of a schema under the subject "Kafka-key"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

  curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"schema": "{\"type\": \"string\"}"}' \
    http://localhost:8081/subjects/Kafka-key/versions
        
Example result:
        
::
  
      {"id":1}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register a new version of a schema under the subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


.. code:: bash

  curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"schema": "{\"type\": \"string\"}"}' \
  http://localhost:8081/subjects/Kafka-value/versions
         
Example result:
        
::
         
  {"id":1}

.. _sr-reg-exist-schema:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register an existing schema to a new subject name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Use case: there is an existing schema registered to a subject called ``Kafka1``, and this same schema needs to be available to another subject called ``Kafka2``.
The following one-line command reads the existing schema from ``Kafka1-value`` and registers it to ``Kafka2-value``.
It assumes the tool ``jq`` is installed on your machine.

.. code:: bash

  curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data "{\"schema\": $(curl -s http://localhost:8081/subjects/Kafka1-value/versions/latest | jq '.schema')}" \
  http://localhost:8081/subjects/Kafka2-value/versions
  
Example result:
  
::

  {"id":1}

.. tip:: You do not need to use the :connect-common:`AvroConverter|userguide.html#configuring-key-and-value-converters` for topic replication or
         schema management, even if the topic is Avro format. The ``ByteArrayConverter`` retains
         the "magic byte", which is the schema ID. When a replicator is created, messages are
         replicated with the schema ID. You do not need to create a schema subject. A best practice
         is to avoid the use of Avro due to the overhead, as there is no real value to it in this context.

         To learn more, see :ref:`replicator_quickstart`, including the use of ``topic.rename.format=${topic}.replica``
         in the subsection on :ref:`config-and-run-replicator`, and :ref:`replicator_failover`.

.. _kafka-key-listing-all-subjects:

~~~~~~~~~~~~~~~~~
List all subjects
~~~~~~~~~~~~~~~~~

The following API call lists all schema subjects.

.. code:: bash

  curl -X GET http://localhost:8081/subjects
      
Example result:
  
::
      
  ["Kafka-value","Kafka-key"]

You can use the ``deleted`` flag at the end of the request to list all subjects, including subjects that have been soft-deleted (``?deleted=true``).

.. code:: bash

    curl -X GET http://localhost:8081/subjects?deleted=true
          
Example result, assuming you had a schema subject called "my-cool-topic-value" that was previously soft-deleted:  
  
::
  
  ["Kafka-value","Kafka-key","my-cool-topic-value"]

.. _list-subjects-for-given-id:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List all subjects associated with a given ID
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To find subjects associated with a given ID, use
`GET /schemas/ids/{int: id}/versions <https://docs.confluent.io/platform/current/schema-registry/develop/api.html#get--schemas-ids-int-%20id-versions>`__.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fetch a schema by globally unique ID 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

      curl -X GET http://localhost:8081/schemas/ids/1
      
Example result:
  
::
      
  {"schema":"\"string\""}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List all schema versions registered under the subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X GET http://localhost:8081/subjects/Kafka-value/versions
      
Example result:
  
::
      
  [1]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fetch Version 1 of the schema registered under subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X GET http://localhost:8081/subjects/Kafka-value/versions/1
      
Example result:
  
::
      
  {"subject":"Kafka-value","version":1,"id":1,"schema":"\"string\""}

.. tip:: If the schema type is JSON Schema or Protobuf, the response will also include the schema type.
         If the schema type is Avro, which is the default, the schema type is not included in the response, per the above example.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Delete Version 1 of the schema registered under subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X DELETE http://localhost:8081/subjects/Kafka-value/versions/1
      
Example result:
  
::
      
  1
  
.. seealso:: :ref:`schemaregistry_deletion`

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Delete the most recently registered schema under subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X DELETE http://localhost:8081/subjects/Kafka-value/versions/latest
      
Example result:
  
::

  2
  
.. seealso:: :ref:`schemaregistry_deletion`

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register the same schema under the subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
      --data '{"schema": "{\"type\": \"string\"}"}' \
       http://localhost:8081/subjects/Kafka-value/versions
       
Example result:
  
::
   
  {"id":1}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fetch the schema again by globally unique ID 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X GET http://localhost:8081/schemas/ids/1
      
Example result:
  
::
   
  {"schema":"\"string\""}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Check if a schema Is registered under subject "Kafka-key"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

      curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
        --data '{"schema": "{\"type\": \"string\"}"}' \
        http://localhost:8081/subjects/Kafka-key

Example result:

::
  
  {"subject":"Kafka-key","version":3,"id":1,"schema":"\"string\""}

.. _sr-test-compat-latest:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Test compatibility of a schema with the latest schema under subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
      --data '{"schema": "{\"type\": \"string\"}"}' \
      http://localhost:8081/compatibility/subjects/Kafka-value/versions/latest
      
Example result:

::

  {"is_compatible":true}
  
.. tip:: Starting with |cp| 6.1.0, you can add ``?verbose=true`` at the end of the
   request to output the reason a schema fails the compatibility test, in cases where it fails.
   To learn more, see :ref:`sr-api-compatibility` in the |sr| API Reference.

.. _sr-top-level-config:

~~~~~~~~~~~~~~~~~~~~~~~~~
Get the top level config
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

    curl -X GET http://localhost:8081/config
    
Example result:

::
  
  {"compatibility":"BACKWARD"}

.. _updating-compatibility-requirements-globally:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Update compatibility requirements globally
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

      curl -X PUT -H "Content-Type: application/vnd.schemaregistry.v1+json" \
        --data '{"compatibility": "NONE"}' \
        http://localhost:8081/config
        
Example result:
  
::

  {"compatibility":"NONE"}

.. _update-compatibility-on-a-subject:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register a schema for a new topic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. tip::
  
   This example and the next few examples refer to a new topic called ``my-kafka`` which will be used to demonstrate subject-level compatibility configuration.
   Assume for these examples that you have created this topic either in the :ref:`Confluent Control Center<c3_add_topics>` or at |ak| command line using. If you would
   like to stick with the command line and create the topic now to follow along, use |ak| commands similar to the following to create the topic, then check for its existence:
   
   .. code:: bash
   
      kafka-topics --create --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1 --topic my-kafka
      kafka-topics --list --bootstrap-server localhost:9092
      
Use the |sr| API to add a schema for the topic ``my-kafka``.

.. code:: bash

    curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema": "{\"type\":\"record\",\"name\":\"Payment\",\"namespace\":\"my.examples\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"amount\",\"type\":\"double\"}]}"}' http://localhost:8081/subjects/my-kafka-value/versions
        
Example result:
  
::

  {"id":1}
  
.. _set-compatibility-on-subject:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Update compatibility requirements on a subject
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

      curl -X PUT -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"compatibility": "FULL"}' http://localhost:8081/config/my-kafka-value
        
Example result:
  
::

  {"compatibility":"FULL"}
  
.. _get-compatibility-on-subject:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Get compatibility requirements on a subject
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

      curl -X GET http://localhost:8081/config/my-kafka-value
        
Example result:  
  
::
  
  {"compatibilityLevel":"FULL"}
  
.. tip:: If the subject you ask about does not have a subject-specific compatibility level set, this command returns an error code.
         For example, if you run the same command for the subject ``Kafka-value``, for which you have not set subject-specific compatibility, you get:

         .. code:: bash
         
             {"error_code":40401,"message":"Subject 'Kafka-value' not found."}
  
.. _show-compatibility-subject-or-global:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Show compatibility requirements in effect for a subject
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can use the flag ``defaultToGlobal`` to determine what compatibility
requirements, if any, are set at the subject level and what requirements will be
used for compatibility checks. These are often, but not always, the same if
a subject has subject-level compatibility set.

- For the subject ``my-kafka-value``, which has a subject-specific compatibility set to "FULL", ``defaultToGlobal=true`` and ``defaultToGlobal=false`` `both` return ``{"compatibilityLevel":"FULL"}``.

  .. code:: bash

      curl -X GET http://localhost:8081/config/my-kafka-value/?defaultToGlobal=true
          
  Example result:  
    
  ::
    
    {"compatibilityLevel":"FULL"}
  
  .. code:: bash

       curl -X GET http://localhost:8081/config/my-kafka-value/?defaultToGlobal=false
        
  Example result:  
    
  ::
    
    {"compatibilityLevel":"FULL"}
  
- For the subject ``Kafka-value``, for which you have not set subject-specific compatibility, ``defaultToGlobal=true`` returns the current global default, for example: ``{"compatibilityLevel":"NONE"}``.

  .. code:: bash

      curl -X GET http://localhost:8081/config/Kafka-value/?defaultToGlobal=true
      
  Example result:
  
  ::
    
    {"compatibilityLevel":"NONE"}

- Whereas, ``defaultToGlobal=false`` on the subject ``Kafka-value`` returns an error code:

  .. code:: bash

      curl -X GET http://localhost:8081/config/Kafka-value/?defaultToGlobal=false
      
  Example result:
    
  .. code:: bash

        {"error_code":40401,"message":"Subject 'Kafka-value' not found."}

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Delete all schema versions registered under the subject "Kafka-value"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

  curl -X DELETE http://localhost:8081/subjects/Kafka-value

Example result:

::

  [3]
  
.. seealso:: :ref:`schemaregistry_deletion`
  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List schema types currently registered in |sr|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

  curl -X GET http://localhost:8081/schemas/types
  
Example result:

::
  
  ["JSON", "PROTOBUF", "AVRO"]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List all subject-version pairs where a given ID is used
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

  curl -X GET http://localhost:8081/schemas/ids/2/versions
  
Example result:

::
  
  [{"subject":"testproto-value","version":1}]
      
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
List IDs of schemas that reference a given schema
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: bash

  curl -X GET http://localhost:8081/subjects/other.proto/versions/1/referencedby
  
Example result:

::
  
  [2]


.. _sr-over-https-api-examples:

Using |sr| over HTTPS
~~~~~~~~~~~~~~~~~~~~~

The curl command examples provided above show how to communicate with |sr| over HTTP.

These examples show how to communicate with |sr| over :ref:`HTTPS <schema_registry_http_https>`.
You can extrapolate from these few examples to know how to run additional commands. When communicating
with |sr| with HTTPS enabled, apply the patterns shown for the curl commands (specifying a certificate,
key, and so forth) to accomplish the other usage examples shown above. For more about configuring and
using |sr| with security enabled, see :ref:`schemaregistry_security`.

~~~~~~~~~~~~~~~~~~~~
Verify HTTPS on |sr|
~~~~~~~~~~~~~~~~~~~~

::
  
  openssl s_client -connect schemaregistry:8082/subjects -cert client.certificate.pem -key client.key -tls1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Register a new version of a schema under the subject "Kafka-key"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

::
  
  curl -v -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema": "{\"type\": \"string\"}"}' --cert /etc/kafka/secrets/client.certificate.pem --key /etc/kafka/secrets/client.key --tlsv1.2 --cacert /etc/kafka/secrets/snakeoil-ca-1.crt https://schemaregistry:8082/subjects/Kafka-key/versions

~~~~~~~~~~~~~~~~~
List all subjects
~~~~~~~~~~~~~~~~~

::
  
  curl -v -X GET --cert /etc/kafka/secrets/client.certificate.pem --key /etc/kafka/secrets/client.key --tlsv1.2 --cacert /etc/kafka/secrets/snakeoil-ca-1.crt https://schemaregistry:8082/subjects/

.. _sr-on-ccloud-api-curl:

Use curl to access |sr| in |ccloud|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: ../includes/curl-sr-ccloud.rst

For more about using |sr| on |ccloud|, see :cloud:`Quick Start for Schema Management on Confluent Cloud|get-started/schema-registry.html`.

Related Content
~~~~~~~~~~~~~~~

* :ref:`schemaregistry_api`
* :ref:`schema_registry_onprem_tutorial`
* `Formats, Serializers, and Deserializers <https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html>`__
* :ref:`schema_evolution_and_compatibility`
* :ref:`schemaregistry_monitoring`