Schema Registry Switchover using Confluent Ansible and Unified Stream Manager

The Schema Registry switchover streamlines and automates the management of Schema Registry clusters within the Confluent ecosystem in order to facilitate automated migration, switchover, and rollback between different Schema Registry deployments, specifically between self-managed Confluent Platform clusters and cloud-managed Confluent Cloud clusters. The Confluent Cloud Schema Registry serves as the authoritative system of record, while the on-prem Confluent Platform Schema Registry functions as a cache for on-prem applications using Schema Linking. This setup allows read operations to be handled by the on-prem Schema Registry, while all write operations must be directed to the cloud Schema Registry.

Ansible Playbooks for Confluent Platform (Confluent Ansible) provides playbooks that automate the process of switching a Confluent Platform Schema Registry from being the primary registry to using Confluent Cloud Schema Registry as the primary. This enables seamless migration of schema management from on-premises to cloud.

This topic describes how to set up Confluent Ansible with Unified Stream Manager to automate Schema Registry switchover from Confluent Platform to Confluent Cloud.

For details about Unified Stream Manager, see Using Schema Registry in a Hybrid Setup.

Requirements

Confluent Platform Schema Registry must be deployed and running.
Confluent Cloud Schema Registry must be accessible.
Unified Stream Manager must be configured.

See Configure and Deploy Unified Stream Manager Using Confluent Ansible.
Valid API credentials for Confluent Cloud must be configured.

Configure automatic Schema Registry switchover

To configure automatic Schema Registry switchover with Confluent Ansible, take the following steps.

Step 1: Configure the inventory file

For the full configuration examples, see the Schema Registry Switchover using Confluent Ansible sample inventory.

all:
  vars:
    password_encoder_secret:           --- [1]

schema_registry:                       --- [2]
  vars:
    unified_stream_manager:            --- [3]
      schema_registry_endpoint:        --- [4]
      authentication_type:             --- [5]
      basic_username:                  --- [6]
      basic_password:                  --- [7]

    schema_exporters:                  --- [8]
      - name:                          --- [9]
        subjects:                      --- [10]
        context_type:                  --- [11]
        context:                       --- [12]
        config:                        --- [13]
          schema_registry_endpoint:
          authentication_type:
          basic_username:
          basic_password:

    sr_switch_over_exporter_name:      --- [14]

    schema_importers:                  --- [15]
      - name:                          --- [16]
        subjects:                      --- [17]
        config:                        --- [18]
          schema_registry_endpoint:
          authentication_type:
          basic_username:
          basic_password:

[1] Required. The secret for enabling schema exporter and importer. For more information, see password.encoder.secret.
[2] The below variables can be specified under the schema_registry role or under all role.
[3] Required to enable forward sync from Confluent Platform to Confluent Cloud. Contains Confluent Cloud Schema Registry connection details.
[4] Required. The endpoint of the remote Confluent Cloud Schema Registry.
[5] Required. The authentication type of the remote Confluent Cloud Schema Registry. The supported type is basic.
[6] Required. The API key of the Confluent Cloud Schema Registry.
[7] Required. The API secret of the Confluent Cloud Schema Registry.
[8] Required. The schema exporter configurations.

Only one exporter is allowed.
[9] Required. The name of the schema exporter. Must match sr_switch_over_exporter_name ([14]).
[10] Required. The subjects of the schema exporter. To export all subjects, use :*: for all subjects in all contexts, or specify patterns: [":.context:*"].
[11] Required. The context type of the schema exporter. Specify how to handle contexts. Supported types are AUTO, CUSTOM, NONE, and DEFAULT.

The default value is AUTO, whereby the exporter will use an auto-generated context in the destination cluster. The auto-generated context name will be reported in the status.

If set to NONE, the exporter copies the source schemas as-is.
[12] Required if context_type is CUSTOM. The context of the schema exporter.
[13] If omitted, Confluent Ansible will use the default values specified in [4], [5], [6], and [7].
[14] The name of the exporter to use for the switchover workflow.

If not specified, the workflow will be an import-only workflow.

If specified, the value must match one of schema_exporters[0].name.
[15] Required. The schema importers configuration.

Only one importer is allowed.
[16] Required. The name of the schema importer.
[17] Required. The subjects of the schema importer. To import all subjects, use :*: for the default context, or specify patterns: [":.context:*"].
[18] If omitted, Confluent Ansible will use the default values specified in [4], [5], [6], and [7].

Configuration examples

The following are examples of inventory file configurations.

Mirror all schemas:

# Unified Stream Manager Configuration
unified_stream_manager:
  schema_registry_endpoint: "https://psrc-xyz.us-east-1.aws.confluent.cloud"
  authentication_type: basic
  basic_username: "your-cc-api-key"
  basic_password: "your-cc-api-secret"

# Schema Exporter Configuration
schema_exporters:
  - name: "cp-to-cc-exporter"
    subjects: [":*:"]
    context_type: "NONE"

# Schema Importer Configuration
schema_importers:
  - name: "cc-to-cp-importer"
    subjects: [":*:"]

# Specify exporter for switchover
sr_switch_over_exporter_name: "cp-to-cc-exporter"

password_encoder_secret: <secret>

Sync Schemas to a specific context:

Export the default context in Confluent Platform to the site1 context in Confluent Cloud, and import all schemas in the site1 context in Confluent Cloud to the default context in Confluent Platform.

# When using contexts
unified_stream_manager:
  schema_registry_endpoint: "https://psrc-xyz.us-east-1.aws.confluent.cloud"
  authentication_type: basic
  basic_username: "your-cc-api-key"
  basic_password: "your-cc-api-secret"
  remote_context: "site1"

schema_exporters:
  - name: "production-exporter"
    subjects: ["*"]
    context_type: "CUSTOM"
    context: "site1"

schema_importers:
  - name: "production-importer"
    subjects: [":.site1:*"]
    context: "."

sr_switch_over_exporter_name: "production-exporter"

password_encoder_secret: <secret>

Export the corp context in Confluent Platform to the site1 context in Confluent Cloud, and import all schemas in the site1.corp context in Confluent Cloud to the corp context in Confluent Platform.

# When using contexts
unified_stream_manager:
  schema_registry_endpoint: "https://psrc-xyz.us-east-1.aws.confluent.cloud"
  authentication_type: basic
  basic_username: "your-cc-api-key"
  basic_password: "your-cc-api-secret"
  remote_context: "site1"

schema_exporters:
  - name: "production-exporter-2"
    subjects: [":.corp:*"]
    context_type: "CUSTOM"
    context: "site1"

schema_importers:
  - name: "production-importer-2"
    subjects: [":.site1.corp:*"]
    context: "site1"

sr_switch_over_exporter_name: "production-exporter-2"

password_encoder_secret: <secret>

Greenfield setup - enable forwarding and importer:

# When using contexts
unified_stream_manager:
  schema_registry_endpoint: "https://psrc-xyz.us-east-1.aws.confluent.cloud"
  authentication_type: basic
  basic_username: "your-cc-api-key"
  basic_password: "your-cc-api-secret"

schema_importers:
  - name: "production-importer-2"
    subjects: [":*:"]

password_encoder_secret: <secret>

Step 2: Run the Schema Registry switchover playbooks

The following tags are available for the switchover playbooks.

Tag	Purpose
Default (no tags)	Exporter setup
schema_exporter	Exporter setup
forward_to_cc	Forward mode only
schema_importer	Importer setup
switchover_to_cc	Complete migration

Run the switchover playbooks with the tags as shown below.

Default run: Runs schema exporter setup only.

ansible-playbook -i <inventory-file> \
  playbooks/primary_sr_switchover_cp_to_cc.yml

or

ansible-playbook -i <inventory-file> \
  playbooks/primary_sr_switchover_cp_to_cc.yml --tags schema_exporter

Forward to Confluent Cloud: Runs pre-checks and forward mode.

ansible-playbook -i <inventory-file> \
playbooks/primary_sr_switchover_cp_to_cc.yml --tags forward_to_cc

Schema importer setup: Runs reverse sync.

ansible-playbook -i <inventory-file> \
  playbooks/primary_sr_switchover_cp_to_cc.yml --tags schema_importer

Full switchover (complete migration): Runs the complete migration workflow, including schema exporter setup, switchover, and reverse sync.
```
ansible-playbook -i <inventory-file> \
playbooks/primary_sr_switchover_cp_to_cc.yml --tags switchover_to_cc
```

Troubleshooting

The following are troubleshooting tips:

Check pre-check error messages for configuration guidance.
Review this documentation for scenario-specific troubleshooting.
Validate your configuration against the examples provided.
Ensure all prerequisites are met before execution.

The following are a few troubleshooting steps.

Unified Stream Manager not configured error

Error Message: Unified Stream Manager remote endpoint must be configured. Current value: ‘none’. Please set unified_stream_manager.schema_registry_endpoint to a valid Confluent Cloud Schema Registry URL.

Solution: Configure unified_stream_manager.schema_registry_endpoint with your Confluent Cloud Schema Registry URL.

URL mismatch error

Error Message: Pre-check failed: Exporter and Unified Stream Manager remote endpoints must match

Solution: Ensure exporter config.schema_registry_endpoint matches unified_stream_manager.schema_registry_endpoint.

Missing exporter name error

Error Message: Pre-check failed: sr_switch_over_exporter_name must be defined

Solution: If schema_exporters is defined, set sr_switch_over_exporter_name to match the name of one of your configured exporters.