Schema Registry C++ Client (libschemaregistry)

The C++ Schema Registry client libschemaregistry provides a Schema Registry client and serializers/deserializers (serdes) for Avro, Protobuf, and JSON Schema. This library can be used with librdkafka but does not depend on it.

Availability

Key features

  • Avro, Protobuf, and JSON Schema serdes

  • Data quality rules using Google Common Expression Language (CEL)

  • Schema migration rules using JSONata

  • Client-side field-level encryption (CSFLE) with AWS KMS, Azure Key Vault, Google Cloud KMS, HashiCorp Vault, and a local test KMS

Install and build

For installation prerequisites (toolchains, package managers) and step-by-step build instructions, see the libschemaregistry README.

Use with CMake

This project installs a CMake package named schemaregistry. Link it as follows:

find_package(schemaregistry CONFIG REQUIRED)
target_link_libraries(myapp PRIVATE schemaregistry::schemaregistry)

Configure Schema Registry access

Configure the Schema Registry endpoint and credentials for your client. For property semantics, see Configuration Reference for Schema Registry Clients on Confluent Cloud.

Typical settings include:

  • schema.registry.url: the Schema Registry endpoint

  • basic.auth.credentials.source: usually USER_INFO for Confluent Cloud

  • basic.auth.user.info: <sr-api-key>:<sr-api-secret>

Minimal Confluent Cloud example

// Create Schema Registry client configuration
auto client_config = std::make_shared<ClientConfiguration>(
    std::vector<std::string>{schema_registry_url});

client_config->setBasicAuth(std::make_pair("<sr-api-key>", "<sr-api-secret>"));

// Create Schema Registry client
client_ = SchemaRegistryClient::newClient(client_config);

Note

Use an https URL for Confluent Cloud. Ensure your environment trusts public certificate authorities. If you use a custom trust store, configure it at the platform level so outbound TLS to Schema Registry succeeds.

If you encounter certificate verification errors, ensure your system’s CA bundle is up to date. On Linux, this typically means updating the ca-certificates package. On macOS, certificates are managed by the system keychain.

Quickstart examples

For complete, runnable producer and consumer examples (including Avro, Protobuf, and JSON Schema, with and without CSFLE), see the examples directory in the repository.

Note

Match the example version with your installed library version. The link above points to the latest main branch. For a specific release, replace main with the version tag (e.g., 0.1.0).

Basic serialization pattern

Here’s a minimal example showing the core pattern for Avro serialization:

// Define these variables for your environment:
std::string schema_registry_url = "https://your-sr-endpoint";
std::string topic_name = "my-topic";

// Create Schema Registry client configuration
auto client_config = std::make_shared<ClientConfiguration>(
    std::vector<std::string>{schema_registry_url});

client_config->setBasicAuth(std::make_pair("SR_API_KEY", "SR_API_SECRET"));

// Create Schema Registry client
client_ = SchemaRegistryClient::newClient(client_config);

// Define the Avro schema
std::string schema_str = R"(
{
  "namespace": "confluent.io.examples.serialization.avro",
  "name": "User",
  "type": "record",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "favorite_number", "type": "long"},
    {"name": "favorite_color", "type": "string"}
  ]
}
)";

// Create Schema object
Schema schema;
schema.setSchemaType("AVRO");
schema.setSchema(schema_str);

// Parse and validate the schema
std::istringstream schema_stream(schema_str);
::avro::compileJsonSchema(schema_stream, valid_schema_);

// Create serializer configuration
std::unordered_map<std::string, std::string> rule_config;
SerializerConfig ser_config(true, std::nullopt, true, false, rule_config);

// Create Avro serializer
serializer_ = std::make_unique<AvroSerializer>(client_, schema, nullptr,
    ser_config);

// Create serialization context
SerializationContext ser_ctx;
ser_ctx.topic = topic_name;
ser_ctx.serde_type = SerdeType::Value;
ser_ctx.serde_format = SerdeFormat::Avro;
ser_ctx.headers = std::nullopt;

// Create GenericDatum with schema and get the record
::avro::GenericDatum datum(valid_schema_);
::avro::GenericRecord& record = datum.value<::avro::GenericRecord>();

// Set field values
record.setFieldAt(0, ::avro::GenericDatum(std::string("Alice")));
record.setFieldAt(1, ::avro::GenericDatum(static_cast<int64_t>(42)));
record.setFieldAt(2, ::avro::GenericDatum(std::string("blue")));

// Serialize the record
std::vector<uint8_t> serialized_data = serializer_->serialize(ser_ctx, datum);

For deserialization and complete producer/consumer examples with Kafka integration, see the examples in the repository.

Serdes

The library provides:

  • AvroSerializer / AvroDeserializer (Avro)

  • ProtobufSerializer / ProtobufDeserializer (Protobuf)

  • JsonSerializer / JsonDeserializer (JSON Schema using jsoncons)

Subject naming, auto-registration, schema references, and contexts follow the standard Schema Registry semantics. For production, consider disabling auto-registration and managing schemas centrally. See Manage Schemas and Data Contracts.

CSFLE (Client-side field-level encryption)

To encrypt sensitive fields, tag them in the schema and configure a KMS provider. For example, an Avro field tagged as PII:

{
  "name": "name",
  "type": "string",
  "confluent:tags": ["PII"]
}

KMS providers: AWS KMS, Azure Key Vault, Google Cloud KMS, HashiCorp Vault, local.

Example (AWS) environment variables:

export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key

For CSFLE concepts and key management, see:

Examples

Key examples in the repository:

Additional examples for JSON Schema and Protobuf (with and without encryption) are also available in the examples directory.

Build the examples (requires librdkafka and Tink for encryption examples):

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -G Ninja \
  -DCMAKE_TOOLCHAIN_FILE="${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake" \
  -DSCHEMAREGISTRY_BUILD_EXAMPLES=ON
cmake --build build --target example

Run examples:

  • Avro Consumer (basic):

    ./avro_consumer -b localhost:9092 -g my_group -t my_topic -u https://<sr-endpoint>

  • Avro Producer (basic):

    ./avro_producer -b localhost:9092 -t my_topic -u https://<sr-endpoint>

  • Avro Producer (encryption):

    ./avro_producer_encryption -b localhost:9092 -t my_topic -u https://<sr-endpoint> \
  --kek-name my-kek --kms-type local-kms --kms-key-id my-key

Known limitations

OAuth is not yet supported for C/C++ clients.

Migration from libserdes

The legacy C library libserdes for Schema Registry is planned for deprecation. New development should use libschemaregistry. For migration guidance, re-create serializers/deserializers using the corresponding C++ serdes and configure Schema Registry access using the settings in Configuration Reference for Schema Registry Clients on Confluent Cloud.

Related content