Schema Registry C++ Client libschemaregistry for Confluent Cloud

The C++ Schema Registry client libschemaregistry provides a Schema Registry client and serializers and deserializers (SerDes) for Avro, Protobuf, and JSON Schema. You can use this library with librdkafka, but it doesn’t depend on it.

Availability

For the latest version information, supported platforms, and compatibility, see the library’s README in the libschemaregistry repository.

Key features

Avro, Protobuf, and JSON Schema SerDes
Data quality rules using Google Common Expression Language (CEL)
Schema migration rules using JSONata
Client-side field-level encryption (CSFLE) with AWS KMS, Azure Key Vault, Google Cloud KMS, HashiCorp Vault, and a local test KMS

Install and build

For installation prerequisites (toolchains, package managers) and step-by-step build instructions, see the libschemaregistry README.

Use with CMake

This project installs a CMake package named schemaregistry. Link it as follows:

find_package(schemaregistry CONFIG REQUIRED)
target_link_libraries(myapp PRIVATE schemaregistry::schemaregistry)

Configure Schema Registry access

Configure the Schema Registry endpoint and credentials for your client. For property semantics, see Configuration Reference for Schema Registry Clients on Confluent Cloud.

Typical settings include:

schema.registry.url: the Schema Registry endpoint
basic.auth.credentials.source: USER_INFO for Confluent Cloud
basic.auth.user.info: <sr-api-key>:<sr-api-secret>

Minimal Confluent Cloud example

// Create Schema Registry client configuration
auto client_config = std::make_shared<ClientConfiguration>(
    std::vector<std::string>{schema_registry_url});

client_config->setBasicAuth(std::make_pair("<sr-api-key>", "<sr-api-secret>"));

// Create Schema Registry client
client_ = SchemaRegistryClient::newClient(client_config);

Use an https URL for Confluent Cloud. Ensure your environment trusts public certificate authorities. If you use a custom truststore, configure it at the platform level so outbound TLS to Schema Registry succeeds.

If you encounter certificate verification errors, ensure your system’s CA bundle is up to date. On Linux, this typically means updating the ca-certificates package. On macOS, certificates are managed by the system keychain.

Quick start examples

For complete, runnable producer and consumer examples (including Avro, Protobuf, and JSON Schema, with and without CSFLE), see the examples directory in the repository.

Match the example version to your installed library version. The examples link points to the main branch. For a specific release, replace main with the version tag, for example, 0.1.0.

Basic serialization pattern

Here’s a minimal example showing the core pattern for Avro serialization:

// Define these variables for your environment:
std::string schema_registry_url = "https://your-sr-endpoint";
std::string topic_name = "my-topic";

// Create Schema Registry client configuration
auto client_config = std::make_shared<ClientConfiguration>(
    std::vector<std::string>{schema_registry_url});

client_config->setBasicAuth(std::make_pair("SR_API_KEY", "SR_API_SECRET"));

// Create Schema Registry client
client_ = SchemaRegistryClient::newClient(client_config);

// Define the Avro schema
std::string schema_str = R"(
{
  "namespace": "confluent.io.examples.serialization.avro",
  "name": "User",
  "type": "record",
  "fields": [
    {"name": "name", "type": "string"},
    {"name": "favorite_number", "type": "long"},
    {"name": "favorite_color", "type": "string"}
  ]
}
)";

// Create Schema object
Schema schema;
schema.setSchemaType("AVRO");
schema.setSchema(schema_str);

// Parse and validate the schema
std::istringstream schema_stream(schema_str);
::avro::compileJsonSchema(schema_stream, valid_schema_);

// Create serializer configuration
std::unordered_map<std::string, std::string> rule_config;
SerializerConfig ser_config(true, std::nullopt, true, false, rule_config);

// Create Avro serializer
serializer_ = std::make_unique<AvroSerializer>(client_, schema, nullptr,
    ser_config);

// Create serialization context
SerializationContext ser_ctx;
ser_ctx.topic = topic_name;
ser_ctx.serde_type = SerdeType::Value;
ser_ctx.serde_format = SerdeFormat::Avro;
ser_ctx.headers = std::nullopt;

// Create GenericDatum with schema and get the record
::avro::GenericDatum datum(valid_schema_);
::avro::GenericRecord& record = datum.value<::avro::GenericRecord>();

// Set field values
record.setFieldAt(0, ::avro::GenericDatum(std::string("Alice")));
record.setFieldAt(1, ::avro::GenericDatum(static_cast<int64_t>(42)));
record.setFieldAt(2, ::avro::GenericDatum(std::string("blue")));

// Serialize the record
std::vector<uint8_t> serialized_data = serializer_->serialize(ser_ctx, datum);

For deserialization and complete producer and consumer examples with Apache Kafka® integration, see the examples in the repository.

SerDes provided by libschemaregistry

The library provides:

AvroSerializer / AvroDeserializer (Avro)
ProtobufSerializer / ProtobufDeserializer (Protobuf)
JsonSerializer / JsonDeserializer (JSON Schema using jsoncons)

Subject naming, auto-registration, schema references, and contexts follow the standard Schema Registry semantics. For production, consider disabling auto-registration and managing schemas centrally. See Manage Schemas in Confluent Cloud.

CSFLE (Client-side field-level encryption)

To encrypt sensitive fields, tag them in the schema and configure a KMS provider. For example, an Avro field tagged as PII:

{
  "name": "name",
  "type": "string",
  "confluent:tags": ["PII"]
}

KMS providers: AWS KMS, Azure Key Vault, Google Cloud KMS, HashiCorp Vault, local.

Example (AWS) environment variables:

export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key

For CSFLE concepts and key management, see:

Example files in the repository

Key examples in the repository:

The examples directory also has examples for JSON Schema and Protobuf, with and without encryption.

Building the examples requires librdkafka. The encryption examples also require Tink. Build the examples:

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -G Ninja \
  -DCMAKE_TOOLCHAIN_FILE="${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake" \
  -DSCHEMAREGISTRY_BUILD_EXAMPLES=ON
cmake --build build --target example

Run examples:

Avro consumer (basic):

./avro_consumer -b localhost:9092 -g my_group -t my_topic -u https://<sr-endpoint>

Avro producer (basic):

./avro_producer -b localhost:9092 -t my_topic -u https://<sr-endpoint>

Avro producer (encryption):

./avro_producer_encryption -b localhost:9092 -t my_topic -u https://<sr-endpoint> \
  --kek-name my-kek --kms-type local-kms --kms-key-id my-key

Known limitations

OAuth is not yet supported for C/C++ clients.

Migration from `libserdes`

libserdes, the legacy C library for Schema Registry, is discontinued. Use libschemaregistry for new development. For migration guidance, re-create serializers and deserializers using the corresponding C++ SerDes and configure Schema Registry access using the settings in Configuration Reference for Schema Registry Clients on Confluent Cloud.