Enforce Data Governance Centrally with Confluent Gateway

Confluent Gateway enforces centralized governance for your Apache Kafka® workloads. You define data policies once in Confluent Schema Registry, and Confluent Gateway enforces them on every producer and consumer request.

Because enforcement happens in Confluent Gateway, only compliant records reach your Kafka clusters, and you can change rules across all routes without reconfiguring client applications or individual brokers. For more information about Schema Registry, see Schema Registry.

To use encryption rules, you need a CSFLE add-on license in addition to your Confluent Gateway add-on license. For more information, see Gateway Prerequisites. You can configure governance in Docker-based deployments and in Confluent for Kubernetes deployments. For CFK, see Enforce Centralized Governance for Confluent Gateway using Confluent for Kubernetes.

Centralized governance capabilities

Confluent Gateway applies the following governance capabilities to records that use Apache Avro®, JSON Schema, or Protobuf:

• Schema ID enforcement: Ensures every record carries a valid Schema Registry schema ID for its topic.

• Deep schema validation: Ensures every record’s payload conforms to its registered schema. For JSON Schema, Confluent Gateway enforces schema constraints, such as minimum and enum, only if you set json.fail.invalid.schema to "true". For more information, see Step 2: Enable governance for record keys and values.

• Field-level encryption: Encrypts and decrypts individual tagged fields in a record, as defined by data contract rules.

• Full payload encryption: Encrypts and decrypts the entire record payload, as defined by data contract rules.

• Data contract rules: Runs the other rules in a schema’s rule set, such as Common Expression Language (CEL) rules, when governance is enabled, the schema carries them, and enableAt allows Confluent Gateway to run them.

• Independent key and value governance: Lets you enable or disable governance for record keys and record values separately.

• Per-topic overrides: Lets you override the global settings on specific topics to support varying workload requirements.

How centralized governance enforcement works

Confluent Gateway doesn’t store validation or encryption policies. It reads them from Schema Registry and enforces them. Governance involves these steps:

1. In Confluent Gateway, enable or disable governance for record keys and record values.

2. In Schema Registry, define the schema and its rule set. The schema and rules determine what Confluent Gateway enforces, and the enableAt setting in the rule set determines whether Confluent Gateway runs the rules.

Confluent Gateway runs the same checks on the produce path and the fetch path. It evaluates each record key and value separately. For each one, it uses the configured enabled setting and subject name strategy, and the schema registered under the resulting subject.

The following table shows what Confluent Gateway enforces for a record key or value:

Confluent Gateway enabled	Schema enableAt	Schema ID enforcement	Deep schema validation	Field-level encryption	Full payload encryption
false	Any	No	No	No	No
true	CLIENT, SERVER, or NONE	Yes	Yes	No (run by the client or server)	No (run by the client or server)
true	GATEWAY, ALL, or not set	Yes	Yes	Yes, if the schema has the rule	Yes, if the schema has the rule

Note the following:

• When enabled is false, Confluent Gateway forwards the key or value unchanged. It doesn’t read the schema ID and performs no enforcement, regardless of the schema’s enableAt setting.

• When enabled is true, Confluent Gateway always performs schema ID enforcement and deep schema validation, regardless of the enableAt setting.

• When a record fails schema ID enforcement or deep schema validation, Confluent Gateway rejects the record on the produce path. On the fetch path, it fails the affected partition with an UNKNOWN_SERVER_ERROR error and returns no records for that partition.

Limitations

The following limitations apply to centralized governance enforcement:

• Confluent Gateway doesn’t publish failed records to a dead letter queue (DLQ). It doesn’t support setting a rule’s onFailure field to DLQ. A rule with onFailure: DLQ rejects the record. To route failed records to a DLQ topic, handle the error in your client application.

• You can’t enable schema ID enforcement and deep schema validation separately. Both run whenever you enable governance for a key or value, including when Confluent Gateway encrypts the same schema with enableAt: GATEWAY.

• Records must carry a schema ID, either in the Schema Registry wire format or as a schema GUID in a record header. Confluent Gateway rejects records produced without one.

• Confluent Gateway doesn’t enforce governance on null keys or null values, such as unkeyed records and tombstones.

• If Confluent Gateway encrypts a topic’s records, it can’t decrypt records that were never encrypted, such as older records, records produced while governance was off, or records written directly to the cluster. When the encryption rule’s onFailure action for reads is ERROR, Confluent Gateway fails the affected partition on fetch and doesn’t return records to the consumer.

Next steps

To configure Confluent Gateway to enforce centralized governance, verify enforcement, and handle errors, see Enforce Centralized Governance with Confluent Gateway.