Skip to main content

confluent_kafka/consumer/
interceptor.rs

1// Copyright 2025 Confluent Inc.
2//
3// Licensed under the Apache License, Version 2.0 (the "License");
4// you may not use this file except in compliance with the License.
5// You may obtain a copy of the License at
6//
7//     http://www.apache.org/licenses/LICENSE-2.0
8//
9// Unless required by applicable law or agreed to in writing, software
10// distributed under the License is distributed on an "AS IS" BASIS,
11// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12// See the License for the specific language governing permissions and
13// limitations under the License.
14
15//! Consumer interceptor plugin trait.
16//!
17//! Translated from `org.apache.kafka.clients.consumer.ConsumerInterceptor`.
18
19use std::collections::HashMap;
20
21use crate::common::TopicPartition;
22use crate::consumer::{ConsumerRecords, OffsetAndMetadata};
23
24/// A plugin interface that allows you to intercept (and possibly mutate)
25/// records received by the consumer. A primary use-case is for third-party
26/// components to hook into the consumer applications for custom monitoring,
27/// logging, etc.
28///
29/// This trait will get consumer config properties via
30/// [`configure`](Self::configure), including the client id assigned by
31/// `KafkaConsumer` if not specified in the consumer config. The interceptor
32/// implementation needs to be aware that it will be sharing consumer config
33/// namespace with other interceptors and serializers, and ensure that there are
34/// no conflicts.
35///
36/// Errors raised by `ConsumerInterceptor` methods will be caught, logged, but
37/// not propagated further. As a result, if the user configures the interceptor
38/// with the wrong key and value type parameters, the consumer will not return
39/// an error, just log the errors.
40///
41/// `ConsumerInterceptor` callbacks are called from the same task that invokes
42/// [`Consumer::poll`](crate::consumer::Consumer::poll).
43///
44/// # Translation notes
45///
46/// The methods return no error: a panic is what Java's thrown exception
47/// becomes. `KafkaConsumer` takes no interceptors yet, and an interceptor
48/// implementing `ClusterResourceListener` or `Monitorable` gets no callback.
49// Bounds: `Send + 'static`, not `Send + Sync`. The interceptor is stored inside
50// `ConsumerInterceptors` as `Vec<Box<dyn ConsumerInterceptor<K, V>>>` with a
51// single owner, so `Sync` is not required; dropping it lets users use interior
52// mutability (`RefCell`, `Cell`) without a `Mutex`. The chain runs per batch
53// inside `poll()`, not per record, and `#[async_trait]` would change Java's
54// synchronous contract.
55#[doc(alias = "org.apache.kafka.clients.consumer.ConsumerInterceptor")]
56pub trait ConsumerInterceptor<K, V>: Send + 'static {
57    /// This is called just before the records are returned by
58    /// [`Consumer::poll`](crate::consumer::Consumer::poll).
59    ///
60    /// This method is allowed to modify consumer records, in which case the new
61    /// records will be returned. There is no limitation on number of records
62    /// that could be returned from this method. I.e., the interceptor can
63    /// filter the records or generate new records.
64    ///
65    /// Any panic raised by this method will be caught by the caller, logged,
66    /// but not propagated to the client.
67    ///
68    /// Since the consumer may run multiple interceptors, a particular
69    /// interceptor's `on_consume()` callback will be called in the order the
70    /// interceptors are configured. The first interceptor in the list gets the
71    /// consumed records, the following interceptor will be passed the records
72    /// returned by the previous interceptor, and so on. Since interceptors are
73    /// allowed to modify records, interceptors may potentially get the records
74    /// already modified by other interceptors. However, building a pipeline of
75    /// mutable interceptors that depend on the output of the previous
76    /// interceptor is discouraged, because of potential side-effects caused by
77    /// interceptors potentially failing to modify the record and panicking. If
78    /// one of the interceptors in the list panics from `on_consume()`, the panic
79    /// is caught, logged, and the next interceptor is called with the records
80    /// returned by the last successful interceptor in the list, or otherwise the
81    /// original consumed records.
82    ///
83    /// # Arguments
84    ///
85    /// - `records` - records to be consumed by the client or records returned by
86    ///   the previous interceptors in the list.
87    ///
88    /// # Translation notes
89    ///
90    /// Java returns the records; here the interceptor modifies `records` in
91    /// place, or replaces the batch with `*records = new_batch;`. Since `&mut`
92    /// lets a panicking interceptor leave a half-modified batch, build the new
93    /// batch first and assign it last: then a panic leaves `*records` unchanged,
94    /// as Java's immutable `ConsumerRecords` is. Do not `std::mem::take` the
95    /// batch before work that can panic, or the next interceptor gets an empty
96    /// batch.
97    // The `&mut` form, rather than taking and returning ownership, avoids a
98    // `K: Clone, V: Clone` bound that would otherwise propagate through
99    // `ConsumerInterceptors::on_consume` into the public `Consumer<K, V>` trait.
100    // Java's `ConsumerRecords` is structurally immutable (`private final`
101    // fields, unmodifiable views), so in Java the "previous-good batch"
102    // guarantee of `ConsumerInterceptors#onConsume` is a property of the type;
103    // here it is a discipline of the interceptor. Interior-mutable state of the
104    // interceptor is its own responsibility on panic, as in Java.
105    #[doc(alias = "org.apache.kafka.clients.consumer.ConsumerInterceptor#onConsume")]
106    fn on_consume(&self, records: &mut ConsumerRecords<K, V>);
107
108    /// This is called when offsets get committed.
109    ///
110    /// Any panic raised by this method will be ignored by the caller.
111    ///
112    /// # Arguments
113    ///
114    /// - `offsets` - A map of offsets by partition with associated metadata
115    #[doc(alias = "org.apache.kafka.clients.consumer.ConsumerInterceptor#onCommit")]
116    fn on_commit(&self, offsets: &HashMap<TopicPartition, OffsetAndMetadata>);
117
118    /// Configure this class with the given key-value pairs.
119    ///
120    /// # Arguments
121    ///
122    /// - `configs` - The configuration
123    #[doc(alias = "org.apache.kafka.common.Configurable#configure")]
124    fn configure(&mut self, configs: &HashMap<String, String>) {
125        let _ = configs;
126    }
127
128    /// This is called when interceptor is closed.
129    #[doc(alias = "org.apache.kafka.clients.consumer.ConsumerInterceptor#close")]
130    fn close(&mut self) {}
131}