Client Deprecation in Confluent Platform¶
The Apache Kafka® 3.7 distribution released in Q1, 2024 included KIP-896. KIP-896 is the proposal to remove support for older client protocol API versions in Kafka 4.0. The 3.7 release marked these older, client protocol API versions deprecated with a notice that they will be completely dropped by 4.0. This page discusses the changes in Confluent client behavior resulting from KIP-896.
The Confluent Platform 8.0 release removes compatibility for deprecated client versions. You are free to choose when you upgrade to Confluent Platform 8.0. Until you upgrade, you remain under the full extent of your pre-8.0 version support plan. Any pre-8.0 Confluent Platform version will maintain compatibility with the older client protocol API versions.
Minimal compatible baseline¶
With the release of Kafka 4.0 in 2025, Confluent sets the new baseline for client protocol API versions to Kafka 2.1.0, which was released in November 2018. As a result, clients connecting to new brokers must use compatible API versions.
Some protocol APIs are used for clients and inter-broker replication, for example, Fetch, ListOffsets, and so forth. To maintain compatibility with these dual-use APIs, Confluent Platform expects a minimum of 2.1.0 for the inter.broker.protocol.version value.
Any environment using versions other than established minimums can encounter one or more of the following version-related errors:
- Kafka brokers that return an
UNSUPPORTED_VERSIONerror when they receive a request with any of the removed API versions. - Java clients (producer, consumer, or admin) that throw
UnsupportedVersionExceptionwhen interacting with brokers that do not support the minimum protocol API versions. - A broker configured with an unsupported inter-broker protocol version that fails during start-up with an error.
The following chart describes the minimal compatible versions of some popular client libraries:
| Client | Version | Released |
|---|---|---|
| Java client for Kafka | 2.1.0 | November 2018 |
| Python (confluent-kafka-python) | 1.8.2 | October 2021 |
| .NET (confluent-kafka-dotnet) | 1.8.2 | October 2021 |
| JavaScript (confluent-kafka-javascript) | 1.8.2 | October 2021 |
| Go (confluent-kafka-go) | 1.8.2 | October 2021 |
| C/C++ (librdkafka) | 1.8.2 | October 2021 |
| KafkaJS | 1.15.0 | November 2020 |
| Sarama | 1.29.1 | June 2021 |
| kafka-python | 2.0.2 | Sep 2020 |
Any librdkafka-based protocols not listed in the preceding table have a version minimum of 1.8.2 as specified in the October 2021 release of Apache Kafka®.
List of deprecated Kafka APIs¶
| API | Deprecated version |
|---|---|
| Produce | V0-V2 |
| Fetch | V0-V3 |
| ListOffset | V0 |
| Metadata | None |
| OffsetCommit | V0-V1 |
| OffsetFetch | V0 |
| FindCoordinator | None |
| JoinGroup | V0-V1 |
| Heartbeat | V0-V1 |
| LeaveGroup | None |
| SyncGroup | None |
| DescribeGroups | None |
| ListGroups | None |
| SaslHandshake | None |
| ApiVersions | None |
| CreateTopics | V0-V1 |
| DeleteTopics | V0 |
| DeleteRecords | None |
| InitProducerId | None |
| OffsetForLeaderEpoch | V0-V1 |
| AddPartitionsToTxn | None |
| AddOffsetsToTxn | None |
| EndTxn | None |
| WriteTxnMarkers | None |
| TxnOffsetCommit | None |
| DescribeAcls | V0 |
| CreateAcls | V0 |
| DeleteAcls | V0 |
| DescribeConfigs | V0 |
| AlterConfigs | None |
| AlterReplicaLogDirs | V0 |
| DescribeLogDirs | V0 |
| SaslAuthenticate | None |
| CreatePartitions | None |
| CreateDelegationToken | V0 |
| RenewDelegationToken | V0 |
| ExpireDelegationToken | V0 |
| DescribeDelegationToken: | V0 |
| DeleteGroups | None |
How to identify deprecated client requests¶
Use the following procedure to locate deprecated client requests in your Confluent Platform environment:
If possible upgrade to the Confluent Platform 7.9.0 version. If you are using an older version and cannot upgrade to the 7.9.0 version, then upgrade to at least the Confluent Platform 7.7.x or 7.8.x version.
Either of these versions make it possible for you to use the new metrics and tags that help you identify deprecated clients in your environment.
Add the following metric to your monitoring platform:
kafka.network:type=RequestMetrics,name=DeprecatedRequestsPerSec,request=(api-name),version=(api-version),clientSoftwareName=(client-software-name),clientSoftwareVersion=(client-software-version)
This metric helps you identify if your cluster has any deprecated APIs.
Inspect the metric.
If the metric is zero, there are no deprecated requests coming to your cluster. If you see a non-zero metric, then enable the
kafka.request.loggerat theINFOlevel. Avoid the large accumulation of log data by enabling theINFOvalue for short periods at a time.Inspect the logs for full detailed information about client requests
You can leverage a specific logging tool such as Elastic search to easily search for deprecated requests or grab the logs from disk. Then, use a tool like
grepto locate entries with arequestApiVersionDeprecated=truevalue.
Recommended upgrades based on client ID¶
Many Kafka clients many not report their official software library or version in their requests. If this is the case, you must inspect a client’s clientId to identify the client type. Some client software embed recognizable identifiers, others do not. Further, some client identifiers have custom formats making it hard to determine their version or library.
The following list provides some examples of client IDs, the likely client type given the ID, and the actions you can take to upgrade the corresponding client:
| Client ID resembles | What type of client? | Suggested action to try |
|---|---|---|
producer-X or consumer-X |
Possibly a Java producer/consumer using the default clientId. The client could can be anything if the customer manually created the clientId. |
If a Java client, upgrade to the latest Java client version. |
beats or filebeats |
Possibly an Elastic’s filebeats product using the Sarama Kafka client under-the-hood. | Base your approach on the beats version. If the version is 7.X or higher, try to resolve the problem with a change in the version configuration to a value of 2.1.0 or higher. If you are using a version older than 7.X, upgrade to the latest version of beats. |
(some-string)- ( github.com/segmentio/kafka-go ) |
A kafka-go library
client maintained by Segment. |
The latest version of segmentio/kafka-go is currently not compliant for consumer applications. Confluent has raised an issue with the project to help remediate this flaw. Customers should use the latest version of the confluent-kafka-go library
which is compliant. |
sarama |
Sarama, a go client maintained by IBM. Sarama serves as the underlying client for other tools like beats | Upgrade to at least 1.29.1 and make sure that the Kafka is set to at least the 2.1.1 version. |
rdkafka or rdkafka-producer |
This client either be a Confluent maintained or a third-party client that uses librdkafka under the hood, for example, rdkafka is used as a default clientId. |
Upgrade to at least the librdkafka library’s 1.8.2 version. Ideally use the latest version. |
kafka-python-producer-X or kafka-python-consumer-X or kafka-python-X.Y.Z |
A popular Python library for Kafka (kafka-python). |
Upgrade to the latest version. |
Kafka-node-client |
A popular JS library for Kafka (kafka-node). |
No longer maintained. Try a different library such as confluent-kafka-javascript or KafkaJS. Though, KafkaJS is also not maintained. |
kaffe_producer_client |
Likely to be the “Kaffe” Kafka elixir client | As this library appears to be inconsistently maintained, consider moving to something with more maintenance such as the kafka_ex:master branch. |
aiokafka or aiokafka-producer-X |
The aiokafka Python client library | Upgrade to the latest version. |
faust and faust-X.Y.Z |
The faust library which serves as a Python alternative for Kafka Streams. | Upgrade to the latest version. |
ruby-kafka |
Possibly the Zendesk “ruby-kafka” client. | This client is officially deprecated by Zendesk. Consider another Ruby Kafka client such as the karafka/rdkafka-ruby client which a third- maintains. |