Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 7.0.14¶

Released April 2024

Community Features¶

Common¶

4fb11814 - Remove auto-generated CODEOWNERS. It’s inaccurate.
52b0f806 - Remove CODEOWNERS file
7a96a215 - Add service.yml
c4b849a5 - update jose4j
4c8d3021 - update jetty to 9.4.54
PR-581 - fix: no implicit activation of docker-arm profile
PR-563 - added a new docker-arm profile to build docker containers in arm64

Kafka¶

PR-1078 - Update jetty to 9.4.54.v20240208

ksqlDB¶

PR-10243 - Update Commons-Compress and commons-io
PR-10238 - Update apache.io.version
PR-10237 - Update commons-compress
PR-10223 - Change groupID to a more secure name
2f84873b - Change groupID to a more secure name
30544a65 - Cherry pick the replacement of easymock to mockito from newer branches.
f969d5c0 - Enable pinned nanoversions.
b49bbcbb - Specify type parameter for CommandStatus.
c7288dd0 - Remove maven cache restore.
cea27ec9 - set java version to 8
cd59a75a - Increase the execution time limit.
26db19fd - Fixed the extra args in service.yml file.
PR-10178 - fix: configure KsqlBoundedMemoryConfigSetter in StandaloneExecutor mode
37e06168 - Fix the parent version in commons.
5467ea20 - set java version to 8
0a1cf171 - Increase the execution time limit.
db81cd09 - Fixed the extra args in service.yml file.

REST Proxy¶

PR-1235 - Deflake KafkaConsumerManagerTest.testBackoffMsControlsPollCalls test
PR-1236 - disable flakey test

Schema Registry¶

PR-3059 - fix: nested schemas were not considered in matching the correct Json Union schema
PR-3038 - feat: Added ReferenceSchema support to allOf
ae0e3140 - fix: invalid behaviour for stringSchema in allOfToConnectSchema
6df56c8f - Optimization: remove known deps from ProtobufSchema
PR-3012 - Minor: Apply service bot changes to earlier branches
abbab267 - Ensure package change for Protobuf is incompatible
3bd1cdd5 - update common-compress
2ef767e3 - Fix merge issue for wire upgrade
177af0d0 - Add Protobuf compat check for moving field into existing oneOf
bbe261f4 - exclude okio
PR-2943 - Update Bouncycastle dependencies to address
d08a0978 - fix NPE when value is nil when print.schema.ids=true is set
13848a40 - Improve the performance of invalid name scrubbing

Version 7.0.13¶

Released December 2023

Community Features¶

Common¶

e99d8b1a - exclude logback-core and logback-classic in line with the changes in kafka-rest
eb200670 - update zookeeper component to match version in ce-kafka
0c034e86 - re-add dependencymanagement sections for old bouncycastle to satisfy downstream validation
33734c5f - add additional version definition to ease the update of the downstream repos
8e71caf2 - update bouncycastle

Kafka¶

590649b4 - CONFLUENT: Bump ZK version to 3.8.3 for 7.0.x
PR-14495 - HOTIFX: fix Kafka versions for system tests (#14495)

REST Proxy¶

PR-1230 - KREST-12372 Upgrade dependency to JDK 1.8 with version 1.77
PR-1221 - KREST-12630 Exclude logback from zookeeper dependency in tests
PR-1219 - KREST-12630 Add missing dependency due to zookeeper upgrade
PR-1212 - KREST-12277 semaphore migration to release branches
PR-440 - KREST-11812 upgrade jaxb-api dependency
ee8dd479 - KREST-12364: update bcpix
PR-427 - KRES-12288 migration semaphore release branches

Schema Registry¶

98a1a4de - update swagger core
PR-2888 - Apply service bot changes to version branches
bafbad8c - DGS-9343 Allow removing enum in JSON Schema to be backward compatible
PR-2874 - DGS-9048: Semaphore Migration of Version Branches
899b38a2 - migrate to semaphore
7f7a7ab1 - add metrics-core to fix the build
d3b1912d - upgrade com.squareup.okio:okio to 3.4.0
c8ab3fd0 - Add log if encoding fails
d81d5a53 - Encode subject names for Maven download plugin
PR-2820 - Cache ProtobufSchema.toDynamicSchema calls (#2818)
PR-2816 - DGS-8960 Fix handling of diamond structure in Protobuf converter

Version 7.0.12¶

Released October 2023

Community Features¶

Common¶

35a1415d - APPSEC-3039: Update jetty and netty
81295738 - update snappy to 1.10.5
aea31af8 - APPSEC-2796: add guava to dependencyManagement

Kafka¶

PR-973 - KSECURITY-2013: Upgraded Jetty to 9.4.53.v20231009 and Netty to 4.1.100.Final
PR-961 - KAFKA-15498: bump snappy-java version to 1.1.10.5 (#14434)

Schema Registry¶

PR-2790 - DGS-8088 Get the mode in scope rather that just for the subject

Version 7.0.11¶

Released September 2023

Community Features¶

Common¶

59440c88 - APPSEC-2855: add generation of SBOMs to maven based Java projects
27a3c1ca - APPSEC-2796: update guava

Kafka¶

PR-942 - KSECURITY-1789: upgrading requests to v2.31.0 (3.0)
PR-935 - KSECURITY-1859: updating netty to 4.1.96 (3.0)

ksqlDB¶

d3f0e149 - fix: fix compile error introduced by merge conflict resolution

REST Proxy¶

Schema Registry¶

PR-2734 - DGS-8112 Allow properties to be omitted during serialization
3f0e1198 - DGS-7927: update io.grpc_grpc-protobuf
PR-2695 - do not reinstantiate objectmapper as it’s expensive (#2686)

Version 7.0.10¶

Released June 2023

Community Features¶

Common¶

PR-530 - Backport log redactor to CP patches.
22a9da57 - APPSEC-2551: Update Prometheus JMX Exporter
5ad03d12 - APPSEC-2551: Update Jackson
PR-514 - Update snakeyaml
b4334b94 - pin jose4j version to match ce-kafka
95ed5124 - include snappy and jetty in the dependency management to match versions with ce-kafka, cleanup jackson imports

Kafka¶

PR-13600 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13526) (#13600)
PR-13596 - KAFKA-14869: Ignore unknown record types for coordinators (KIP-915, P… (#13596)
b9ecd4de - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
PR-13499 - KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable (#13499)
PR-13386 - KAFKA-14809 Fix logging conditional on WorkerSourceTask (#13386)
PR-13273 - KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
PR-911 - KSECURITY-1627 KSECURITY-1556: update snappy and jetty

ksqlDB¶

PR-9810 - Bump minimist and mkdirp
4ce2d5f3 - fix: Replace regex in CommandParser with a split by space
cc0501d8 - fix: Simplify regex in DdlDmlRequestValidators to avoid catastrophic backtracking
aefe6338 - Fix log4j-extensions
08853cd3 - Fix log4j-extensions

REST Proxy¶

ab17aa29 - Fix the overzealous conflict resolution
5be3406e - KREST-9942: Remove SnakeYaml from dependency management
a54dde7e - CNC-366: Update Jetty

Schema Registry¶

0f13c16e - fix checkstyle
20526ee5 - solve conflicts
e0626f64 - Update the test to make it clear to customer
dc6f2096 - update the doc
33659feb - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
3ffd8fc7 - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
1ca2a39d - Update the codes with checking string for http connect timeout
ffba37fe - * changed the config names
4c383ee4 - Apply suggestions from code review
7155f0fb - * checkstyle error
7e2dec9c - * removing unused import
9ab1b0ec - * adding defaults
01c6d7a7 - * Made SR client http connect and read timeout configurable.
89aa28be - * Made SR client http connect and read timeout configurable.
PR-2654 - DGS-7432 Fix ClassCastException when getting params option
PR-2647 - DGS-7422 Optimization: omit compat check during IMPORT
PR-2642 - DGS-7367 Propagate error messages even if not JSON
PR-2616 - DGS-7005 Convert socket timeouts to RetriableException for converters
f45fe28e - DGS-6698: Remove SnakeYaml from the dependencyManagement
PR-2602 - DGS-6929 Ignore invalid Avro defaults for source connectors
PR-2590 - DGS-6701 Fix getSchemaById to match subject if passed
ed4ec914 - DGS-6698: Remove SnakeYaml from the dependencyManagement
PR-2589 - DGS-6613 Ignore invalid Avro defaults in Avro Converter
PR-2588 - DGS-6603 Fix default conversions in JsonSchemaConverter
PR-2693 - DGS-7746 Fix perf regression due to DGS-6331
PR-2684 - DGS-7675 Fix NPE when print.schema.ids=true
1bf961b7 - Update json-schema

Commercial Features¶

Confluent Server¶

add log redactor dependency
CC-18990 : Backport log redactor connect 7.0.x
KSECURITY-1568: Upgrading jose4j
KGLOBAL-2035: Unblock local follower fetch requests when records are appended to mirror leader logs
KSECURITY-1556: updated jetty version
KAFKA-14887: FinalizedFeatureChangeListener should not shut down when
KSECURITY-1460, KSECURITY-1466: Updated snakeyaml, json-smart and nim
KSECURITY-1459: updated golang.org/x/net mod.
KSECURITY-1455: updated golang.org/x/text mod.
KSECURITY-1453: Updated version of golang.org/x/sys
KGLOBAL-3350 Add more logging for persistent connection creation in Outbound connection manager
KSECURITY-1627: Updated snappy-java
KSECURITY-1618: update guava
KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires

Security¶

CIAM-2998 - Log redactor backport for confluent-security-plugins
KREST-10128 Create LicenseManager instance in iteration run instead o

Secret Registry¶

unpin netty, as rest-utils brings updated version

Control Center¶

[MMA-13078] revisit session caching in basic auth
[MMA-13089] Handle large message JSON on topic inspection
[MMA-13042] Fixing log statement for websocket time out
[MMA-13216] update jose4j

MQTT Proxy¶

CC-19108 | Rename isChannelClosed to isAuthenticated.
CC-19108 | Use an atomic reference to avoid publishing message when the channel is closed or marked for closure.

Metadata Service¶

update
Update pom.xml
CIAM-2986 - Older MDS builds are not running all the tests they should be
update comment, clean up whitespaces
another attempt with mina-core
pin mina core in cli, scope mina-core as test in main pom
pin mina-core in testingLdap only
SEC-xyz: Update mina-core

Replicator¶

disable builds of connect-replicator version 5.5.x
used spotbugs version variable instead
use spotbugs annotations instead of findbugs
resolving compile issue
KGLOBAL-3224 removed find bugs usage
remove aws-maven plugin to unblock the downstream validation for common: RCCA-10412, APPSEC-2551

Version 7.0.9¶

Released March 2023

Community Features¶

Common¶

PR-512 - Add snakeyaml to dependency management
7bcaab9e - Fix typo in with method calls
3868e27b - Fix typo
783c5721 - Return the original “version_range” if it isn’t an actual Maven version range
PR-502 - Fix: override maven version plugin’s default versionrange
9350d231 - Update resolver-maven-plugin to 0.6.0
PR-497 - Upgrade Netty to 4.1.86.Final
PR-496 - Bump to 7.4.4

Kafka¶

PR-13245 - KAFKA-14704; Follower should truncate before incrementing high watermark (#13245)
PR-874 - KC-2332: Upgrade netty to 4.1.86
PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
PR-861 - KAFKA-9648: Add configuration to adjust listen backlog size for Accep
PR-12935 - KAFKA-14432: RocksDBStore relies on finalizers to not leak memory (#12935)

ksqlDB¶

PR-9821 - Fix: migrations tool without basic auth works again
PR-9770 - Upgrade netty to 4.1.86 #2
PR-9717 - Fix: change password-based configs to Type.PASSWORD

REST Proxy¶

69ed0cd9 - Remove incorrect synchronization
PR-1108 - Set kafka-schema-registry version in dependency mamanagement
7a7f5dcd - Remove unnecessary ${project.version} tag
PR-1102 - Fix transitive dependency of kafka-schema-registry
PR-1101 - Fix dependency org.yaml_snakeyaml, upgrade to fixed

Schema Registry¶

PR-2577 - DGS-6373 Ignore leading dot when merging Protobuf custom options
PR-2575 - DGS-6331 Handle javaType for oneOfs during JSON deserialization
PR-2568 - DGS-6306 Allow multiple oneofs in Protobuf converter
PR-2550 - DGS-6192 Include default ctx when using subjectPrefix w/wildcard ctx
PR-2540 - DGS-6075 : Add explicit definition of snakeyaml.
PR-2517 - MINOR cherry-pick Protobuf validate optimization to 7.0.x
PR-2519 - DGS-6014 cherry pick Maven plugin enhancement to 5.5.x
PR-2514 - DGS-5950: Allow optional map key in Avro
PR-2474 - Fix handling of Protobuf repeated options
PR-2471 - Preserve metadata associated with primitive types during normalization
PR-2468 - DGS-5567 Handle Protobuf map options in toCanonicalString()

Commercial Features¶

Confluent Server¶

KMETA-478 Enable zk audit logs by default
Fix version comparison in kafkatests (#8922)
KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
KSECURITY-897: Make expiration timeout for ZK ACL change notification path configurable
KC-2328: Update yaml.v2 to 2.2.4
KSECURITY-965: Update git version 1.13.0
KSECURITY-953: Update azure-identity to 1.7.3
KSECURITY-981: Add FeatureZNode zk node path to secure root paths
KC-2333: Update io.netty libraries to 4.1.86.Final
CPKAFKA-3718: Add ObserverReplicasCount metric
KSTORAGE-82: Tiered storage topic deletion support
MINOR: disable use of ConfluentLeaderAndIsrRequest in 5.4.x
[KENGINE-314]: Fix a NPE when FetchSession close.
MINOR: Use LeaderAndIsr rather than ConfluentLeaderAndIsr for 2_4_IV1
KAFKA-9038: [WIP] Allow creating partitions for topics partitions not in reassignment
CONFLUENT: Use single audit log provider for MDS and its hosting broker
CPKAFKA-3730: Fix createTopicPolicy incompatibility in ce-kafka
CPKAFKA-3816: Use CaughtUpReplicasCount for consistency

Security¶

Removed From Supported Operations to follow RBAC Rules
Added fix for unit test
Fixed import
Changed the design of RBAC for Exporters
Removed extra line changes
Added RBAC for Schema linking Phase 1
Remove logging statement
Update netty-codec-http to 4.1.86.Final

Secret Registry¶

Remove unused variables
Update RestService.java

Control Center¶

RCCA-9557 - fix broker flapping issue 7.0 - missed cherry-pick
Increase wait start time for control center in test
Make sure control center stop properly
Change PORT to avoid conflict with TelemetryReporterIntegrationTest
KC-2261 timeout websocket connection via jwt token
Update CODEOWNWERS to match latest definition from master
Refactoring- made code more functional and improved readability.
Updated NotFoundException message in scenario where topicName provided returns 0 no consumergroupdata.
Updated code based on PR feedback.
Updated consumerGroupId API to also support topicName queryParam.
Improving test cases with scenarios of single consumergroup covering multiple topics. 2. updated consumerGroupId to not support topicName queryParam, it is not required.
Adding unit test for CachedConsumerOffsetsResource
Adding support for topicName in query param to return only consumer groups for that topic
Reading password using getPassword instead of getString, this fixes the casting exception
Fixing failing test case, due to merge issue
Renaming Health_check to health check
MMA-12911: Propagate numPartitions and replication factor from CreateTopicsResult
MMA-12912: Disable offset commits, don’t supply group ID
Adding config based functionality to force http1 for armeria health checks
MINOR: Update repo to use mvn-wrapper to speed up builds
Fix codeowners to make c3 default ownens as well
Cherry pick 7746 codewoner

MQTT Proxy¶

Remove the pin of netty version to use the (current) version defined in common

Metadata Service¶

Clean testng
[Fix build] Drop down to testNG 7.5 - which is the last version to support jdk8
Clean up surefire
CIAM-2615 - Bump TestN to 7.7.0
Fix NotNull import
Squashed ‘mk-include/’ content from commit 7df56b0fff
CIAM-2570: Backport CIAM-2183 fix to CP in active 6.1.x
CIAM-2570: Backport CIAM-2183 fix to CP in active 6.0.x
Fixed build error relate to FIPS jar for 7.0.x branch
Increase buffer size for Proxy Servlet

Replicator¶

Fixed deprecated method usage
Changed to 5 sec to honor TASK_SHUTDOWN_GRACEFUL_TIMEOUT_MS_CONFIG
Changed default to 120 sec to match topic config sync
KGLOBAL-3060 added consumer poll timeout ms property

Version 7.0.8¶

Released February 2023

Commercial Features¶

Security¶

Security Fixes

Version 7.0.7¶

Released December 2022

Community Features¶

Common¶

PR-492 - APIF-3112: Upgrade jmx_prometheus_javaagent to 0.17.2.
f5764727 - Merge conflict resolution.
PR-489 - APIF-3117: Upgrade scala to 2.13.10.
PR-485 - APIF-2957: Upgrade Jackson and Protobuf versions.

Kafka¶

PR-845 - CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
PR-842 - DP-9030: Use the new withGradleFile closure
PR-12869 - KAFKA-14382: wait for current rebalance to complete before triggering followup (#12869)
PR-12809 - [KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)
PR-11690 - KAFKA-13599: Upgrade RocksDB to 6.27.3 (#11690)
PR-833 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
a330e221 - Revert “Revert “KAFKA-13673: disable idempotence when config conflicts (#11788)””
c076dd9e - Revert “Revert “KAFKA-13598: enable idempotence producer by default and validate the configs (apache#11691)””
PR-819 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2

ksqlDB¶

PR-9708 - APIF-3117: Unpin scala version.
PR-9697 - APIF-3122: Unpin Protobuf version.

REST Proxy¶

PR-1072 - KREST-7107 Shouldn’t provide overly detailed info or stack traces in responses -

Schema Registry¶

PR-2460 - DGS-5544 Support nulls when using optional fields in proto2
PR-2459 - DGS-5400 Support subjectPrefix containing wildcard context and subject
PR-2450 - Pin Snakeyaml version
PR-2443 - DGS-5423 Check subject when looking up ID across contexts
PR-2440 - Update the Snakeyaml dependency
8e5ce168 - Unpin Protobuf version.
PR-2414 - DGS-5253 Handle cycles when resolving references
PR-2405 - DGS-4754 Handle empty record default at field level
PR-2399 - DGS-5084 Ignore compat check in IMPORT mode
PR-2395 - DGS-4971 Handle map types with enhanced.protobuf.schema.support

Commercial Features¶

Confluent Server¶

CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
DP-9030 - Switch to codeartifact repo
Increase vagrant aws timeout for system test
Bouncycastle Issue fix for connect service [7.0.x]
KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
Revert “Revert “KAFKA-13673: disable idempotence when config conflicts (#11788)””
Revert “Revert “KAFKA-13598: enable idempotence producer by default and validate the configs (#11691)””
Upgrade jackson-databind version to 2.13.4.2
Upgrade bc fips drivers
Upgrade to snakeyaml version 1.32
CIAM-2424 - Name RestClient threads for stackdump identification (#7887)
Upgrade to protobuf version 3.19.6
KGLOBAL-932: Fix timing issues in cluster link integration tests (#5085)
DP-8085 - Migrate to Sempahore self-hosted agent (#6817)
KGLOBAL-935: Fix timing issues with auto mirror creation test and its
KGLOBAL-934: Increase wait time in tests for auto-mirroring to make t
KGLOBAL-943: Recreate local admin client for destination when cluster
CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
RCCA-8564: log a warning if LDAP login fails becuase of network issues

Security¶

Exclude bcpkix-jdk15on as compile dep and add as test dependency
Pin bc-nonfips dep as a test dependency
Pin bouncycastle fips dependency in confluent-security-plugins
Removed bcfips from common
excluded bcfips jars
AUTHN-1087: [7.0.x only] Enforce Netty versions in kafka-rest plugin
AUTHN-1087: Upgrade vertx to 3.9.14
AUTHN-1325: Update bc-fips version to 1.0.2.3
- fixed checkstyle error
- Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas

Secret Registry¶

Exclude Non-FIPS dep and use bc-fips dependency

Control Center¶

MMA-9023: memory bounding for streams applications [5.4.x, master]
Revert “MMA-12206 upgrade io.netty:netty-codec-http”

MQTT Proxy¶

Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
Upgrading netty

Metadata Service¶

CIAM-2392 Bump PSQL Version
Remove spaces
Add back newline
Implement writeTransactionMarkers in DelegateConfluentAdmin
CIAM-2304: Add unit tests for SecurityAdmin

Replicator¶

REPL-2055-: Log WakeupException at debug level
RCCA-7678: Reverse proxy header check added

Version 7.0.6¶

Released August 2022

Community Features¶

Common¶

PR-472 - Upgrade Netty to 4.1.79.Final
PR-464 - Migrate from confluent-log4j to reload4j [5.4.x – 7.0.x]

Kafka¶

PR-792 - Upgrade Jetty to 9.4.48.v20220622
PR-764 - : migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.0)
93a17e7e - : upgrading netty to v4.1.79.Final (3.0) #757

ksqlDB¶

PR-9337 - bugfix: remove log4j from the classpath (#9334)

REST Proxy¶

PR-1049 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-1047 - : Migrate from confluent-log4j to slf4j-reload4j.
PR-350 - Cherry-pick https://github.com/confluentinc/rest-utils/pull/349 to 5.4.x
3d1250af - APIF-2739: Upgrade Jetty to 9.4.48.v20220622.
PR-320 - APIF-2714: Switch from confluent-log4j to reload4j (5.4.x)
PR-329 - APIF-2705: Update jersey version to 2.36.

Schema Registry¶

PR-2359 - DGS-4395 Fix message indexes of normalized Protobuf with map
PR-2351 - DGS-4358 Fix NPE in Protobuf converter for null map value
0577dffb - Add comments
042d3e3c - DGS-4303: Support different stat types in SchemaRegistryMetric
PR-2342 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-2339 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-2336 - DGS-4249 Fix name resolution during Protobuf normalization

Commercial Features¶

Confluent Server¶

Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
KAFKA-14107: Upgrade Jetty version.
Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (7.0.x)
Avoid sending partial updates during LDAP group manager startup
Fix exception handling in RBAC writer coordinator
Upgrading google-cloud-storage to v2.6.1 (7.0.x)
Upgrading aws-java-sdk-s3 to v1.12.268 (7.0.x)
Upgrading netty version to 4.1.79.Final (backport 7.0.x)
Update bouncy castle to 1.70 (back port 7.0.x)
Cherry pick KAFKA-13474 on master to 7.0.x

Security¶

add auth token provider to ksql security extensions
revert pom
add auth token provider to ksql security extensions
AUTHN-1085: Fix NPE in debug log
APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.

Secret Registry¶

Migrate from log4j to reload4j for connect-plugin
CC-17641,CC-17644 Bump jetty version
APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.

Control Center¶

MMA-12201 Fix the mistake in resolving merge conflict
MMA-12201 remove slf4j from transitive deps
MMA-12206 upgrade transitive dependency of netty
MMA-11617: allow for trailing slash in path
MMA-12206 upgrade io.netty:netty-codec-http
MMA-12201 Exclude slf4j in transitive dependency
MMA-12201: Use reload4j version properties from common
MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
MMA-12201 Replace log4j with reload-4j

MQTT Proxy¶

CC-17620: remove dependency on slf4j-log4j12 and confluent-log4j

Metadata Service¶

CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS

Replicator¶

KSECURITY-481: Migrate confluent-log4j to reload4j.

Version 7.0.5¶

Released July 2022

Community Features¶

ksqlDB¶

PR-9260 - fix: Excludes Guava from Guava-retrying in order to manage Guava depe
617df451 - merge conflicts
4ef7a1df - merge conflicts

Schema Registry¶

PR-2331 - DGS-4172 Bound size of Avro datumReader/Writer caches
PR-2329 - DGS-4163 Pass topic to schema formatters
PR-2328 - DGS-4162 Handle refs at different levels in Protobuf converter
c4c0b7f7 - Fix merge issue
PR-2327 - DGS-1648 Allow Protobuf msg fullname to be passed for console producer
5ea29980 - Fix test in Protobuf converter
PR-2321 - DGS-4016 Fix handling of UInt32Value in Protobuf converter
PR-2320 - Revert remove schema in exception
PR-2318 - DGS-3974 Fix StackOverflow with recursive JSON schema in converter

Commercial Features¶

Control Center¶

MMA-11938 Make connect healthcheck path configurable
Deprecate CONTROL_CENTER_STREAMS_RETRIES_CONFIG and remove default usage

Metadata Service¶

CIAM-2141 - Remove usages of MiniKdcWithLdapService

Version 7.0.4¶

Released June 2022

Community Features¶

Common¶

Upgrading gson
PR-456 - DGS-3674 Upgrade gson to 2.8.6
PR-446 - Upgrade jackson related packages
PR-445 - Unify protocol buffer version.

Kafka¶

PR-707 - KSECURITY-220: Update jackson packages to 2.13.2
PR-11980 - fix: make sliding window works without grace period (#kafka-13739) (#11980)
PR-12006 - KAFKA-13794: Follow up to fix producer batch comparator (#12006)
PR-11991 - KAFKA-13794; Fix comparator of inflightBatchesBySequence in TransactionManager (#11991)
PR-11908 - KAFKA-13748: Do not include file stream connectors in Connect’s CLASSPATH and plugin.path by default (#11908)
PR-11966 - KAFKA-13418: Support key updates with TLS 1.3 (#11966)
PR-11946 - KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (#11946)
PR-11939 - KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)
PR-11933 - KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (#11933)
PR-11909 - KAFKA-13750; Client Compatability KafkaTest uses invalid idempotency configs (#11909)
PR-11891 - KAFKA-13727; Preserve txn markers after partial segment cleaning (#11891)
8e30984f - Bump version to 3.0.1

ksqlDB¶

PR-8968 - fix: preserve old schema behavior for protobuf wrapped primitives (7.0.x)

REST Proxy¶

b22efb66 - Patch gson to be 2.9.0
PR-1016 - KREST-5096 Fix NPE in Topics Resource when no key, or no value specified
PR-942 - Fix(#341): Remove deadlock when rebalancing by multiple consumer instances
PR-1026 - Backport consumer deadlock fix to 5.4.x

Schema Registry¶

PR-2306 - DGS-3783 Allow same schema w/diff ids during IMPORT
PR-2299 - DGS-3141 - adding new v1/metadata/version endpoint
PR-2294 - APPSEC-1156: Patch Gson to be 2.9.0
PR-2290 - DGS-3674 Upgrade to Protobuf 3.19.4
PR-2288 - DGS-3672 Allow CachedSchemaRegistryClient to register same schema
46b18280 - Fix checkstyle issues
PR-2287 - Merge Conflict Resolution (from 6.2.x to 7.0.x)
PR-2285 - Merge Conflict Resolution (from 5.4.x to 5.5.x)
PR-2283 - DGS-3659 Handle null default in JSON Schema converter
PR-2282 - DGS-3654 Ensure isKey is passed in deserializers
bbabd4cd - remove kotlin-stdlib-common dep
a5c4a3b7 - Override dependency of kotlin-scripting-compiler-embeddable to Kotlin version
Bump kotlin componenets version
96471721 - Upgrade kotlin to 1.6.0
PR-2259 - Fix upgrade to Jackson 2.13
PR-2257 - DGS-3548 Optimize cache for JSON Schema converter
PR-2253 - DGS-3535: Remove enforcement of building with JDK8
PR-2241 - DGS-3484 Allow NaN when parsing JSON
PR-2223 - DGS-3342 Add reserved fields for Protobuf enums
PR-2227 - DGS-3417 Fix default value conversions in JsonSchemaConverter

Commercial Features¶

Confluent Server¶

build: kafka system tests should support larger EBS volumes for newer instances (#6371)
Upgrade Gson
KENGINE-152: do not use idempotent producer for license log
KGLOBAL-1108: Fix OffsetsForLeaderEpoch version for cluster link source version < 2.8
use correct cp urls in dockerfile - 7.0.x
Disable idempotent producer by default in Connect centralized licenses (#6193)
KSECURITY-221: Update jackson packages to 2.13.2 version
MINOR: Only call getSnapshot once per metric when processing Yammer metrics
KSECURITY-195: Update org.jetbrains.kotlin packages to 1.6.0
KSECURITY-198: Update protobuf-java packages to 3.19.4
Backport of KMETA-8 to 7.0.x
Fix broken tests because of KAFKALESS-1032 cherry-pick
KAFKALESS-900: Fix replication factor in BrokerFailureDetectorTest and use monotonic time
KAFKALESS-1032: Use wall clock to prune task history
CNK-1774: SocketServerTest.testIdleConnection fails
KAFKALESS-998: Disable idempotence of all SBC used producers
Revert “KAFKA-13598: enable idempotence producer by default and validate the configs (apache#11691)”
Revert “KAFKA-13673: disable idempotence when config conflicts (#11788)”
Bump version to 3.0.1

Control Center¶

Bump maven-shade-plugin to 3.3.0
Fix build break
Override transitive dependency for analytics library of kotlin-stdlib-common
Add exclusion to override dependency kotlin-stdlib-jdk8
LOG_DIR to match rest of confluent components
MMA-11694: Add retry logic for ControlCenterIntegrationTest
MMA-11096, MMA-11718, MMA-11820: distinguish deserializers for key and value

Metadata Service¶

CIAM-2020 - Fix null string in MDSLoggingFilter
CIAM-2019 - Fix 6.x MDS builds from Jackson update
MDS-38 MDS-39 - Make MDSLoggingFilter case-insensitive
MINOR SEC-3087 SEC-3028 Avoid NPE in integration test
RCCA-6743: Test that we can allow rest client to disable hostname verification
This release version includes a fix to prevent stack trace exposure from getting surfaced in HTTP responses for malformed requests.

Version 7.0.3¶

Released April 2022

Commercial Features¶

Metadata Service¶

MDS-38 MDS-39 - Make MDSLoggingFilter case-insensitive

Version 7.0.2¶

Released March 2022

Community Features¶

Common¶

PR-428 - Upgrade Netty
8333c7ad - use 3.17.3 protobuf for version after 7.0.x
605b9b9d - use 3.17.0 protobuf for version after 5.5.x
PR-422 - MMA-11603: Add centralized protobuf version from 5.3.x onwards
PR-421 - Update confluent-log4j version
PR-413 - Remove redundant Netty Version definition.
PR-414 - Upgrade Avro to 1.11.0
PR-412 - CVE-2021-43797: Upgrade Netty
PR-411 - Upgrade Log4j2 to 2.17.1
PR-403 - Upgrade Log4j2

Kafka¶

PR-11742 - KAFKA-13636: Fix for the group coordinator issue where the offsets are deleted for unstable groups (#11742)
PR-11691 - KAFKA-13598: enable idempotence producer by default and validate the configs (#11691)
PR-11726 - KAFKA-13637: Use default.api.timeout.ms as default timeout value for KafkaConsumer.endOffsets (#11726)
PR-11689 - KAFKA-12841: Fix producer callback handling when partition is missing (#11689)
PR-653 - KC-1933; Upgrade confluent-log4j to 1.2.17-cp2.2
PR-11671 - KAFKA-13388; Kafka Producer nodes stuck in CHECKING_API_VERSIONS (#11671)
PR-11607 - KAFKA-13544: fix FinalizedFeatureChangeListener deadlock (#11607)
PR-11552 - KAFKA-13488: Producer fails to recover if topic gets deleted midway (#11552)
PR-11563 - KAFKA-13461: Don’t re-initialize ZK client session after auth failure if connection still alive (#11563)

ksqlDB¶

REST Proxy¶

PR-984 - KREST-4608 port changes to 6.2.x onwards
PR-983 - KREST-4605 Merge the master changes to back levels too
PR-982 - KREST-4577 Port changes to 6.2.x onwards
PR-980 - Disable producer idempotency until KAFKA-13668 is fixed.
PR-963 - KREST-4113 Don’t delete all ACLs if an empty DELETE request is received.
PR-969 - KREST-3836 update maven to pull from https
PR-962 - KREST-4112 Fix Create ACL request example.
5514b21c - MMA-11468: Add option to enable nosniff protection
19657b29 - Remove Redundant Netty Definition
PR-305 - Remove Redundant Netty Definition
PR-302 - APIF-2027: Upgrade jetty-server to 9.4.44.v20210927.
PR-301 - APIF-2005: Change the Maven repository protocol: http → https

Schema Registry¶

Note: The Protobuf Converter will now convert Protobuf wrapper types to primitives instead of structs. To retain the old behavior, set wrapper.for.raw.primitives=false.

PR-2191 - DGS-3109 Ensure context lookups use unqualifed subjects
PR-2187 - DGS-3066 Handle unions in AvroSchema.metaEquals
PR-2186 - DGS-3061 Only cache top-level schemas in AvroData
PR-2182 - MINOR: Update generated enum classes in protobuf-serializer tests
PR-2175 - DGS-2990: Backport PR 2174 to 5.2.x
PR-2174 - DGS-2990 Removing message value from exception to avoid logging custo
PR-2173 - DGS-2988: Disable producer idempotence in KafkaStore
PR-2169 - DGS-2965 Make context wildcard the default subjectPrefix
PR-2168 - DGS-2958 Search contexts when retrieving schema
PR-2164 - DGS-2937 Use default scale of 0 in Avro converter
PR-2155 - DGS-2885 Ignore complex custom Protobuf options
PR-2151 - DGS-2875: Return correct default value for connect union
PR-2148 - DGS-2830 Check for addition/removal of required Protobuf fields
PR-2147 - DGS-2826 Add Avro UUID conversion; minor optimization
PR-2140 - DGS-2814 Ensure id generation is thread-safe in converters
PR-2128 - MINOR: Update 6.1.x OpenAPI Spec
PR-2126 - DGS-2719: add explicit dependency for kotlin-scripting-jvm
PR-2125 - DGS-2727 Handle multiple refs to same Protobuf enum
PR-2124 - DGS-2722 Use identity comparison for DatumReader/Writer caches
PR-2122 - DGS-2707 Ensure Protobuf reserved range uses exclusive end
PR-2119 - DGS-2701 Pass topic to context strategy during deserialization
PR-2118 - DGS-2700 Fix doubling of context delimiter w/custom context strategy
PR-2117 - DGS-2680 Use exact schema returned by ID when looking up version
PR-2113 - DGS-1753 Add flag to support optional fields in Protobuf converter
PR-2111 - DGS-1654 Generate synthetic oneofs for proto3 optional
PR-2108 - DGS-2646 Support name method on ProtobufSchema with only enums
PR-2102 - DGS-2621 Support raw primitives in the Protobuf converter
PR-2107 - DGS-2636 Collect properties for allOf in JSON Schema converter
PR-2106 - Exclude log4j2 transitive dependencies
PR-2104 - Exclude log4j2 dependency
PR-2100 - Backport PR2039 to 5.5.x branch
PR-2103 - DGS-2626 Handle combined schema with title and/or object
PR-2101 - feat: Update wire-schema version to 4.0.0

Commercial Features¶

Confluent Server¶

Ignore failing quota_test (#5651)
KCFUN-243: SaslAuthenticatorTest.[testPublicCredential, testSslClientAuthRequestedForSaslSslListener, testSslClientAuthRequiredForSaslSslListener] fail frequently
Upgrade TCNative version
Cherry-pick “Disbaled bubbling errors from JsonSerializer and Deserializers for connect that print the message in stack trace (#5558)” onto 7.0.x
Cherrypick to 7.0.x “KSTORAGE-1915: Check if usage of S3’s GetBucketLocation API can be eliminated”
[CLI-1430] Add GCM functionality to DecryptionEngine (#5563)
CONFLUENT: Check version to select ZK CLI class name
CONFLUNET: Fix failing multitenant RBAC tests
CONFLUENT: Disable idempotence since retries is force to 0 in KafkaStoreConfig
CONFLUENT: Add junitJupiter lib dependency to ce-broker-plugins, ce-auth-providers tests
KGLOBAL-1025: Auto mirror create should exclude all linked topics. (#
KC-1504: fix license topic consumer endOffsets TimeoutException
CPKAFKA-7648: retry fetching license topic metadata in LicenseStore
[DO NOT MERGE] clientMetadata
KC-1933; Upgrade confluent-log4j to 1.2.17-cp2.2 (7.0.x)
KGLOBAL-1009: Fix testDeleteAutoMirroredTopics intermittent failures
KGLOBAL-923: Allow mirror topic offsets to be committed under a transaction
CONFLUENT: Update zookeeper functions to use provided version
KAFKA-9648: Add configuration to adjust listen backlog size for Accep

Security¶

Added fix for empty bulkAuthorize requests
Disable producer idempotence in SR security plugins
DG-1249, DG-1257, DG-1353, DG-1423, DG-1462, DG-1491, DGS-2950: Fix testGetLastOffsetWithExistingMessages
APPSEC-827:Upgrade bc-fips
DGS-2730: Use fail-open logic in license validation

Secret Registry¶

Control Center¶

MMA-11637: use AdminClient to create topic
MMA-11603: Update protobuf version
MMA-11121: Fix compatibility issues for partition size metric
MMA-11478: enable cors check on websocket
MMA-11451: fix @JsonProperty usage to yield expected output
(MINOR): update armeria version and exclude netty artifacts
(MINOR): fix merge
MMA-11468: Add option to enable nosniff protection
MMA-10986: fix health check resource for connect
MMA-10571: add SR basic auth to armeria healthcheck
(MINOR): fix armeria test
MMA-9126: Fix broken RocksDB stats endpoint
MMA-9292: unit test for ClusterDataExporter extraction of security protocol
MMA-9292: control-center-export script reads security.protocol from prop file
MMA-10114: Change default values for command topic configuration
MMA-10668: reduce the broker id retention to 30ms
MMA-11374: fix multiple entries with same key
MMA-9479: Remove JDK dependency from control center stop script
DGS-2570: call /scope instead of /permissions to get SR scope in RbacServiceVisibilityFilter
Added support for bigint format for numbers coming from ksql JSON
MMA-11702: Fixed the “New action” button on the Alerts Actions list page

Known issues

In Kafka 3.0/3.1, producer clients turned idempotency on by default. We have not tested the tier topic producer in this mode and it doesn’t require the idempotent producer. Using it increases the blast radius of issues with transaction coordinators to affect tiering.

MQTT Proxy¶

CCMSG-1677 handle enable idempotence producer by default by explicitly setting the idempotence based on delivery guarantees
CCMSG-1677 handle enable idempotence producer by default by explicitly setting the idempotence based on delivery guarantees
Fixing error in repeated manual merge.
Fixing error in repeated manual merge.
[CC-16389] Changing the maven URL in pom.xml from http to https
Revert “[CC-16389] Changing the maven URL in pom.xml from http to https”
[CC-16389] Changing the maven URL in pom.xml from http to https

Metadata Service¶

MDS-33 - Add AES GCM enable variable
CIAM-1761 - Update postgres db driver to 42.3.3 for cve issue
CIAM-1737 Bump PSQL Version
SEC-238 MDS Test : Silent Proxy of Writes

Version 7.0.1¶

Released November 2021

Community Features¶

Common¶

PR-383 - Upgrade Netty to 4.1.68.Final

Kafka¶

PR-11439 - KAFKA-13406: Skip assignment validation for built-in cooperativeStickyAssignor (#11439)
PR-11323 - KAFKA-12226: Commit source task offsets without blocking on batch delivery (#11323)
PR-10563 - KAFKA-12487: Add support for cooperative consumer protocol with sink connectors (#10563)
PR-11429 - KAFKA-13396: Allow create topic without partition/replicaFactor (#11429)
PR-11463 - KAFKA-13430: Remove broker-wide quota properties from the documentation (#11463)
PR-8844 - KAFKA-9887: Fix failed task or connector count on startup failure (#8844)
PR-11426 - KAFKA-13391: Don’t fsync directory on Windows OS (#11426)
5b7824d2 - Upgrade Netty version to 4.1.68.final
PR-10649 - KAFKA-12762: Use connection timeout when polling the network for new connections (#10649)
PR-11174 - KAFKA-9747: Creating connect reconfiguration URL safely (#11174)
PR-11337 - KAFKA-13309: Fix InMemorySessionStore#fetch/backwardFetch order issue (#11337)
PR-11347 - KAFKA-13296: Warn if previous assignment has duplicate partitions (#11347)
PR-11351 - KAFKA-13315: Log layer exception during shutdown that caused an unclean shutdown (#11351)
PR-11344 - KAFKA-13312; NetworkDegradeTest#test_rate should wait until iperf server is listening (#11344)
PR-11080 - KAFKA-13149; Fix NPE when handling malformed record data in produce requests (#11080)
PR-11283 - KAFKA-13249: Always update changelog offsets before writing the checkpoint file (#11283)
PR-11319 - KAFKA-13288; Include internal topics when searching hanging transactions (#11319)

ksqlDB¶

PR-8389 - Fix: Upgrade Netty version to 4.1.68.final

REST Proxy¶

PR-924 - KREST-1936: Remove mask filtering of producer configs.
PR-292 - Upgrade Netty Version

Schema Registry¶

PR-2090 - DGS-2415: Reject subjects with control chars
PR-2085 - Fix widening -> narrowing issue in casting
PR-2082 - DGS-2487: Adding missing cache to schema registry client
PR-2083 - DGS-2428: Minor rename to normalizeSchemas
PR-2079 - DGS-2507: Use fullName if enhanced.protobuf.schema.support=true
PR-2080 - DGS-2506: Add reference.lookup.only for Protobuf serializer
PR-2064 - DGS-2428: Add config to ignore ordering when registering/looking up schema
PR-2075 - DGS-2491: Ensure ProtobufSchema.name returns qualified name
PR-2070 - DGS-2401: Add converter flag to allow old doc/default behavior
PR-2039 - Fixes #1930: Support Jakarta EE 9 by eliminating dependencies on jersey ey-common and jakarta.ws.rs in schema-registry-client. (#1933)
PR-2036 - Fix default value for array of records
PR-2068 - Demonstrate wrong cache hit for union type reader schema
PR-2065 - DGS-2293 Add serializer config to specify schema ID (#2048)
PR-2061 - DGS-2417 Change moving Protobuf messages as compatible
PR-2058 - DGS-2393: Fix special chars in options for oneof/enum

Commercial Features¶

Confluent Server¶

KGLOBAL-863: Fix –pending-stopped-only to print topics only in the PENDING_STOPPED state and with partitions in the PENDING_STOPPED state (#4933)
KGLOBAL-890: Add dedicated admin client for dest reverse connections to avoid timeouts with unavailable links
KGLOBAL-891: Only display topic names in exception message while dele
KENGINE-49: Cache ReplicaManager.fetchPartitionPruningEnabled to avoid lock contention
KGLOBAL-923: Don’t add mirror partitions to transactions, fail as invalid request
KGLOBAL-913: Disable auto-topic creation in cluster link tests to workaround license timing issues
KGLOBAL-912: Filter out excluded groups when committing offsets for mirror promotion
KGLOBAL-712: Fixing bug that didn’t update task ms configs
KGLOBAL-845: Close cluster link network client if admin client creation fails due to invalid config
KGLOBAL-721: Fix output error for kafka-cluster-links –list
KGLOBAL-827: Fix MirrorCommand output for partitions without leader, source offset for stopped mirror
KGLOBAL-215: Source origination link system test
KGLOBAL-818: Handle NOT_CONTROLLER error for initiateReverseConnection
KGLOBAL-801: Don’t include topic filters from source links for validating auto-mirroring overlap
KGLOBAL-709: Receive buffer auto tuning for cluster linking connections
KGLOBAL-499: Add a link config to specify the topic configs to sync
SEC-2731: Improve Ldap handling for stale connections (#4497)

Control Center¶

(MINOR): Fix SR test due to SR method signature change
MMA-11217: Use test containers for integration test
(MINOR): MMA-11236: Clean and reorganize checkstyle suppressions

Metadata Service¶

SEC-2805: Manually tested fix for MDS forwarding logic to handle unicode and request with query params

Replicator¶

Upgrade jakarta.el:jakarta.el-api
REPL-1843: Track uncommited records given to Connect framework.

Confluent Platform Docker Images¶

Confluent Platform Docker images formerly shipped with world writable data and configuration directories (chmod a+w). Those permissions are now removed, and adjustments made to make those directories writable from the image’s appuser USER.
Confluent Platform Docker images are now built with yum --nodocs ... to prevent installing Red Hat Documentation. This slims down the images slightly. Confluent documentation from Confluent packages are still installed.

Version 7.0.0¶

Released October 2021

Community Features¶

Common¶

PR-381 - DGS-2161: Update netty version to 4.1.65.Final
PR-376 - Upgrade Jackson to 2.12.3.
PR-361 - Upgrade bcprov-jdk15on to 1.68
PR-365 - Upgrade log4j2 to 2.13.2
PR-371 - KAFKA-12756: Update ZooKeeper to v3.6.3
PR-309 - SEC-1789: Upgrade bouncycastle to 1.68

Kafka¶

d3cebd79 - DGS-2161: Update netty version to 4.1.65.Final
PR-11301 - KAFKA-13276: Prefer KafkaFuture in admin Result constructors (#11301)
PR-11294 - KAFKA-13266; InitialFetchState should be created after partition is removed from the fetchers (#11294)
PR-11300 - KAFKA-13258/13259/13260: Fix error response generation (#11300)
PR-11308 - KAFKA-13277; Fix size calculation for tagged string fields in message generator (#11308)
PR-11307 - KAFKA-13262: Remove final from MockConsumer.close() and delegate implementation (#11307)
PR-11295 - KAFKA-13270: Set JUTE_MAXBUFFER to 4 MB by default (#11295)
PR-11267 - KAFKA-13234; Transaction system test should clear URPs after broker restarts (#11267)
PR-11264 - KAFKA-13231; TransactionalMessageCopier.start_node should wait until the process if fully started (#11264)
PR-11164 - KAFKA-13155; Fix concurrent modification in consumer shutdown (#11164)
PR-11231 - KAFKA-13214; Consumer should not reset state after retriable error in rebalance (#11231)
PR-11239 - KAFKA-13219: BrokerState metric not working for KRaft clusters (#11239)
PR-11083 - KAFKA-13010: Retry getting tasks incase of rebalance for TaskMetadata tests (#11083)
PR-11184 - KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
PR-11216 - KAFKA-13198: Stop replicas when reassigned (#11216)
PR-11221 - KAFKA-13207: Skip truncation on fetch response with diverging epoch if partition removed from fetcher (#11221)
17e2b249 - KAFKA-13209: Upgrade jetty-server
PR-11191 - KAFKA-13173; Ensure KRaft controller handles concurrent broker expirations correctly (#11191)
PR-11189 - KAFKA-13161; Update replica partition state and replica fetcher state on follower update (#11189)
PR-11194 - KAFKA-12779: rename namedTopology in TaskId to topologyName (#11194)
PR-11179 - KAFKA-13165: Validate KRaft node id, process role and quorum voters (#11179)
PR-11171 - KAFKA-13132; Upgrading to topic IDs in LISR requests has gaps introduced in 3.0 (part 2) (#11171)
PR-11177 - KAFKA-13167; KRaft broker should send heartbeat immediately after starting controlled shutdown (#11177)
PR-11178 - KAFKA-13168: KRaft observers should not have a replica id (#11178)
PR-11168 - KAFKA-13160: Fix bug in BrokerMetadataPublisher handling of default resources (#11168)
PR-10602 - KAFKA-12724: Add 2.8.0 to system tests and streams upgrade tests. (#10602)
PR-11166 - KAFKA-13159 Enable additional transaction system tests in KRaft (#11166)
PR-11165 - Enable transactions_test.py system test in kraft mode (#11165)
PR-10909 - KAFKA-12158; Better return type of RaftClient.scheduleAppend (#10909)
PR-11116 - KAFKA-13114; Revert state and reregister raft listener (#11116)
PR-11145 - KAFKA-13151; Disallow policy configs in KRaft since they are not yet supported (#11145)
PR-11126 - KAFKA-13132; Ensure topicId is updated on replicas even when the leader epoch is unchanged (#11126)
PR-11131 - KAFKA-13137; KRaft Controller Metric MBean names incorrectly quoted (#11131)
PR-10693 - KAFKA-12625: Fix the NOTICE file (#10693)
PR-11135 - KAFKA-13143; Remove Metadata handling from ControllerApis (#11135)
PR-10915 - KAFKA-13041: Enable connecting VS Code remote debugger (#10915)
PR-11136 - KAFKA-13141; Skip follower fetch offset update in leader if diverging epoch is present (#11136)
PR-11098 - KAFKA-13099; Transactional expiration should account for max batch size (#11098)
PR-11134 - KAFKA-12851: Fix Raft partition simulation (#11134)
PR-11132 - KAFKA-13139: Empty response after requesting to restart a connector without the tasks results in NPE (#11132)
PR-11002 - KAFKA-13026: Idempotent producer (KAFKA-10619) follow-up testings (#11002)
PR-11109 - KAFKA-13113; Support unregistering Raft listeners (#11109)
PR-11057 - KAFKA-13008: Try to refresh end offset when partitionLag returns empty (#11057)
PR-11053 - KAFKA-13015: Ducktape System Tests for Metadata Snapshots (#11053)
PR-11113 - KAFKA-13128: wait for all keys to be fully processed in #shouldQueryStoresAfterAddingAndRemovingStreamThread (#11113)
PR-11118 - KAFKA-13127; Fix stray topic partition deletion for kraft (#11118)
PR-11108 - KAFKA-13116: Fix message_format_change_test and compatibility_test_new_broker_test failures (#11108)
PR-11115 - KAFKA-13129: replace describe topic via zk with describe users (#11115)
PR-10921 - KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread (#10921)
b894e9ec - KAFKA-13104: Controller should notify raft client when it resigns #11082
PR-11084 - KAFKA-13100: Create KRaft controller snapshot during promotion (#11084)
PR-10811 - KAFKA-12598: ConfigCommand should only support communication via ZooKeeper for a reduced set of cases (#10811)
PR-11078 - KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724) (#11036) (#11078)
PR-11016 - KAFKA-13058; AlterConsumerGroupOffsetsHandler does not handle partition errors correctly. (#11016)
637c0b0a - KAFKA-13090: Improve kraft snapshot integration test
d375c309 - KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
PR-11071 - KAFKA-13098: Fix NoSuchFileException during snapshot recovery (#11071)
PR-11026 - KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11026)
PR-11065 - Fix perf regression on LISR requests by asynchronously flushing the partition.metadata file (#11056) (#11065)
PR-10952 - KAFKA-12257; Consumer mishandles topics deleted and recreated with the same name (#10952)
PR-11059 - KAFKA-12930,KAFKA-12929: Deprecate Java 8 and Scala 2.12 (#11059)
PR-10753 - KAFKA-12803: Support reassigning partitions when in KRaft mode (#10753)
PR-10941 - KAFKA-10847: Remove internal config for enabling the fix (#10941)
PR-11035 - KAFKA-13072: Make RemoveMembersFromConsumerGroupHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11035)
PR-11021 - KAFKA-13062: Make DeleteConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11021)
PR-11022 - KAFKA-13063: Make DescribeConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11022)
PR-11019 - KAFKA-13059: Make DeleteConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11019)
PR-10877 - KAFKA-12925: adding presfixScan operation for missed implementations (#10877)
2abe6851 - KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
PR-8259 - KAFKA-7421: Ensure that the Connect PluginClassLoader and DelegatingClassLoader are truly parallel capable and resolve deadlock occurrences (#8259)
PR-11007 - KAFKA-10589; Rename kafka-replica-verification CLI command line arguments for KIP-629 (#11007)
PR-11008 - KAFKA-10588; Rename kafka-console-consumer CLI command line arguments for KIP-629 (#11008)
0707a8d3 - KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
PR-10985 - KAFKA-12984: make AbstractStickyAssignor resilient to invalid input, utilize generation in cooperative, and fix assignment bug (#10985)
PR-11032 - KAFKA-13073: Fix MockLog snapshot implementation (#11032)
PR-11031 - KAFKA-13067 Add internal config to lower the metadata log segment size (#11031)
PR-11040 - KAFKA-13078: Fix a bug where we were closing the RawSnapshotWriter to early (#11040)
PR-11041 - KAFKA-13080: Direct fetch snapshot request to kraft controller (#11041)
PR-10986 - KAFKA-12983: reset needsJoinPrepare flag before rejoining the group (#10986)
72159ea8 - Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
PR-11011 - KAFKA-13051; Require principal builders implement KafkaPrincipalSerde and set default (#11011)
PR-11003 - KAFKA-12360: Document new time semantics (#11003)
PR-10935 - KAFKA-13003: In kraft mode also advertise configured advertised port instead of socket port (#10935)
PR-10794 - KAFKA-12677: Return not_controller error in envelope response itself in KRaft mode (#10794)
PR-11013 - KAFKA-13056; Do not rely on broker for snapshots if controller is co-resident (#11013)
PR-11012 - KAFKA-13057; Add KRaft “broker” to several RPC’s listeners (#11012)
PR-11000 - KAFKA-13045: Adding a test for batched offsetFetch requests with one group repeating (#11000)
PR-9541 - KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message (#9541)
PR-11010 - KAFKA-13053; Bump kraft frame version for incompatible changes from 2.8 (#11010)
PR-10560 - KAFKA-12660; Do not update offset commit sensor after append failure (#10560)
PR-10903 - KAFKA-13023: make “range, cooperative-sticky” as the default assignor in V3.0 (#10903)
PR-8571 - KAFKA-7613: Enable -Xlint:rawtypes for connect, fixing warnings (#8571)
PR-10962 - KAFKA-12234: Implement request/response for offsetFetch batching (KIP-709) (#10962)
PR-10931 - KAFKA-12998: Implement broker-side KRaft snapshots (#10931)
258ed5a7 - KAFKA-12155: Metadata log and snapshot cleaning #10864
PR-10975 - KAFKA-13035 updated documentation for connector restart REST API to … (#10975)
PR-10974 - KAFKA-12979; Implement command to find hanging transactions (#10974)
PR-10960 - KAFKA-12981; Ensure LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are read/updated in sync (#10960)
PR-10971 - KAFKA-12992; Make kraft configuration properties public (#10971)
PR-9878 - KAFKA-9687: KIP-707: Add KafkaFuture.toCompletionStage() (#9878)
PR-10965 - KAFKA-13029; Set appropriate fields for FindCoordinatorRequest based on version (#10965)
PR-10932 - KAFKA-12958: add an invariant that notified leaders are never asked to load snapshot (#10932)
PR-10919 - KAFKA-12985: Upgrade jetty to 9.4.42 (#10919)
PR-10805 - KAFKA-12436: Deprecate MirrorMaker v1 (KIP-720) (#10805)
PR-10854 - KAFKA-12717: Remove internal Connect converter properties (KIP-738) (#10854)
51796bcd - KAFKA-10587; Rename kafka-mirror-maker CLI command line arguments for KIP-629
PR-10946 - KAFKA-12997: Expose the append time for batches from raft (#10946)
PR-10917 - KAFKA-10847: improve throughput of stream-stream join with spurious left/outer join fix (#10917)
PR-10813 - KAFKA-9559: Change default serde to be null (#10813)
PR-10896 - KAFKA-12964: Collect and rename snapshot files prior to async deletion. (#10896)
PR-10940 - KAFKA-13007; KafkaAdminClient getListOffsetsCalls reuse cluster snapshot (#10940)
PR-10743 - KAFKA-12663: Update FindCoordinator to support batch lookups (KIP-699) (#10743)
PR-9497 - KAFKA-10619: Configure producer with idempotence and acks all by default (KIP-679) (#9497)
PR-10652 - KAFKA-9726: Add IdentityReplicationPolicy to MirrorMaker2 (#10652)
PR-10949 - KAFKA-13019: Add MetadataImage and MetadataDelta classes for KRaft Snapshots (#10949)
PR-10936 - KAFKA-13002: listOffsets must downgrade immediately for non MAX_TIMESTAMP specs (#10936)
PR-10892 - KAFKA-13011; Update deleteTopics Admin API (#10892)
PR-10822 - KAFKA-4793: Connect API to restart connector and tasks (KIP-745) (#10822)
PR-10812 - KAFKA-12863: Configure controller snapshot generation (#10812)
PR-10926 - KAFKA-8613: New APIs for Controlling Grace Period for Windowed Operations (#10926)
PR-10918 - KAFKA-12756: Update ZooKeeper to v3.6.3 (#10918)
PR-10939 - Avoid increasing app ID when test is executed multiple times (#10939)
PR-10221 - KAFKA-12379: Allow configuring the location of the offset-syncs topic with MirrorMaker2 (#10221)
PR-10899 - KAFKA-12952 Add header and footer records for raft snapshots (#10899)
PR-10763 - KAFKA-12520: Ensure log loading does not truncate producer state unless required (#10763)
PR-10930 - KAFKA-12996; Return OFFSET_OUT_OF_RANGE for fetchOffset < startOffset even for diverging epochs (#10930)
PR-10858 - KAFKA-12926: ConsumerGroupCommand’s java.lang.NullPointerException at negative offsets while running kafka-consumer-groups.sh (#10858)
PR-10913 - KAFKA-12631; Implement resign API in KafkaRaftClient (#10913)
PR-10824 - KAFKA-12718: SessionWindows are closed too early (#10824)
PR-10893 - KAFKA-12909: add missing tests (#10893)
PR-10894 - KAFKA-12951: restore must terminate for tx global topic (#10894)
PR-10923 - KAFKA-12976; Remove UNSUPPORTED_VERSION error from delete topics call (#10923)
PR-10916 - KAFKA-12938: Fix and reenable testChrootExistsAndRootIsLocked test (#10916)
PR-10922 - KAFKA-12790: Remove SslTransportLayerTest.testUnsupportedTlsVersion (#10922)
PR-10840 - KAFKA-12849: KIP-744 TaskMetadata ThreadMetadata StreamsMetadata as API (#10840)
PR-10760 - KAFKA-12541; Extend ListOffset to fetch offset with max timestamp (KIP-734) (#10760)
PR-10900 - KAFKA-12967; KRaft broker should forward DescribeQuorum to controller (#10900)
PR-10190 - KAFKA-12336 Custom stream naming does not work while calling stream[K… (#10190)
PR-10879 - KAFKA-12991; Fix unsafe access to AbstractCoordinator.state (#10879)
PR-10878 - KAFKA-12898; Owned partitions in the subscription must be sorted (#10878)
PR-10841 - KAFKA-12482 Remove deprecated rest.host.name and rest.port configs (#10841)
PR-10862 - KAFKA-12928: Add a check whether the Task’s statestore is actually a directory (#10862)
PR-10883 - KAFKA-12949; Match null when eventQueue poll hits eventTimeoutMs in TestRaftServer (#10883)
PR-10814 - KAFKA-12888; Add transaction tool from KIP-664 (#10814)
PR-10869 - KAFKA-10546: Deprecate old PAPI (#10869)
PR-10834 - KAFKA-12904: Corrected the timeout for config validation REST API resource (#10834)
PR-10336 - KAFKA-12483: Enable client overrides in connector configs by default (KIP-722) (#10336)
PR-10335 - KAFKA-12484: Enable Connect’s connector log contexts by default (KIP-721) (#10335)
PR-10377 - KAFKA-12515; Feature support should be ignorable in ApiVersion response (#10377)
PR-10818 - KAFKA-12889: log clean relative index range check of group consider empty log segment to avoid too many empty log segment left (#10818)
PR-10880 - KAFKA-12870; Flush in progress not cleared after transaction completion (#10880)
PR-10902 - KAFKA-12837; Process entire batch reader in the BrokerMetadataListener commit handler (#10902)
PR-10733 - KAFKA-12816 Added tiered storage related configs including remote log manager configs. (#10733)
PR-10754 - KAFKA-12835: Topic IDs can mismatch on brokers (after interbroker protocol version update) (#10754)
PR-10836 - KAFKA-12906; Added RecordDeserializationException containing partition and offset (#10836)
PR-10872 - KAFKA-12945: Remove port, host.name and related configs in 3.0 (#10872)
PR-10588 - KAFKA-12662: add unit test for ProducerPerformance (#10588)
PR-10863 - KAFKA-12890; Consumer group stuck in CompletingRebalance (#10863)
PR-10701 - KAFKA-10437: Update WordCount examples to use new PAPI (#10701)
PR-10888 - KAFKA-12955: Fix LogLoader to pass materialized view of segments for deletion (#10888)
PR-10861 - KAFKA-12909: disable spurious left/outer stream-stream join fix for old JoinWindows API (#10861)
PR-10867 - KAFKA-12931: KIP-746: Revise KRaft Metadata Records (#10867)
PR-10804 - KAFKA-12877: Make flexibleVersions mandatory (#10804)
PR-10786 - KAFKA-12787; Integrate controller snapshoting with raft client (#10786)
PR-10749 - KAFKA-12773; Use UncheckedIOException when wrapping IOException (#10749)
PR-10584 - KAFKA-12701: NPE in MetadataRequest when using topic IDs (#10584)
PR-10882 - KAFKA-12948: Remove node from ClusterConnectionStates.connectingNodes when node is removed (#10882)
PR-10846 - KAFKA-12914: StreamSourceNode should return null topic name for pattern subscription (#10846)
PR-10871 - KAFKA-8940: decrease session timeout to make test faster and reliable (#10871)
PR-10847 - KAFKA-12921: Upgrade zstd-jni to 1.5.0-2 (#10847)
PR-10820 - KAFKA-12892: Disable testChrootExistsAndRootIsLocked (#10820)
PR-10865 - KAFKA-12934: Move some controller classes to the metadata package (#10865)
PR-10736 - KAFKA-9295: revert session timeout to default value (#10736)
PR-10850 - KAFKA-12924 Replace EasyMock and PowerMock with Mockito in streams metrics tests (#10850)
PR-9414 - KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup (#9414)
PR-10849 - KAFKA-12922: MirrorCheckpointTask should close topic filter (#10849)
PR-10835 - KAFKA-12905: Replace EasyMock and PowerMock with Mockito for NamedCacheMetricsTest (#10835)
PR-10665 - KAFKA-9009: increase replica.lag.time.max.ms to make the test reliable (#10665)
PR-10803 - KAFKA-12874; Increase default consumer session timeout to 45s (#10803)
PR-10609 - KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure (#10609)
PR-10823 - KAFKA-12897: KRaft multi-partition placement on single broker (#10823)
PR-10793 - KAFKA-12338; Remove unused MetadataParser (#10793)
PR-9441 - KAFKA-10614: Ensure group state (un)load is executed in the right order (#9441)
PR-10821 - KAFKA-12892: Use dedicated root in ZK ACL test (#10821)
PR-580 - CONFLUENT: Complete version regex check in system tests
PR-10646 - KAFKA-8897 Follow-up: Consolidate the global state stores (#10646)
PR-10471 - KAFKA-12597: Remove deprecated –zookeeper option in ReassignPartitionsCommand (#10471)
PR-10752 - KAFKA-12620 Allocate Producer IDs in KRaft controller (#10752)
PR-10664 - KAFKA-12749: Changelog topic config on suppressed KTable lost (#10664)
PR-10787 - KAFKA-12864: Move KafkaEventQueue into server-common. #10787 (#10787)
PR-10552 - KAFKA-12675: improve the sticky general assignor scalability and performance (#10552)
PR-10808 - KAFKA-12880: Remove deprecated Count and SampledTotal in 3.0 (#10808)
PR-10797 - KAFKA-12867: Fix ConsumeBenchWorker exit behavior for maxMessages config (#10797)
PR-10616 - KAFKA-12709; Add Admin API for ListTransactions (#10616)
PR-10457 - KAFKA-12596: remove –zookeeper option from topic command (#10457)
PR-10795 - KAFKA-12866: Avoid root access to Zookeeper (#10795)
PR-10765 - KAFKA-12519: Remove built-in Streams metrics for versions 0.10.0-2.4 (#10765)
PR-10790 - KAFKA-12865 : Fix doc error in Admin.describeAcls API (#10790)
PR-10744 - KAFKA-8410: KTableProcessor migration groundwork (#10744)
PR-10704 - KAFKA-12791: ConcurrentModificationException in AbstractConfig use by KafkaProducer (#10704)
PR-10778 - KAFKA-12856: Upgrade Jackson to 2.12.3 (#10778)
PR-10710 - KAFKA-12796: Removal of deprecated classes under streams-scala (#10710)
PR-10762 - KAFKA-12819: Add assert messages to MirrorMaker tests plus other quality of life improvements (#10762)
PR-10017 - KAFKA-12260: Avoid hitting NPE for partitionsFor (#10017)
PR-10717 - KAFKA-12800: Configure generator to fail on trailing JSON tokens (#10717)
PR-577 - KC-1749: Upgrade Jetty, commons-codec, etc
PR-576 - KC-1748: Upgrade Jetty and commons-codec
PR-10482 - KAFKA-12499: add transaction timeout verification (#10482)
PR-10504 - KAFKA-12620 Allocate producer ids on the controller (#10504)
PR-10724 - KAFKA-12808: Remove Deprecated Methods under StreamsMetrics (#10724)
PR-10375 - KAFKA-12522: Cast SMT should allow null value records to pass through (#10375)
57c14224 - KAFKA-12820: Upgrade maven-artifact dependency
PR-10572 - KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller (#10572)
PR-10705 - KAFKA-12342: Remove MetaLogShim and use RaftClient directly (#10705)
PR-10735 - KAFKA-12779: KIP-740, Clean up public API in TaskId and fix TaskMetadata#taskId() (#10735)
PR-10737 - KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs (#10737)
PR-10729 - KAFKA-12809: Remove deprecated methods of Stores factory (#10729)
PR-10730 - KAFKA-12813: Remove deprecated schedule method in ProcessorContext (#10730)
PR-10720 - KAFKA-12815: Preserve context for KTable.transformValues when getting value from upstream state store (#10720)
PR-10727 - KAFKA-12810: Remove deprecated TopologyDescription.Source#topics (#10727)
PR-564 - APPSEC-60: Upgrade org.apache.maven:apache-maven
PR-10700 - KAFKA-12789: Remove Stale comments for meta response handling logic (#10700)
PR-10719 - Fix compile errors from KAFKA-12543 (#10719)
PR-10431 - KAFKA-12543: Change RawSnapshotReader ownership model (#10431)
PR-10561 - KAFKA-12686 AlterIsr and LeaderAndIsr race condition (#10561)
PR-10494 - KAFKA-12788: improve KRaft replica placement (#10494)
PR-10633 - KAFKA-12751: Reset AlterIsr in-flight state for duplicate update requests (#10633)
PR-10675 - KAFKA-12574: remove internal Producer config and auto downgrade logic (#10675)
PR-10707 - KAFKA-12792: Fix metrics bug and introduce TimelineInteger (#10707)
PR-10542 - KAFKA-12313: KIP-725: Streamlining configs for Windowed Deserialisers (#10542)
PR-10709 - KAFKA-12794: Fix trailing json tokens in DescribeProducersRequest.json (#10709)
PR-10620 - KAFKA-12736: KafkaProducer.flush holds onto completed ProducerBatch(s) until flush completes (#10620)
PR-10606 - KAFKA-12728: Upgrade gradle to 7.0.2 and shadow to 7.0.0 (#10606)
PR-10688 - KAFKA-12778: Fix QuorumController request timeouts and electLeaders (#10688)
PR-10634 - KAFKA-12754: Improve endOffsets for TaskMetadata (#10634)
PR-10680 - Rework on KAFKA-3968: fsync the parent directory of a segment file when the file is created (#10680)
PR-6592 - KAFKA-8326: Introduce List Serde (#6592)
PR-10679 - KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller (#10679)
PR-10676 - KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata (#10676)
PR-10667 - KAFKA-12772: Move all transaction state transition rules into their states (#10667)
PR-10638 - KAFKA-12758 Added server-common module to have server side common classes. (#10638)
PR-10657 - KAFKA-5876: KIP-216 Part 4, Apply InvalidStateStorePartitionException for Interactive Queries (#10657)
PR-10643 - KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix (#10643)
PR-10645 - KAFKA-12464: minor code cleanup and additional logging in constrained sticky assignment (#10645)
PR-10390 - KAFKA-12536: Add Instant-based methods to ReadOnlySessionStore (#10390)
PR-10640 - KAFKA-10847: Set StreamsConfig on InternalTopologyDriver before writing topology (#10640)
PR-3547 - KC-1618: Adapt the KRaft README and example configs for CP (#3547)
PR-10568 - KAFKA-8897: Upgrade RocksDB to 6.19.3 (#10568)
PR-10296 - KAFKA-12451: Remove deprecation annotation on long-based read operations in WindowStore (#10296)
PR-10636 - KAFKA-12752: Bump Jersey deps to 2.34 (#10636)
PR-10509 - KAFKA-12464: enhance constrained sticky Assign algorithm (#10509)
PR-10532 - KAFKA-8531: Change default replication factor config (#10532)
PR-10014 - KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes (#10014)
PR-10613 - KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied (#10613)
PR-10271 - KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage. (#10271)
PR-9779 - KAFKA-10767: Adding test cases for all, reverseAll and reverseRange for ThreadCache (#9779)
PR-10294 - KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore (#10294)
PR-10612 - KAFKA-10847: Add internal flag to disable KAFKA-10847 fix (#10612)
PR-10597 - KAFKA-5876: KIP-216 Part 3, Apply StreamsNotStartedException for Interactive Queries (#10597)
PR-10446 - KAFKA-12661 ConfigEntry#equal does not compare other fields when value is NOT null (#10446)
PR-10085 - KAFKA-12154; Raft Snapshot Loading API (#10085)
PR-10557 - KAFKA-12683: Remove deprecated UsePreviousTimeOnInvalidTimestamp (#10557)
PR-10615 - KAFKA-12648: basic skeleton API for NamedTopology (#10615)
PR-10548 - KAFKA-12396: added null check for state stores key (#10548)
PR-555 - fix: Upgrade Jersey to 2.34
PR-10480 - KAFKA-12265; Move the BatchAccumulator in KafkaRaftClient to LeaderState (#10480)
PR-10611 - KAFKA-12730; Avoid duplicate logout if Kerberos login fails (#10611)
PR-10462 - KAFKA-10847: Fix spurious results on left/outer stream-stream joins (#10462)
PR-10573 - KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2 (#10573)
PR-554 - fix: Upgrade jersey to 2.34
PR-10466 - KAFKA-12417: streams copyDependentLibs should not copy testRuntime configuration jars (#10466)
PR-10547 - KAFKA-12284: increase request timeout to make tests reliable (#10547)
PR-10599 - KAFKA-12716; Add Admin API to abort transactions (#10599)
PR-8923 - KAFKA-6435: KIP-623 Add internal topics option to streamResetter (#8923)
PR-10519 - KAFKA-12344 Support SlidingWindows in the Scala API (#10519)
PR-10558 - KAFKA-12684: Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand (#10558)
PR-9627 - KAFKA-10746: Change to Warn logs when necessary to notify users (#9627)
PR-9821 - KAFKA-5876: Apply UnknownStateStoreException for Interactive Queries (#9821)
PR-9640 - KAFKA-10283; Consolidate client-level and consumer-level assignment within ClientState (#9640)
PR-10483 - KAFKA-12586; Add DescribeTransactions Admin API (#10483)
PR-10580 - KAFKA-12704: Improve cache access during connector class instantiation in config validations (#10580)
PR-10574 - KAFKA-12700: override toString method to show correct value in doc (#10574)
PR-10575 - KAFKA-12702: Fix NPE in networkListeners from BrokerServer (#10575)
PR-10565 - KAFKA-12691: Add case where task can be considered idling (#10565)
PR-10478 - KAFKA-12553: Refactor recovery logic to introduce LogLoader (#10478)
PR-10537 - KAFKA-10847: Delete Time-ordered duplicated records using deleteRange() internally (#10537)
PR-10508 - KAFKA-12633: Remove deprecated APIs in TopologyTestDriver (#10508)
PR-9549 - KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader (#9549)
PR-10545 - KAFKA-12672: Added config for raft testing server (#10545)
PR-10470 - KAFKA-12612: Remove checksum from ConsumerRecord/RecordMetadata for 3.0 (#10470)
PR-10536 - KAFKA-12667: Fix incorrect error log on StateDirectory close (#10536)
PR-10529 - KAFKA-12650: fix NPE in InternalTopicManagerTest (#10529)
PR-10409 - KAFKA-9295: improve KTableKTableForeignKeyInnerJoinMultiIntegrationTest (#10409)
PR-10343 - KAFKA-12471: Implement createPartitions in KIP-500 mode (#10343)
PR-10218 - KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager. (#10218)
PR-10511 - KAFKA-12584; Remove deprecated Sum and Total classes (#10511)
PR-10526 - KAFKA-12655 Update Jetty: 9.4.38.v20210224 → 9.4.39.v20210325 (#10526)
PR-10469 - KAFKA-12611: Fix using random payload in ProducerPerformance incorrectly (#10469)
PR-10531 - KAFKA-12658: Include kafka-shell jar and dependencies in release tar (#10531)
PR-10512 - KAFKA-12637: Remove deprecated PartitionAssignor interface (#10512)
PR-10411 - KAFKA-7606: Remove deprecated options from StreamsResetter (#10411)
PR-10042 - KAFKA-9527: fix NPE when using time-based argument for Stream Resetter Tool (#10042)
PR-10503 - KAFKA-9988: Suppress uncaught exceptions in log messages during Connect task shutdown (#10503)
PR-10258 - KAFKA-12408: Document omitted ReplicaManager metrics (#10258)
PR-10510 - KAFKA-12607; Test case for resigned state vote granting (#10510)
PR-10381 - KAFKA-8410: Migrate KStream Stateless operators to new Processor API (#10381)
PR-10301 - KAFKA-9831: increase max.poll.interval.ms to avoid unexpected rebalance (#10301)
PR-10293 - KAFKA-12449: Remove deprecated WindowStore#put (#10293)
PR-10427 - KAFKA-12591; Remove deprecated quota.producer.default and quota.consumer.default configurations (#10427)
PR-10486 - KAFKA-12492: Fix the formatting of example RocksDBConfigSetter (#10486)
PR-10481 - KAFKA-12619; Raft leader should expose hw only after committing LeaderChange (#10481)
PR-10254 - KAFKA-12406 Integrate client quotas with KRaft broker (#10254)
PR-10492 - KAFKA-12457; Add sentinel ID to metadata topic (#10492)
PR-10502 - KAFKA-12630: Remove deprecated KafkaClientSupplier#getAdminClient in Streams (#10502)
PR-10421 - KAFKA-12568: Remove deprecated APIs in KStream, KTable and Joined (#10421)
PR-10380 - KAFKA-12527: Remove deprecated PartitionGrouper annotation (#10380)
PR-10302 - KAFKA-7785: move internal DefaultPartitionGrouper (#10302)
PR-10331 - KAFKA-10847: Add new RocksDBTimeOrderedWindowStore that persists (time-key)-value records (#10331)
PR-9851 - KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup… (#9851)
PR-10389 - KAFKA-12384: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch (#10389)
PR-10131 - KAFKA-5146: remove Connect dependency from Streams module (#10131)
PR-10474 - KAFKA-12602: Fix LICENSE file (#10474)
PR-10366 - KAFKA-12467: Implement QuorumController snapshot generation (#10366)
PR-10438 - KAFKA-12579: Remove various deprecated clients classes/methods for 3.0 (#10438)
PR-10142 - KAFKA-12294; forward auto topic request within envelope on behalf of clients (#10142)
PR-10445 - KAFKA-12548; Propagate record error messages to application (#10445)
PR-10393 - KAFKA-12539; Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity (#10393)
PR-9992 - KAFKA-10759 Add ARM build stage (#9992)
PR-10477 - KAFKA-12615: Fix Selector.clear() javadoc typo (#10477)
PR-10450 - KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0 (#10450)
PR-10405 - KAFKA-3968: fsync the parent directory of a segment file when the file is created (#10405)
PR-10325 - MONOR: Remove redudant LocalLogManager (#10325)
PR-10447 - KAFKA-12587 Remove KafkaPrincipal#fromString for 3.0 (#10447)
PR-10396 - KAFKA-12474: Handle failure to write new session keys gracefully (#10396)
PR-10282 - KAFKA-12426: Missing logic to create partition.metadata files in RaftReplicaManager (#10282)
PR-10452 - KAFKA-12593: Fix Apache License headers (#10452)
PR-10458 - KAFKA-12600: Remove deprecated config value default for client config client.dns.lookup (#10458)
PR-10451 - KAFKA-12952: Remove deprecated LogConfig.Compact (#10451)
PR-10454 - Initial commit (#10454)
PR-10430 - KAFKA-12575: Eliminate Log.isLogDirOffline boolean attribute (#10430)
PR-10443 - KAFKA-8405; Remove deprecated kafka-preferred-replica-election command (#10443)
6703cc20 - KAFKA-12583: Upgrade netty to 4.1.62.Final
PR-10436 - KAFKA-12577; Remove deprecated ConfigEntry constructor for 3.0 (#10436)
PR-10440 - KAFKA-12581: Remove deprecated Admin.electPreferredLeaders (#10440)
PR-10435 - KAFKA-12578: Remove deprecated security classes/methods for 3.0 (#10435)
PR-10426 - KAFKA-12571: Eliminate LeaderEpochFileCache constructor dependency on logEndOffset (#10426)
PR-10342 - KAFKA-12288: remove task-level filesystem locks (#10342)
PR-10418 - KAFKA-12509 Tighten up StateDirectory thread locking (#10418)
PR-10401 - KAFKA-12552: Introduce LogSegments class abstracting the segments map (#10401)
PR-10404 - KAFKA-12557: Fix hanging KafkaAdminClientTest (#10404)
PR-10425 - KAFKA-12573; Remove deprecated Metric#value (#10425)
PR-10416 - KAFKA-8784: remove default close for RocksDBConfigSetter (#10416)
PR-10407 - KAFKA-12523: handle TaskCorruption and TimeoutException during handleCorruption and handleRevocation (#10407)
PR-10378 - KAFKA-7106: remove deprecated Windows APIs (#10378)
PR-10412 - KAFKA-12562: Remove deprecated APIs in KafkaStreams and returned state classes (#10412)
PR-10387 - KAFKA-12537: fix application shutdown corner case with only one thread (#10387)
PR-10300 - KAFKA-12452: Remove deprecated overloads of ProcessorContext#forward (#10300)
PR-10397 - KAFKA-12508: Disable KIP-557 (#10397)
PR-10275 - KAFKA-12434; Admin support for DescribeProducers API (#10275)
PR-10392 - KAFKA-12435: Fix javadoc errors (#10392)
PR-10281 - KAFKA-12432: AdminClient should time out nodes that are never ready (#10281)
PR-9858 - KAFKA-12173 Migrate streams:streams-scala module to JUnit 5 (#9858)
PR-10379 - KAFKA-12524: Remove deprecated segments() (#10379)
PR-10374 - (Cherry-pick) KAFKA-9274: handle TimeoutException on task reset (#10000) (#10374)
PR-10371 - KAFKA-12479: Batch partition offset requests in ConsumerGroupCommand (#10371)
PR-10309 - KAFKA-12181; Loosen raft fetch offset validation of remote replicas (#10309)
PR-10372 - (Cherry-pick) KAFKA-9274: handle TimeoutException on task reset (#10000) (#10372)
PR-10360 - KAFKA-12508: Emit records with same value and same timestamp (#10360)
PR-10220 - KAFKA-12383: Get RaftClusterTest.java and other KIP-500 junit tests working (#10220)
PR-10369 - KAFKA-12514: Fix NPE in SubscriptionState (#10369)
PR-10150 - KAFKA-3745: Add access to read-only key in value joiner (#10150)
PR-10355 - KAFKA-12500: fix memory leak in thread cache (#10355)
PR-10286 - KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper (#10286)
PR-10361 - KAFKA-9189: Use MetadataCache instead of ZooKeeper during controlled shutdown to avoid hang (#10361)
PR-10289 - KAFKA-12440; ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot (#10289)
PR-10276 - KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog’s validateOffsetAndEpoch (#10276)
PR-10299 - KAFKA-10070: parameterize Connect unit tests to remove code duplication (#10299)
PR-10356 - KAFKA-12503: inform threads to resize their cache instead of doing so for them (#10356)
PR-10332 - KAFKA-10697: Remove ProduceResponse.responses (#10332)
PR-10227 - KAFKA-12382: add a README for KIP-500 (#10227)
PR-10317 - KAFKA-10357: Add setup method to internal topics (#10317)
9ffa6d6d - KAFKA-12293: Remove JCenter from buildscript and delete buildscript.gradle
PR-10341 - KAFKA-12491: Make rocksdb an api dependency for streams (#10341)
PR-10304 - KAFKA-12454: Add ERROR logging on kafka-log-dirs when given brokerIds do not exist in current kafka cluster (#10304)
PR-10323 - KAFKA-12459; Use property testing library for raft event simulation tests (#10323)
PR-10322 - KAFKA-12455: Fix OffsetValidationTest.test_broker_rolling_bounce failure with Raft (#10322)
PR-10318 - KAFKA-12330; FetchSessionCache may cause starvation for partitions when FetchResponse is full (#10318)
PR-10267 - KAFKA-12427: Don’t update connection idle time for muted connections (#10267)
PR-10320 - revert stream logging level back to ERROR (#10320)
PR-10232 - KAFKA-12352: Make sure all rejoin group and reset state has a reason (#10232)
PR-10135 - KAFKA-10348: Share client channel between forwarding and auto creation manager (#10135)
PR-10285 - KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1 (#10285)
PR-10311 - KAFKA-12462: proceed with task revocation in case of thread in PENDING_SHUTDOWN (#10311)
PR-10310 - KAFKA-12460; Do not allow raft truncation below high watermark (#10310)
PR-10305 - KAFKA-10357: Add missing repartition topic validation (#10305)
PR-10021 - KAFKA-12205; Delete snapshots less than the snapshot at the log start (#10021)
PR-10266 - KAFKA-10357: Add validation method for internal topics (#10266)
PR-10092 - KAFKA-12287: Add WARN logging on consumer-groups when reset-offsets by timestamp or duration can’t find an offset and defaults to latest (#10092)
PR-10284 - KAFKA-12441: remove deprecated method StreamsBuilder#addGlobalStore (#10284)
PR-9744 - KAFKA-10062: Add a methods to retrieve the current timestamps as known by the Streams app (#9744)
PR-10118 - KAFKA-10192: Increase max time to wait for worker to start in some integration tests (#10118)
PR-528 - KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes
PR-10252 - KAFKA-12403; Ensure local state deleted on RemoveTopicRecord received (#10252)
PR-10240 - KAFKA-12381: remove live broker checks for forwarding topic creation (#10240)
PR-10211 - KAFKA-12347: updating TaskMetadata (#10211)
PR-525 - Enable downstream validation
PR-10260 - Remove ReassigningPartitions metric when shutting down ReplicaManager (#10260)
PR-10253 - KAFKA-12376: Apply atomic append to the log (#10253)
PR-10203 - KAFKA-12415 Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies (#10203)
PR-10263 - KAFKA-12393: Document multi-tenancy considerations (#334) (#10263)
PR-10257 - KAFKA-12407: Document Controller Health Metrics (#10257)
PR-9950 - KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string (#9950)
PR-10185 - KAFKA-12284: Wait for mirrored topics to be created (#10185)
PR-10173 - KAFKA-9548 Added SPIs and public classes/interfaces introduced in KIP-405 for tiered storage feature in Kafka. (#10173)
b77deece - KAFKA-12400: Upgrade jetty
PR-10215 - KAFKA-12375: don’t reuse thread.id until a thread has fully shut down (#10215)
PR-10217 - KAFKA-12254: Ensure MM2 creates topics with source topic configs (#10217)
PR-10216 - KAFKA-12177: apply log start offset retention before time and size based retention (#10216)
PR-10206 - KAFKA-12369; Implement ListTransactions API (#10206)
PR-10163 - KAFKA-10357: Extract setup of changelog from Streams partition assignor (#10163)
PR-10228 - KAFKA-10251: increase timeout for consuming records (#10228)
PR-10223 - KAFKA-12394; Return TOPIC_AUTHORIZATION_FAILED in delete topic response if no describe permission (#10223)
PR-9717 - KAFKA-10766: Unit test cases for RocksDBRangeIterator (#9717)
PR-10052 - KAFKA-12289: Adding test cases for prefix scan in InMemoryKeyValueStore (#10052)
PR-10137 - KAFKA-12268: Implement task idling semantics via currentLag API (#10137)
PR-10205 - KAFKA-12323 Follow-up: Refactor the unit test a bit (#10205)
PR-10016 - KAFKA-10340: Proactively close producer when cancelling source tasks (#10016)
PR-10141 - KAFKA-12329; kafka-reassign-partitions command should give a better error message when a topic does not exist (#10141)
PR-10199 - KAFKA-12374: Add missing config sasl.mechanism.controller.protocol (#10199)
PR-8812 - KAFKA-10101: Fix edge cases in Log.recoverLog and LogManager.loadLogs (#8812)
PR-10213 - KAFKA-12375: fix concurrency issue in application shutdown (#10213)
PR-9990 - KAFKA-12235: Fix a bug where describeConfigs does not work on 2+ config keys (#9990)
PR-9235 - KAFKA-10449: Add some important parameter desc in connect-distributed.properties (#9235)
PR-10194 - KAFKA-12365; Disable APIs not supported by KIP-500 broker/controller (#10194)
PR-10059 - KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while building SslTransportLayer (#10059)
PR-10200 - KAFKA-12367; Ensure partition epoch is propagated to Partition state (#10200)
PR-10183 - KAFKA-12267; Implement DescribeTransactions API (#10183)
PR-10165 - KAFKA-12350: Correct the default value in doc (#10165)
PR-10170 - KAFKA-12323: Set timestamp in record context when punctuate (#10170)
PR-10178 - KAFKA-12361; Use default request.timeout.ms value for Connect producers (#10178)
PR-10191 - KAFKA-12341: Ensure consistent versions for javassist (#10191)
PR-10174 - KAFKA-12357: Do not inline methods from the scala package by default (#10174)
PR-10024 - KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024)
9e452950 - KAFKA-12324: Upgrade jetty
PR-10152 - KAFKA-12339: Add retry to admin client’s listOffsets (#10152)
PR-10153 - KAFKA-12340: Fix potential resource leak in Kafka*BackingStore (#10153)
PR-10145 - KAFKA-12335 Upgrade junit from 5.7.0 to 5.7.1 (#10145)
PR-10070 - KAFKA-12276: Add the quorum controller code (#10070)
PR-9631 - KAFKA-9672: Leader with ISR as a superset of replicas (#9631)
690f72dd - KAFKA-12334: Add the KIP-500 metadata shell
PR-10129 - KAFKA-10817; Add clusterId validation to raft Fetch handling (#10129)
PR-10158 - KAFKA-12343: Handle exceptions better in TopicAdmin, including UnsupportedVersionException (#10158)
PR-10072 - KAFKA-9274: Throw TaskCorruptedException instead of TimeoutException when TX commit times out (#10072)
PR-10143 - KAFKA-12332; Error partitions from topics with invalid IDs in LISR requests (#10143)
PR-10063 - KAFKA-12258; Add support for splitting appending records (#10063)
PR-10091 - KAFKA-9524: increase retention time for window and grace periods longer than one day (#10091)
PR-10138 - KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch (#10138)
PR-10666 - KAFKA-12278; Ensure exposed api versions are consistent within listener (#10666)
PR-10133 - Fixed README and added clearer error message. (#10133)
PR-10073 - KAFKA-12303: Fix handling of null values by Flatten SMT (#10073)
PR-9906 - KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of… (#9906)
PR-10102 - KAFKA-12272: Fix commit-interval metrics (#10102)
PR-10123 - KAFKA-12327: Remove MethodHandle usage in CompressionType (#10123)
PR-10122 - KAFKA-12326: Corrected regresion in MirrorMaker 2 executable introduced with KAFKA-10021 (#10122)
PR-9780 - KAFKA-10021: Changed Kafka backing stores to use shared admin client to get end offsets and create topics (#9780)
b3313b86 - KAFKA-12297: Make MockProducer return RecordMetadata with values as per contract
PR-10098 - KAFKA-12321 the comparison function for uuid type should be ‘equals’ rather than ‘==’ (#10098)
PR-9430 - KAFKA-5235: GetOffsetShell: Support for multiple topics and consumer configuration override (KIP-635) (#9430)
PR-9986 - JUnit extensions for integration tests (#9986)
PR-9865 - Fill in the 2.8 release note for Authorizer (#9865)

ksqlDB¶

PR-8051 - chore: commons-codec override
PR-7936 - refactor: support BYTES on QTT for Protobuf
PR-7932 - fix: dont use default package for version.properties
PR-7921 - fix: increase default command response timeout
PR-7909 - feat: enable BYTES for LPAD and RPAD
PR-7923 - fix: checkstyle remove separation in imports
PR-7917 - Bytes split function
PR-7904 - feat: allow expressions on left table columns in FK-joins
PR-7880 - Improves RQTT by using offsets to measure readiness
PR-7915 - fix: no named topologies yet
PR-7889 - feat: added missing tostring method
PR-7818 - build: update multiple modules for spotbug 4.3.0 and re-enable plugin
PR-7773 - feat: make SchemaRegistry permission validations on KSQL requests
PR-7582 - fix: silently ignores KEY qualifier if enclosed in parentheses (#5852)
PR-7871 - feat: allow Java clients to set HTTP/2 multiplex limit
PR-7876 - feat: enables BYTES for CONCAT and CONCAT_WS
PR-7813 - build: update execution module for spotbug 4.3.0 and re-enable plugin
PR-7872 - build: suppress spotbug warning to fix build
PR-7865 - feat: update len function to accept BYTES
PR-7817 - build: update rest-model module for spotbug 4.3.0 and re-enable plugin
PR-7816 - build: update rest-app module for spotbug 4.3.0 and re-enable plugin
PR-7815 - build: update parser module for spotbug 4.3.0 and re-enable plugin
PR-7812 - build: update engine module for spotbug 4.3.0 and re-enable plugin
PR-7810 - build: update common module for spotbug 4.3.0 and re-enable plugin
PR-7809 - build: update api-client module for spotbug 4.3.0 and re-enable plugin
PR-7804 - feat: implement bytes comparisons
PR-7861 - feat: Update SUBSTRING function to accept BYTES types
PR-7821 - feat: don’t start queries when corruption is detected during startup
PR-7823 - feat: support BYTES column types in Java client API
PR-7831 - feat: add TO_BYTES/FROM_BYTES functions for bytes/string conversions
PR-7729 - fix: Clean up a terminated query’s state stores
PR-7822 - build: fix deprecation warnings from KIP-744
PR-7791 - feat: implement serdes for bytes
PR-7788 - feat: add interface for metrics reporter
PR-7769 - feat: add observability metric skeleton
PR-7778 - feat: add BYTES type to ksqlDB
PR-7785 - fix: create more user friendly transient query ids
PR-7783 - refactor: wrap SchemaRegistry authorization errors into a KsqlSchemaAuthorizationException
PR-7761 - feat: add ARRAY_CONCAT UDF
PR-7750 - fix: return an error message on http2 /query-stream endpoint
PR-7742 - feat: add FROM_DAYS and update UNIX_DATE function
PR-7744 - feat: add DATEADD and DATESUB functions
PR-7739 - fix: enable KAFKA-10847 bugfix when GRACE PERIOD is used on Joins
PR-7733 - feat: add PARSE_DATE and FORMAT_DATE functions
PR-7740 - feat: enable date/time for the query interpreter
PR-7738 - feat: pull query bandwidth based throttling
PR-7727 - feat: add TIMEADD and TIMESUB functions
PR-7722 - feat: add PARSE_TIME and FORMAT_TIME functions
PR-7737 - fix: enable schema inference for timestamp/time/date
PR-7734 - feat: implement comparisons for TIME/DATE
PR-7719 - fix: fixed nondeterminism in UdfIndex
PR-7724 - fix: remove time/date component when casting timestamp to date/time
PR-7718 - feat: implement serde for DATE
PR-7709 - fix: enable time unit functions for interpreter
PR-7693 - fix: block out of order migrations in migrations tool
PR-7708 - feat: implement casting for DATE/TIME
PR-7664 - feat: serde for TIME type
PR-7678 - feat: call KStreams API grace() when GRACE PERIOD is used in joins
PR-7702 - fix: dont incorrectly log events for each pull query
PR-7700 - feat: define how DATE data is formatted
PR-7661 - Scalable push queries end to end
PR-7674 - feat: define how TIME data is formatted
PR-7667 - fix: make current java clients compatible with pre-0.15 servers
PR-7662 - feat: Support GRACE PERIOD syntax on WITHIN clause with simple window
PR-7672 - docs: Additional quickstarts
PR-7643 - docs: Removing a duplicated clause.
PR-7670 - fix: longer IT timeouts
PR-7627 - fix: Fixes race condition exposing uninitialized query
PR-7641 - feat: add the DATE and TIME sql types
PR-7639 - fix: Sync Vertx version
PR-7587 - feat: Adds Scalable Push Query Routing
PR-7625 - docs: document parallel build
PR-7620 - fix: Update Maven wrapper and document its use to fix version resolution
PR-7615 - feat: use Connect default precision for Avro decimals if unspecified (MINOR)
PR-7601 - fix: DROP stream for persistent query doesn’t always drop underlying query
PR-7577 - fix: allow for KSQL_GC_LOG_OPTS env variable to be passed through wit
PR-7591 - feat: ungate support for foreign key joins
PR-7585 - fix: qualified select * now works in n-way joins with repartitions and multiple layers of nesting
PR-7556 - feat: add idle timeout
PR-7554 - fix: better error message on illegal arithmetic with NULL values
PR-7544 - fix: reject mismatched decimals from avro topics
PR-7523 - chore: Run query anon logger
PR-7538 - docs: specify HTTP1.1 for curl
PR-7533 - fix: npe on transient query close (#7530)
PR-7517 - feat: build physical plan for FK table-table joins
PR-7430 - feat: Adds Scalable Push Query physical operators
PR-7483 - fix: change the isError to not use the state
PR-7424 - feat: Adds ScalablePushRegistry and peeking ability in a persistent query
PR-7452 - feat: Allow ksqlDB to detect FK-join table-table join condition
PR-6685 - feat: added NULLIF function (#6567)
PR-7453 - chore: query anonymizer interface
PR-7458 - fix: Fixes org.mock-server to be 5.11.2 to utilize newer netty
PR-7434 - feat: made class to compute essential meta-data
PR-7427 - fix: worker-poll tasks shouldn’t block main loop
PR-7420 - fix: long-running queries shouldn’t block the main event loop
PR-7418 - fix: revert schema registry version
PR-7357 - feat: implemented working query anonymizer
PR-7401 - feat(migrations): add –define flag to apply command
PR-7373 - fix: preserve the rest of a struct when one field has a processing error
PR-7366 - feat(migrations): support DEFINE and UNDEFINE statements
PR-7390 - feat: emit an error reason before closing websocket
PR-7246 - feat: Materialize Table-Table join results
PR-7365 - fix: add sessionVariables to /ksql endpoint validator
PR-7337 - fix: source key format compatibility with non primitive types
PR-7320 - refactor: reorganize query management and clean up listeners
PR-7335 - feat: enable variable substitution for java client
PR-7331 - feat: scatter gather task metadata for query descriptions
PR-7336 - fix: append additional fields to row values in KsqlMaterialization.java to fix pull query filtering
PR-7324 - fix: Add line breaks to error message
PR-7321 - fix: Make topic offset public
PR-7272 - feat: Detailed pull query metrics broken down by type and source
PR-7284 - feat: add ‘show connector plugins’ syntax
PR-7271 - feat: enable variable substitution for /query-stream and /ksql endpoints
PR-7309 - refactor: change how we store stats on SourceDescription
PR-7238 - fix: Bubble up errors from HARouting unless using StandbyFallbackException
PR-7304 - fix: only list known connect worker properties when listing server props
PR-7296 - feat(cli): print stats/errors breakdown by host
PR-7289 - Set should fail for config that cannot be changed in session
PR-7295 - docs: Normalize usage of ‘record’, ‘message’ and ‘event’ in Kafka doc
PR-7252 - feat(executor): Allow SourceDescription returning executors to return cluster wide stream stats
PR-7268 - fix: fix NPE when backing a record that has null key/values
PR-7222 - feat(java client): Add connector functions to java client
PR-7243 - fix(java client): allow java client to accept statements with more than one semicolon
PR-7151 - feat: expose TaskMetadata in QueryDescription
PR-7244 - refactor(executor): factor out a class allowing you to merge local results with results for running the same statement on other nodes
PR-7003 - fix: Append state.dir directive to ksql-server.properties
PR-7235 - feat: update sourcedescription and errorstats to have statistics as structured objects
PR-7212 - fix: Ensures BaseSubscriber.makeRequest is called on context in PollableSubscriber

REST Proxy¶

PR-904 - KREST-2408: Try longer consumer poll in getConsumerLag integration test
PR-901 - KREST-2303: Try harder to find ConsumerLag in integration test.
PR-897 - KREST-2260: Use super user when creating topic in AuthorizationErrorTest.
PR-894 - Fix ambiguous describeConfigs reference in tests
PR-893 - Remove ZooKeeperClient/KafkaZkClient usage from tests
PR-885 - KREST-1577 Replace a manually set Avro version with the Avro version variable.
PR-870 - Use a more lenient test for testBackoffMsControlsPollCalls.
PR-862 - KREST-1458: Always allow OPTIONS requests.
PR-859 - KREST-1334 Reduce time testing during ce-kafka-images packaging
PR-858 - KREST-728: Add OpenAPI spec for Produce V3.
PR-832 - KREST-46: Removed property ${confluent.maven.repo}
PR-847 - Fixing openapi spec and ConsumerAssignment lag relationship
PR-844 - MMA-10490: include count of partitions in topic(s) response
b0e7b11c - MMA-10463: list all broker configs
72b4889c - fix: resolve ProduceController early/outside of lambda
PR-835 - KREST-865 Remove string usages of the class name of the deprecated SimpleAclAuthorizer
PR-834 - KREST-865 Remove usage of a deprecated Admin Client ConfigEntry constructor
PR-824 - KREST-771 Fix build breakage by also updating ProduceAction.java
PR-823 - KREST-771 Implement an allowlist for API endpoints.
PR-817 - Fix mtls test env run script.
PR-810 - MMA-9932 list all topic configs
002df165 - Add test environment using mTLS.
c1a6387d - increasing poll time…
8076cb46 - retrying only the http response
PR-808 - KREST-117: Add missing example for create acl
dd71233f - moving expectedOffsets assignment out of retry
52fc9300 - adding additional retry to getConsumerLag_returnsConsumerLag
1e7652a0 - adding retry to ConsumerLagsResourceIntegrationTest
PR-287 - SEC-2720: Add SecureRequestCustomizer for correct resolution of http vs https requests.
6fa4e875 - MMA-11014: Wrap correct exception
PR-286 - MMA-11014: wrap SSLException
PR-280 - KREST-578: Add configurable DosFilter support. (#376
PR-270 - Upgrade Jetty to 9.4.43.v20210629
PR-263 - KREST-1403: Set RFC7230 mode for connections
PR-262 - KREST-1346: Add option to apply rate-limit globally across all connections.
3caee6f9 - Remove flakiness source from some tests.
PR-258 - KREST-661: Add support for HTTP/2 on Java 11 and later
PR-234 - Update Jetty: 9.4.36.v20210114 → 9.4.38.v20210224
PR-238 - KREST-578: Add configurable DosFilter support.
PR-240 - KREST-860 Fix deprecated usages of org.apache.kafka.common.Metric.value() in master
PR-230 - MMA-10032: Make configure handler methods public

Schema Registry¶

Note: ZooKeeper is no longer supported for leader election when using Schema Registry.

PR-2052 - DGS-2315: Only tombstone previous schemas during hard deletion
PR-2043 - DGS-2259 Omit defaults for fields with type Message
PR-2026 - DGS-2078 Fix optional Avro schema generation
PR-2024 - DGS-2076 Fix NPE when cross referencing schema w/o name
PR-2023 - DGS-2073 Fix Protobuf converter for multiple map refs
PR-2022 - DGS-2036 Retain deprecated option on Protobuf messages, enums, etc.
PR-2018 - DGS-2036 Retain deprecated option on Protobuf fields
PR-2017 - DGS-2034 Add force query param when setting mode
PR-2016 - DGS-2026 Allow generating JSON schemas with additionalProperties=true
PR-2014 - DGS-2020 Allow optional to be overridden in JSON SchemaWrapper
PR-2013 - DGS-2018 Ensure fixed names in Avro union are scrubbed
PR-2012 - DGS-2018 Ensure names in Avro union are scrubbed
PR-2011 - DGS-2016 Add Protobuf compat rule for label
PR-2010 - Rename list in ContextResources to listContexts
PR-2008 - DGS-1987 Check for invalid contexts
PR-2003 - DGS-1976 Add support for context wildcard
PR-2000 - DGS-1965 Handle default ctx in context filter
PR-1990 - DGS-1879 changed timeout parameter
PR-1989 - Upgrade commons-compress to 1.21 (5.4.x and later)
PR-1985 - Add confluent-log4j as dependency
PR-1977 - DGS-1909 Fix AVRO_FIELD_DEFAULT_FLAG_PROP in AvroData
PR-1974 - Add batch methods to update handler
PR-1972 - DGS-1902 Use fully qualified names for Protobuf Map types
PR-1969 - SEC-2492 exclude guava version
PR-1968 - DGS-1880 Allow additionalProperties to be set in converter
PR-1967 - Logging schema size when register
PR-1966 - DGS-1869 Clean up mock client to be similar to cached client
PR-1965 - Remove use of WrappedParsedSchema in mock client
PR-1963 - DGS-1878 Remove use of SynchronizedCache
PR-1962 - DGS-1872 Remove use of VerifiableProperties from serdes
PR-1961 - DGS-1869 Reduce synchronization in CachedSchemaRegistryClient
PR-1959 - DGS-1563 Minor API cleanup
PR-1956 - DGS-1853 Fix handling of slashes with refs in maven plugin
PR-1955 - DGS-1563 Add getGroupId method
PR-1954 - DGS-1850 Prevent duplicate imports in Protobuf converter
PR-1950 - DGS-1563 Add contexts resource
PR-1925 - DGS-1563 Add support for contexts
PR-1949 - DGS-1515 Remove ZK utils used by ZK leader election
PR-1947 - DGS-1829 Handle duplicate refs during Protobuf Schema compat check
PR-1944 - Build fix for SR related to kafka prepareResponse function change
PR-1943 - DGS-1826 Fix NPE when struct default has null
PR-1945 - DGS-1825 Fix DataException when scrubbing invalid names (Avro)
PR-1942 - DGS-1825 Fix DataException when scrubbing invalid names (Protobuf)
PR-1939 - DGS-1820 Pass ObjectMapper to JSON Schema generator
PR-1937 - DGS-1820 JSON Schema validation should take a JsonNode
PR-1936 - DGS-1515 Remove deprecated ZK for leader election
910c9d9f - DGS-1794 Fix read only buffer in Protobuf converter
PR-1931 - DGS-1794 Don’t return read-only byte buffers from Protobuf converter
PR-1928 - DGS-1758: Allow AvroData take JSON null as default
PR-1923 - DGS-1737: MockSchemaRegistryClient doesn’t generate new schema id with protobuf
PR-1865 - DGS-1654 Upgrade Protobuf to 3.17.0; add optional support
PR-1922 - DGS-1752 Add rest of google common types for Protobuf
PR-1920 - DGS-404: Improving error messages - adding 40408 and 40409 for subject level compatibility and mode error
PR-1918 - small fix in deleteSchemaVersion for MockSchemaRegistryClient
PR-1917 - DGS-1734: MockSchemaRegistryClient need to perform a correct deleteSchemaVersion
PR-1907 - DGS-1658: Preserve field names when rendering Protobuf objects as JSON
PR-1901 - DGS-1726 Only convert wrappers to primitive if flag is set
PR-1902 - DGS-1727 Don’t throw exception when comparing array item
PR-1897 - DGS-1716 Remove logging of configs
PR-1893 - DGS-1607 Skip well-known Protobuf types on client side
PR-1896 - DGS-1704: override commons-compress
PR-1895 - DGS-1704: Upgrade jackson-mapper-asl
PR-1888 - DGS-1696 Pass previous value to LookupCache hooks
PR-1886 - DGS-1653: Add configs properties for SR Maven Plugin
PR-1858 - DGS-1622 Expose checkpoints to cacheInitialized method
PR-1856 - DGS-1607 Support rest of well known Protobuf types
PR-1848 - DGS-585 Add struct/empty Protobuf types
PR-1840 - DGS-585 Add Any/Duration logical types
PR-1838 - Fix documentation on the built package path
PR-1820 - DGS-1499 Add subject query param to schema REST APIs
PR-1827 - DGS-1522 Preserve sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, uint32, uint64 type in ProtobufConverter
PR-1830 - DGS-1528 refactor out SubjectValue class
PR-1828 - init was deprecated in MessageFormatter
PR-1823 - DGS-1513 Store offset and timestamp with cached values
PR-1818 - DGS-1484: throw InvalidConfigurationException to avoid NPE when schem
PR-1814 - Serializer now throws a user friendly message if not configured
PR-1813 - Fix NPE when checking compatibility
PR-1794 - Fix String.format in compatibility errorMessages
PR-1790 - DGS-1361 Add API to Test Compatibility against specified subject
PR-1789 - Change subject-level mode DELETE response type from String to ModeGet
PR-1780 - DGS-1328 Add DELETE API for subject config and subject mode
PR-1783 - Update to avro-data to support fixed type fields inside of a union
PR-1782 - DGS-1346 Add defaultToGlobal queryParam to GET /mode/{subject}
PR-1763 - Add logical type converter to Kafka avro serializer/deserializer (#1762)
PR-1323 - API update for Jetty 9.4.26.v20200117

Commercial Features¶

Confluent Server¶

All changes listed in the Kafka section also apply to Confluent Server

KGLOBAL-852: Send the correct consumer groups to the coordinator (#4718)
KGLOBAL-834: Prevent early cancellation of DelayedFetches which fetch from tiered storage
KGLOBAL-840: Fix bug with backward compatibility to old source clusters w/ consumer offset sync
KENGINE-33/KENGINE-51; lastCaughtUpTime is always reset by leader / Expansion should only consider caught-up followers
KSTORAGE-1834: followerRestorePoint is the Last Tiered Segment with offsets before localLogStartOffset
KGLOBAL-832: Fix local listener override for source initiated links, add multi-listener tests
KC-1906 Call maybeExpandIsr in all cases when handling AISR response
KGLOBAL-294: Reverse connection calls are stuck when source controlle
KGLOBAL-822: Fix bug for altering topic filters for cluster links
KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown (#11351)
KAFKALESS-655: Clear Executor state on exceptions encountered before proposals start executing
KC-1902: Disallow MDS with KRaft
KGLOBAL-742: LINK_FAILED replica status sholdn’t stop failover. (#4531)
CPKAFKA-7455: upgrade system test archive failures (#4528)
CC-13747 CIAM-1263 FF-4419: Limit max retries in MDS URL Refresh loop
CPKAFKA-7608: reassign license topic in unclean leader election syste
KSTORAGE-1802: remove linux-diskstats metric from CP 7.0
KGLOBAL-779: Avoid truncation in circular mirror test when recreating source topic
KGLOBAL-751: Improve error handling for source initiated link creation without dest link
KGLOBAL-750: Allow failed links to be reconfigured after password encoder secret update
KGLOBAL-741: Improve error message for invalid cluster link configuration
CIAM-1253 Fix Logging of various timeouts in ce-rest-auth RestClient
KGLOBAL-739: Notify cluster link fetcher thread on high watermark update to resume fetching
CONFLUENT: Remove force_use_zk_connection from topic command
SEC-1537 RestClient Retry Logic Fixed (#4395)
KC-1878 Debug Logging for LDAP Context creation
KAFKALESS-592: Revert rebalance storm prevention code to require full fixes.
KGLOBAL-686: Detect circular mirrors for older IBPs by tracking epoch update state
KGLOBAL-713: Use topic ids to detect source topic deletion and recreation to avoid circular mirror
KGLOBAL-720: Don’t fail LeaderAndIsr request for mirror partitions if link metadata not yet processed
KC-1800: Upgrade Guava to 30.0-android
KGLOBAL-719: Report link count metric only on controller
KSTORAGE-1713: Document the code asking user to ignore KIP-405 tiered storage configuration in CE-Kafka
KC-1850: Ensure topic ID upgrade does not overwrite pending isr state when upgrading to topic IDs (7.0.x)
KAFKALESS-528: Add shutdown support to multi-broker removal.
KGLOBAL-700: Add more TRACE level logging for periodic tasks
KAFKALESS-354: Update the Broker Removal State Machine to include all necessary phases
KAFKALESS-586: Do all removal ProgressListener callbacks in the PhaseExecutor
KC-1865: KRaft BrokerServer missing certain features/fixes
KC-1875: Do not enable both KRaft and CL in ducktape
KGLOBAL-692: Don’t truncate mirror topic for source epoch 0 when source topic is recreated
KC-1869; IllegalStateException in ControlMetadataBatch due to topicId mismatch
KC-1852: Upgrade commons-compress
KGLOBAL-478: Replace references to source connection origination with source initiated link
KAFKA-13209: Upgrade jetty-server
KAFKALESS-437: Return BrokerShutdownStatus based on request version in order to not break older clients
KAFKALESS-398: Add UNKNOWN status value to balancer operation statuses
KGLOBAL-696: Remove unnecessary purgatory when cluster linking is disabled
MINOR: Controlled shutdown is triggered twice when a Broker shuts down
KGLOBAL-695: Change purgatory to val from method to mitigate RCCA-3631
KGLOBAL-548: Rename topic mirror to mirror topic in codebase
KGLOBAL-689: Adding lag to kafka-mirrors –describe
KGLOBAL-682: Propagate SSL handshake failures during cluster link validation to clients
KGLOBAL-672: Process cluster link pause state during link creation and broker startup
KAFKALESS-588: Update system tests to remove multiple brokers at once
MMA-9932 Max timestamp changes from KIP-734
KAFKALESS-594: Extend BrokerShutdownManager to shut down multiple brokers
KGLOBAL-665: Fixed bug with auto mirror topic create and cluster link config updates
KAFKA-12155: Metadata log and snapshot cleaning #10864
KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-m
KGLOBAL-645: Don’t count non-persistent failures towards persistent failure timeout
KGLOBAL-663: Added RetriableException to return true for isRetriable to prevent excessive logging
KGLOBAL-657: describeMirrors() state should return STOPPED for promoted/failover mirrors irrespective of cluster link state.
AUDIT-494: Add configuration setting to include client addresses in audit logs (default false)
KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-mirrors cli.
KGLOBAL-658: Changed error message for auto-mirrored topic deletion
KSTORAGE-1644: Allow empty last segment to have missing offset index during recovery
KAFKALESS-402: Add replica exclusion system test
KGLOBAL-633: Update buffered bytes for reverse channels, fix memory leak
KAFKALESS-568: Rephrase broker failure detection message
KSTORAGE-1655: Broker-side envelope encryption for tiered data
KGLOBAL-491: Implementing Confluent specific listConsumerGroupOffsets API for CL
KGLOBAL-571: Add system test for auto mirror topic creation
KGLOBAL-620: Introduce new mirror topic state LINK_PAUSED to represent cluster link level pause.
KC-1843: Validate KIP-500 compatibility with confluent configurations
DGS-1903: Add config option for missing schemas TTL
KGLOBAL-606: Disallow addition of conflicting topic filters
KSTORAGE-1622: Update the documentation for Tier Object Store Compatibility Checker
KAFKALESS-538: Handle offline brokers for RemoveBroker request idempotency guarantees
SBC: Wait for Executor task to finish in broker removal
KGLOBAL-426: enable cluster linking in default server.properties file
KSTORAGE-1700: Increase the Future await timeout allotted to each ArchiveTaskTest.
KAFKALESS-478: Modify validation to handle multiple brokers
KGLOBAL-634: Allow mirror topic creation in kafka-mirrors without source-topic argument
Add a new thread pool to execute state machines
KGLOBAL-605: Prevent deletion of auto-mirrored topic
KAFKALESS-538: Make the broker removal API idempotent
PRSP-836 : Add broker configs to include list
KC-1817; Serialize principal details as JSON in request logs and add extra fields
KAFKA-13104: Controller should notify raft client when it resigns #11082
KC-1824: KC-1825: Infinite recursion in Partition with ZkIsrManager & Deadlock
KC-1810: Upgrade commons.io in ce-kafka
KGLOBAL-552: Populate describe mirrors response with last source fetch high water mark for stopped mirrors.
KGLOBAL-564: Enable cluster link fetcher destination lag flow control
KGLOBAL-569: Automatic mirroring
CONFLUENT: Make cluster link extensions in KafkaAdminClient package-private
KAFKA-13090: Improve kraft snapshot integration test
KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
KAFKA-12964: Collect and rename snapshot files prior to async deletion
KSTORAGE-1580: fix NPE when record==null in append
Safely log configs for IncrementalAlterConfigs error
KC-1789: asynchronously flush partition metadata file
KGLOBAL-601: Add more info for debugging cluster link lag
KAFKALESS-479: Modify BrokerRemovalStateTracker to track state for multiple brokers at once
KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
KAFKALESS-478: Add minimal multi broker removal validation
KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
KGLOBAL-603: Convert the Kafka Cluster Link command line option ‘–link-name’ to ‘–link’
KSTORAGE-1206 Enable system tests to be run with durability audit enabled
KAFKALESS-537: Update system tests to work wtih new broker removal.
KCFUN-84: Make sure Broker pick up the default config.
KSTORAGE-1653: add spurious offset out of range assertion to the tiered storage system tests
EVENTS-278: Update TelemetryReporter on how to load exporter
KGLOBAL-564: Throttle cluster linking fetch
KAFKALESS-540: Create topics for executor test before creating exclus
KAFKALESS-537: System test changes for exclusion-enabled broker remove.
KGLOBAL-440: Change mirror-topic-byte-rate to mirror-topic-byte-total
KAFKALESS-477: Shutdown-free single-broker removal with exclusion.
KAFKA-13007; KafkaAdminClient getListOffsetsCalls reuse cluster snaps
KGLOBAL-354: Make socket.send.buffer.bytes dynamically updatable on brokers
KAFKA-12520: Ensure log loading does not truncate producer state unless required
KSTORAGE-1674: fetch below log start offset may incorrectly scan forward
KAFKALESS-401: Prohibit replica placement on brokers that are excluded
KAFKA-12155: Metadata log and snapshot cleaning #10864
Upgrade protobuf to latest 3.17.3
KAFKALESS-523: Topic placement constraints not used in CreatePartitions
CNK-1420; Add broker tags to the DescribeCluster API
KPERF-3; Augment Fetch Session w/ Partitions’ metadata and process only active partitions
KAFKA-10587; Rename kafka-mirror-maker CLI command line arguments for KIP-629
KDATA-33: Add checksum validation of tier state file
KC-1567; Add support to disable delete retention for __consumer_offsets
KSTORAGE-1664: Force defaults for TierObjectMetadata entries except OpaqueData
KAFKALESS-447: Fix Property based test build configuration
KC-1778: partition metadata file flushed twice on partition creation
KGLOBAL-579: Fix mirror topic creation
KAFKALESS-436: Additional test for balancer startup failure
adding support for exporting messageSizeHistogram in Log.scala
KAFKALESS-466 Generate topic level properties for balancing
KC-1784: Use URI instead of URL for RBAC writer comparison to avoid hostname resolution issues
CNK-1600: Fix FetchRequestBenchmark.testFetchRequestForConsumer
KSTORAGE-1503: add metric for spurious OffsetOutOfRangeException
KPERF-103; Check shrink/expand status when AlterIsr returns
KAFKALESS-421: Update broker removal persistence records for Multi-Broker Removal
KCFUN-77: Add interface to retrieve TLV (type-length-value)
KSTORAGE-1450 Add metrics for unclean leader recovery
KAFKALESS-502: Interrupt in-progress SBC operations when new replica exclusions are detected
KDATA-26: Integrate TierMetadata with CheckedFileIO implementation
KENGINE-4; KAFKA-12890; Consumer group stuck in CompletingRebalance
KGLOBAL-533: Migrate mirror topic creation in CLI from kafka-topics to kafka-mirrors
KSTORAGE-1572 - Better Error Reporting for S3 Region Setting
KAFKALESS-441: Add Property based test for Replica Placement goal
KSTORAGE-1641: Fix LogLoader to pass materialized view of segments for deletion
KCFUN-37: Add interfaces and plumbing for PROXY protocol support
KAFKALESS-514: Update SBC to use Junit5
KAFKALESS-395: PBT same goal and cluster means same proposal
KSTORAGE-1624: Logs with holes in them may trigger spurious offset out of ranges
KAFKA-12701: NPE in MetadataRequest when using topic IDs
CIAM-971: Support user resourceId for RBAC authorization
KGLOBAL-529: Better handling of auth failures when stopping mirrors
CCLOG-403: Connector Data Preview Rest Extension
CCLOG-403: Connector Tracing Update delivery timeout.
KGLOBAL-527: Add ttl for old encryption secret, re-encrypt cluster link credentials during restart
KSTORAGE-1606: FileTierPartitionIterator incorrectly handles mixed size prefixed entries
Fix failing CL sytem tests from from KGLOBAL-532
KCFUN-74-2: Dynamically access averageServiceRequestTime config
KGLOBAL-532 Removed redundant mirror-action stop command
KC-1761: Controller Event Loop Perf Regression
KGLOBAL-440: Remove lkc from topic and link-name tags.
[KSTORAGE-1457] separate fetcher for in sync replicas
KGLOBAL-475: Add new error code for cluster link operations while CL is disabled.
KGLOBAL-445: Rename the mirror/replica state from LINK_NOT_AVAILABLE to SOURCE_UNAVAILABLE to make the error string more customer-friendly.
remove relocation for dependencies that no longer exist
KGLOBAL-440: Mirror topic lag
CPKAFKA-7255: Change SIMPLE_AUTH to ACL_AUTH
KC-1656: Metrics and improved tests for ConnectionMaxAge work
CONFLUENT: server_prop_overides was renamed to server_prop_overrides
CPKAFKA-6198: Fix broker removal system test parsing error
CONFLUENT: Upgrade protobuf and flatbuffers plugins
KSTORAGE-1609: Fix ConsumeBenchWorker exit behavior for maxMessage
KSTORAGE-1611: Allow retention settings in trogdor script and fix few other things
[WIP][CPKAFKA-3168] Augment ProduceResponse error messaging for specific culprit records as part of KIP-467
KSTORAGE-1609: Fix ConsumeBenchWorker exit behavior for maxMessage
KDATA-63: Postpone startoffset lapse under heavy user delete event, for a window of time for self recovery
KSTORAGE-1569: Incorporate LogLoader and recovery logic refactor from AK
CONFLUENT: rename all _confluent-license usages to _confluent-command
KC-1647: Include confluent. prefix for close.connections.on.credential.delete config
CNK-941: Connection rate auto-tuning
KAFKALESS-444, -445 Added NetworkCapacityGoals PBT Tests
KC-1764: Ensure dynamic logins are initialized before broker plugins are started
CONFLUENT: Handle TopicAuthorization while checking “_confluent-license” topic existence
CCLOG-403: Add support for data tracing in connect [Connector Data Previews]
KAFKALESS-412: Do not run SBC operations when there are broker replica exclusions present (part 1/2)
CONFLUENT: Fix downstream build failures after recent LicenseStore changes
KDATA-14: Add healthcheck for Durability Audit
KDATA-15 : DA to be more tolerant for missing registration and retention change event.
KAFKA-12791: ConcurrentModificationException in AbstractConfig use by KafkaProducer (#10704)
KAFKA-12856: Upgrade Jackson to 2.12.3 (#10778)
KC-1266: Update LicenseManager to support both _confluent-command and _confluent-license topics
KPERF-7; Add listener mechanism in the Log and Partition layers
Connection metrics for Broker Load measurement
KAFKALESS-442: Add ReplicaCapacityGoal PBT
KAFKALESS-443: Add DiskCapcityGoal PBT
METRICS-3264 Make TELEMETRY_PROXY_PASSWORD from STRING type to PASSWORD
KAFKALESS-415: Keep SBC AdminClient open through full shutdown.
KCFUN-7: Make sure handshake time is measured upon failure.
KSTORAGE-1478: topic recreated after deletion due to LISR request
CPKAFKA-3264: TierTopicManager thread dies materializing CATCHUP TierPartitionStates
KC-1656: Add a unit test for connection max age updates and handling.
KSTORAGE-1596: Fix topic configuration and params in trogdor scripts
CONFLUENT: Add comment to Partition.topicId() that was lost during the merge
KC-1332: reduce queue memory usage in producer id maintainence
CONFLUENT: Improve ConfluentMetricsFormatter.getString when default is null
KDATA-60: Materialize delete partition
KSTORAGE-1594: Use segment.bytes topic config instead of confluent.tier.segment.hotset.roll.min.bytes
KGLOBAL-487: Make new fields in ListClusterLinksResponse ignorable for compatibility
KSTORAGE-1593: Fix exception stacktrace printing in JmxMetricsScraper
KGLOBAL-510: Fix testDestLeaderChanges intermittent failure
KDATA-21 : Integration test gaps DA.
KAFKALESS-384: Add property based test for RackAwareGoal
KAFKALESS-378, KAFKALESS-393: Add invariants for PBT
KAFKALESS-372: Implement replica exclusion handling and persistence
SEC-298 - Log errors when MDS can not start
KC-1646: Dynamic SASL/PLAIN login for Kafka clients in broker plugins
KDATA-24: CheckedFileIO package for providing checksum backed File IO
KSTORAGE-1397: Fix test for Duplicate Registration with TierTopicConsumer
METRICS-2878: add event emitter api
KGLOBAL-487: Add version check for available field for version < 2
KSTORAGE-1528: Remove flakey tests for TierObjectStoreFactory
KSTORAGE-1268: Filtering ability for durability db dumping tool
KAFKALESS-369: Introduce kafka-replica-exclusions.sh CLI
CNK-1373: Delay external connectors until both metadata and acceptors load
KSTORAGE-1569: Incorporate changes from AK leading up to recovery logic refactor
CONFLUENT: Re-enable testClientSideTimeoutAfterFailureToSend
KSTORAGE-1569: Introduce LogSegments class abstracting the segments map
KSTORAGE-1544: Convert confluent.tier.max.partition.fetch.bytes.override into a dynamic broker config
KSTORAGE-1487: support Tiered Storage TLS for S3/PureStorage via configuration properties
KSTORAGE-292: fsync the parent of FileTierPartitionState files
KSTORAGE-1397: Prevent race on maybeBeginTierMaterialization
KGLOBAL-114: Removing controller metrics to consolidate mirror topic metrics
KC-1610: Upgrade dataformat-cbor version (#3589)
CONFLUENT: Fix RbacRolesTest
KDATA-6: Prevent logging of AuditJob’s skip message for compacted partitions
MINOR: Adds a check before reading empty partitionMetadataFile to avoid NPE
CONFLUENT: Don’t publish tier-object-store-compatibility-checker
KAFKALESS-369: Admin API changes for {Alter,Describe}BrokerReplicaExclusions
CONFLUENT: Adjusting get_offset_shell parameters in reassign_partitions_test.py
CONFLUENT: Pass correct requestHandlerAvgIdleMetricName
CNK-1567: Expire old connections unconditionally (at SocketServer level)
KAFKALESS-358: Update remove broker cli to support multi broker removal with optional –no-shutdown flag
CONFLUENT: Remove stray artifacts block
KAFKALESS-356: Update removeBrokers AdminApi with shutdownCompleted flag and BrokerReplicaExclusion status
Disable BrokerLoad for KRaft servers
Guard against null client ID in request size calculation
KC-580: remove epoch cache flush when recovery point is not advanced without flushing
QEC-5889: Allow SBC commands to run with custom security config (#3531)
KC-1624: fix perf regression due to removal of BufferSupplier (#3555)
KC-1610: Upgrade dataformat-cbor version
KC-1618: Adapt the KRaft README and example configs for CP
Replaced CL source soak with new one
KSTORAGE-1567 Fix errorStateReachedViaFenceEvent flag for log start offset recovery
KAFKALESS-369: Add DescribeBrokerReplicaExclusions RPC (part 2/3)
KAFKALESS-369: Add AlerBrokerReplicaExclusions RPC (part 1/3)
CNK-1685: Add configs for broker-wide limit on produce and consume bandwidth
KSTORAGE-1568: object durability audits must use key baseOffset not virtualBaseOffset
KC-1624: fix perf regression due to removal of BufferSupplier
KC-1614: Update BrokerFailureDetector to use executeRemoval
QEC-5889: Allow Balancer commands to run with custom security config
KDATA-10: Send audit request for delete record request before start offset request
KGLOBAL-411: Don’t update destination high watermark when processing fetch response
KPERF-2; Leader resets LastCaughtUpTimeMs of caught-up replicas when it appends to its log
CNK-1695: Log cause of unfixable hard goals in goal violation self-healing
ST-5293: Upgrading version of netty
CONFLUENT: Fix javadoc on KafkaBasedLog constructor
KGLOBAL-414: Reset AlterIsr in-flight state for duplicate update requests
KSTORAGE-1455: Bump object store dependencies to latest
KSTORAGE-1525: Track inbound throughput of compacted partitions
CONFLUENT: Change visibility of LicenseConfig constants to public
CIAM-635 Add reason argument to AuthWriter
CONFLUENT: stopPartitions should call clusterLinkingManager.removePartitionsAndMetadata
CONFLUENT: TierMessageFormatter should inherit from o.a.k.c.MessageFormatter
CONFLUENT: Pass no channelManager in AutoTopicCreationManagerTest.testErrorWithCreationInZk
CONFLUENT: Fix admin test failures due to bug in ConfigHelper.filterTopicConfigs
CONFLUENT: Fix failing testGenerateAssignmentWithInvalidPartitionsFails
CONFLUENT: Close backend in TierMetadataValidatorTest
CONFLUENT: Remove PartitionTest.testDelayedFetchAfterAppendRecords
Handle node.id in metrics context and null auditLogProvider for KRaft
CONFLUENT: Compare filtered configs instead of config counts in PlaintextAdminIntegrationTest
CONFLUENT: Convert UUID usage into Uuid in Log and MergedLog
CONFLUENT: Remove some of the default arguments from the Log/MergedLog constructors and apply methods
CONFLUENT: Add listeners to all of the CE requests
KSTORAGE-1511: Remove the delay of 1ms when scheduling flush in Log.roll()
KSTORAGE-1545: Add test verifying that the TierReplicaManager deletion hook is called
CIAM-827: Create Roles.json entry for Data Plane DG Role(s)
[KSTORAGE-1474] set unclean.leader.election.enable to false for tier state topic (#3429)
KSTORAGE-1545: Partition deletion does not stop leadership of tier tasks
KSTORAGE-1379: Durability Audit Update commit offset after alert
KAFKALESS-400: Handle non-PLAINTEXT setups for SBC commands (#3488)
KAFKALESS-409: Handle License topic MinISR of 2 in SBC tests
KAFKALESS-400: Handle non-PLAINTEXT setups for SBC commands
KAFKALESS-409: Handle License topic MinISR of 2 in SBC tests
CIAM-809 : Create new role(s) for SA issue
KAFKALESS-365: Validate cluster configuration before creating cluster object
KGLOBAL-347: Make fetcher thread count dynamically configurable
Start licesne validator after unfencing KRaft broker
CONFLUENT: Fix all of the tests in LogManagerTest
Handle node.id in metrics context and null auditLogProvider for KRaft
KAFKALESS-387: Respect broker concurrency limit even for “dataless” m
Fixed BrokerEpochIntegrationTest, ensure compatability for error, and minor style fix
KAFKA-12381: remove live broker checks for forwarding topic creation (#10240)
KC-1580 Remove clearPending method from AlterIsrManager
KAFKALESS-387: Respect broker concurrency limit even for “dataless” moves
KAFKALESS-367: Add first PBT test for cluster model
CONFLUENT: Manually apply upstream revert
KAFKA-12600: Remove deprecated config value default for client config client.dns.lookup
KAFKALESS-366: Add support for property based testing in SBC
CCLOG-503 Enabling partitioning key while produce events from KafkaExporter
CONFLUENT: Add listeners to all of the CE requests
[KSTORAGE-1474] set unclean.leader.election.enable to false for tier state topic
KCFUN-5: Fix NPE thrown by clusterId
KSTORAGE-1494: Prevent deletion of empty compacted segments
KSTORAGE-1497 Tool to inject TierPartitionUnfreezeLogStartOffset event
KGLOBAL-282: Ensure mirrors progress after non-fatal errors when issues are fixed
CCLOG-353: Emitting of ConnectLogEvents when connector/task fails
KSTORAGE-1304 - Restore Log Start Offset Feature Implementation
KAFKALESS-342: Retry topic description failures in the Executor
DGS-1350: Reduce schema validation logging output
KAFKALESS-348: Fix previous status null case returned from EvenClusterLoadStateRecord
KSTORAGE-1476: perf regression in log load caused by incorrect use of containsKey
KSTORAGE-1471 Fix fallback bootstrap server (#3405)
CL-14: Failed cluster link doesn’t doesn’t reflect in mirror topic state
KSTORAGE-1471 Fix fallback bootstrap server
KSTORAGE-1442: refactor TierPartitionState data structures to enable future compaction support
SBC-335: Handle KafkaSampleStore loading that fails.
SBC-309: Update EvenClusterLoad status in persistenceStore and return from it
SBC-306: Retry on InvalidReplicaAssignmentException
KGLOBAL-325: Fixed bug where sync acls task could get stuck
KAFKA-12583: Upgrade netty to 4.1.62.Final
KAFKA-12400: Upgrade jetty
KAFKA-12334: Add the KIP-500 metadata shell
KAFKA-12297: Make MockProducer return RecordMetadata with values as per contract

Security¶

Fix various test failures
APPSEC-287: Override commons-codec
DGS-2010 Add support for contexts in authorizers
Update KsqlRestAuthorizer with new checkPrivileges() method
Fix signature of ConnectorsResource.restartConnector
DGS-1903: Add missing schema cache to LruSchemaRegistryClient
SEC-2533 upgrade commons-io to 2.11.0
SEC-2493 Remove bouncy-castle explicit dependency
DGS-1563 Add contexts resource
DGS-1563 Add support for contexts
Fix SR ACL tests due to ZK removal
FF-4684 mTLS support, second attempt
DGS-1691: Perform license checks in background thread
FF-4684 Enable Schema Registry RBAC authorization with mTLS authentication (reopen PR346)
KQE-229: Update vertx-core to 3.9.7
Replace usage of deprecated SimpleAclAuthorizer with AclAuthorizer
Pins Ksql to a jetty-jaas 9.4.28.v20200408
DGS-1350: Reduce schema validation logging output
DGS-1499 Add subject param to APIs
SEC-2023: TestUtils method was renamed upstream
DGS-908: Schema validation metrics api integration
DGS-1351: Add more schema validation metrics
DGS-1328 Add authorizations for DELETE of Config and Mode endpoints

Secret Registry¶

MINOR: Remove usage of WorkerConfig.REST_PORT_CONFIG config
CC-15204: Upgrade bcprov-jdk15on to 1.68
Remove deprecated sasl enabled mechanisms usage
Remove usages of ClientDnsLookup.DEFAULT

Control Center¶

MMA-11113: Feature flag changes for new C3 broker settings
PG-389: short-lived http connection for non-idempotent reqs and configurable mds client
MMA-10607: fix testBadClientDoesNotStallRegistration by awaiting method invocation before verify
KGLOBAL-684: Change naming convention to match upstream
fix build failure due to StreamsConfig change
MMA-7623: hide stack trace in error response (#2178)
Use confluent repackaged log4j
MMA-10895: add c3 property prefix to auth.bearer.roles.claim
Upgrade protobuf from 3.17.0 to 3.17.3
PG-205: Make bin scripts compatible with all c3 modes
fix protobuf version in accordance to DGS-1654
replace default jar with fat hjar
refactor-injector: centralize creation of Guice injectors
add new feature flags to enable/disable embedded rest proxy
More deprecated APIs to remove
Remove TestDriver Deprecated API
MMA-10673: Introduce a new SSL-specific SMTP port configuration for email alerts
Upgrade Armeria to match the netty version
PG-182: Create MMO version of ControlCenter
PG-178: Create MMO version of HealthCheck
Replace usages of methods/classes deprecated for removal in Apache Kafka 3.0 (part 2)
Replace usage of kafka.common.MessageFormatter with o.a.k.c.MessageFormatter
Use KafkaStreams.store(StoreQueryParameters) instead of the overload removed in Apache Kafka 3.0
PG-190: Create MMO version of KafkaHelper and AlertsResource
PG-198: Filter available resources based on running mode
PG-181: Create Mode annotation and update C3 application
Add websocket parameter consumerPollDelay
PG-138: Add new endpoint in AlertsResource to return available triggers
PG-124: Refactor ControlCenterApplication to use reflection
Add control-center mode to feature flags
PG-109: add confluent.controlcenter.mode.enable to change c3 mode
give unique path to v3 web-socket, implement ControlCenterKafkaRestContext.getProducer
Replace usage of deprecated ConsumerRecord/RecordMetadata.checksum()
Fix KafkaResourceTest due to breaking change in ce-kafka
Disable audit logging to fix build
add C3Stream.split to fix build
MMA-10034: add cluster access check and add refresh logic
CL-130: Fixed imports after changing packages in ce-kafka
MMA-9949: Integrate KafkaRestApplication in C3
MMA-9967: loosen restrictions for customized topics’ data export
MMA-9971: remove “config” flag as part of usage prompt

MQTT Proxy¶

Added telemetry-events-api as compile dependency.
Fix SocketChannel mock in PipelineFactoryTest
CC-15588: Added confluent-serializers-new as explicit dependency.
DGS-2161: Inherit netty.version from confluentinc/common
CCMSG-1251: log4j migration to confluent repackaged version
CCMSG-1101: Upgrade Netty to 4.1.63.Final
CC-14445: Remove compile dependency on Kafka broker
Replace usage of deprecated SaslConfigs.SASL_ENABLED_MECHANISMS

Metadata Service¶

SEC-2666: Fix test breakage caused by Jackson 2.12 CoercionConfig
Override commons-codec to 1.13 in 5.3 Only
SEC-2440 - MDS - Fix MdsScope Validation and messages and add tests
add negative tests for testing changes made in lookupPrincipalsWithRoleOnResource
add positive tests for testing changes made in lookupPrincipalsWithRoleOnResource
correct spelling
Add comment to explain try-catch block
Add try catch to first authorize on SECURITY_METADATA
Fix lookupPrincipalsWithRoleOnResource to authorize on valid ResourcePattern using DESCRIBE_ACCESS operation
CIAM-989 Move Data Governance roles to own namespace (#605)
SEC-1034: log4j migration to confluent repackaged version
CIAM-1060: Fixed CPD crashed issue due to wrong credentials to DB and Events Kafka
CIAM-1048 - DB Migrate Extractor State table to stag, prod, cpd
CIAM-1050: Improve checking in HttpRefreshRoleBindingTest
CIAM-725: Extractor Full Org Refresh via Http call
CIAM-347 Prevent OrgAdmin from deleting their own OrgAdmin role
Update test to check for Kafka HTTP Server.
CIAM-1017 Allow multiple policies at a binding scope
CIAM-999 Use LaunchDarkly to control whether to publish updated role
Fixed an accidental change in halyard config
rbac-extraxtor deployment fix
CIAM-726 Integration test of Dataplane enforcement
CIAM-990 Escape input in healthcheck query
CIAM-710 Sync rolebindings periodically
Fix NPE in Scope Comparator
CIAM-709: Build Extractor Rest Foundation
CIAM-709: Fixed halyard chart bug
Adding stage/prod semaphore deployment reference
CIAM-709: Added more configurations for rbac-extractor
CIAM-696:Register the new version for cpd
rbac-api-server should have explicit dependency on confluent-licensing-new
Add log4j package in rbac-extractor pom file
CIAM-946 Migrate Stag to include reason field in role_binding
Add jakarta rs api explicit dependency
Add explicit guava dependency to rbac-common
CIAM-946 Migrate Stag to include reason field in role_binding
CIAM-693: Adding Sample Config to rbac-extractor
CIAM-706 Implement RBAC Writes Infrastructure.
SEC-2393 Remove reliance on KafkaPrincipal.fromString()
CIAM-945 Remove cc_rbac_api user via dbmigrate
CIAM-707 Recently Updated Role Bindings query
CIAM-703 JDBC Query Infrastructure
CIAM-680: Updated Image.repository in halyard.mk
CIAM-680: Fix in helm/halyard to handle mds specific version
CIAM-680:Dummy check-in to re-initiate build
CIAM-680:Updated for halyard auto-deploy
CIAM-941–Fixed master build error.
CIAM-941 – Fixed double build docker image which run mvn install twice.
Increase DB Test query timeouts (3 -> 10 seconds)
CIAM-914 Migrate prod to have two rbac_api users
CIAM-894 Authorizer test for new DG RBAC dataplane roles
CIAM-680: Adding cc-semaphore.mk to makefile
CIAM-914 Migrate stag to have two rbac_api users
CIAM-680: Added cc-cpd.mk for helm version update
Use postgres 12.5
CIAM-680 Support for halyard/helm version update and deployment
CIAM-702: Add schema for ResourcesRequest to Swagger Spec
CIAM-920 — Fixed version and labels docker image
CIAM-324 DB Migration to Support multiple db users for password rotation
CIAM-863: Updating directory for halyard set-version
removed chart.target from the Makefile
CIAM-863: Added helm charts and halyard spec files
CIAM-679 Build rbac-extractor docker image
CIAM-892 Allow multiple namespaces to be specified in /roles and /rol
CIAM-635 : Add endpoints to allow for Organization Merges
CIAM-843 Address Integration Test Gaps for V2 Resources
Increase DB Test query timeouts
CIAM-809: SA Issue : Create new role(s) for SA issue
CIAM-675 Create Pseudo-Extractor shaded jar
CIAM-240 - Create v2 tests for Authorize to parity Topic/DB Framework
CIAM-765 Refactor/Rename Validation Classes common for V1 and V2 API
CIAM-611: Add smoke test for use in ce-kafka-images build
CIAM-354: CLI : Misspelled Role Name does not throw error for rolebinding list
CIAM-457: Move healthcheck endpoint to v2alpha version
CIAM-659 - Add implementations for new interface methods added upstream

Replicator¶

REPL-1810: Removing ZK dependencies from Replicator Verifier
REPL-1774: Increase default value of offset.translator.batch.size
REPL-1655: Removed logging of properties that could expose secrets

Confluent Platform Docker Images¶

Confluent Platform Docker Images will no longer ship with Yum/Apt configuration that allowed users to update running Confluent Platform containers to the next Confluent Platform release. Confluent recommends upgrading Confluent Platform service components running in containers by replacing the running container with the next Confluent Platform released docker image and using docker volumes to persist data between upgrades.

Confluent Platform Documentation¶

DOCS-10709: Update ksqlDB links to use versioned role in Overview topic
DOCS-10671 - Add missing release notes
DOCS-10677: Update metadata for the Health+ overview topic
DOCS-10117 - Release notes for 7.0.0
DOCS- 10612 - Update System Requirements page for Control Center
DOCS-10176: Update steps in CP quick start for downloading tar file
Docs 9478 - add anchor to disable metrics reporter
DOCS-9329: Update monitoring topics with autogenerated metrics for CP 7.0 (and DOCS-10602)
Docs 9149 - Management-only mode changes and docs refresh
DOCS-9333: Update the CP ksqlDB Upgrade Guide for 7.0
Generate Changelog for 7.0.0
DOCS-9326: Update the Streams Upgrade Guide for CP 7.0
DOCS-10116: Update the CP upgrade guide for 7.0.x (and DOCS-8752)
DOCS-9344: Fix metadata description in ksqlDB tutorials index
DOCS-9344: Update CP ksqlDB resources for CP 7.0
DOCS-8936: Add RBAC role bindings to Kafka Streams Security topic
DOCS-10428: Move CP Quick Start topic up one level in the TOC
ST-6458: Add CP8.0 Deprecation Notice
DOCS-9302: Update Health+ content for CP 7.0
DOCS-10232: Add new Telemetry Reporter metrics (and DOCS-9330)
DOCS-9500 7.0.x compatibility docs
DOCS-10221: Retire legacy ksqlDB tutorials
DOCS-9334: Update CP ksqlDB compatibility table for CP 7.0
DOCS-9334: Update CP ksqlDB Supported Versions table for CP 7.0
DOCS-9356: Bump ksqlDB version to 0.21.0
ST-5633: Add instructions for Confluent Clients repo
DOCS-9993: Remove legacy quickstart/includes directory
DOCS-7844- Refactor the CP quick start
KGLOBAL-604 Update CP 7.0 cluster linking docs commands: change –link-name to –link
DOCS-9139 Streamline DOCS README, move Sphinx/RST info to wiki
DOCS-9065: Copy new PAPI content from AK docs
DOCS-7416 Add compatibility API endpoint for Schema Registry new in CP 7.0