Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Version 7.5.4

Released April 2024

Community Features

Common

  • 4fb11814 - Remove auto-generated CODEOWNERS. It’s inaccurate.
  • 52b0f806 - Remove CODEOWNERS file
  • 7a96a215 - Add service.yml
  • c4b849a5 - update jose4j
  • 4c8d3021 - update jetty to 9.4.54
  • PR-581 - fix: no implicit activation of docker-arm profile
  • PR-563 - added a new docker-arm profile to build docker containers in arm64

Kafka

  • PR-1127 - Update jose4j to 0.9.4 (#15284)
  • PR-1073 - Update jetty to 9.4.54.v20240208
  • PR-1015 - update bcprov-jdk15on
  • PR-15177 - Checkpoint restored offsets instead of written offsets (#15177)

ksqlDB

  • PR-10243 - Update Commons-Compress and commons-io
  • PR-10238 - Update apache.io.version
  • PR-10237 - Update commons-compress
  • PR-10223 - Change groupID to a more secure name
  • 2f84873b - Change groupID to a more secure name
  • 5612f336 - Increase execution time limit to 6 hours.
  • PR-10198 - Set timeout to 6 hours on branches starting on 7.3.x
  • 30544a65 - Cherry pick the replacement of easymock to mockito from newer branches.
  • PR-10195 - Add pinned nano version for common on 7.3.x
  • PR-10196 - Add pinned nano version for common for 7.4.x
  • f969d5c0 - Enable pinned nanoversions.
  • b49bbcbb - Specify type parameter for CommandStatus.
  • c7288dd0 - Remove maven cache restore.
  • cea27ec9 - set java version to 8
  • cd59a75a - Increase the execution time limit.
  • 26db19fd - Fixed the extra args in service.yml file.
  • PR-10189 - fix: configure KsqlBoundedMemoryConfigSetter in StandaloneExecutor mode
  • PR-10175 - fix: configure KsqlBoundedMemoryConfigSetter in StandaloneExecutor mode
  • PR-10178 - fix: configure KsqlBoundedMemoryConfigSetter in StandaloneExecutor mode
  • 37e06168 - Fix the parent version in commons.
  • 5467ea20 - set java version to 8
  • 0a1cf171 - Increase the execution time limit.
  • db81cd09 - Fixed the extra args in service.yml file.

REST Proxy

  • PR-1250 - Allow to set KafkaAvroSerializerConfig in KafkaRestConfig (#1229)
  • 05925773 - cluster_id is not validated [cp version]
  • PR-1235 - Deflake KafkaConsumerManagerTest.testBackoffMsControlsPollCalls test
  • PR-1236 - disable flakey test

Schema Registry

  • PR-3059 - fix: nested schemas were not considered in matching the correct Json Union schema
  • 38e98145 - Handle nulls in enum in JSON Schema converter
  • 78ae7f7b - Fix merge issue
  • c84f735e - Update client/src/main/java/io/confluent/kafka/schemaregistry/rules/RuleContext.java
  • PR-3038 - feat: Added ReferenceSchema support to allOf
  • 2808930e - Add test to register same schema with no ID in IMPORT mode
  • ae0e3140 - fix: invalid behaviour for stringSchema in allOfToConnectSchema
  • 6dcb41f7 - Fix tests
  • 6a138838 - Allow exact same schema to be re-imported
  • 814141db - Update ProtobufSchema.java
  • 885098ab - Update JsonSchema.java
  • ff5b2158 - Add support for CONDITION with CEL_FIELD
  • 22f36f3f - Optimization: cache Tink primitives
  • 8b8eebe0 - Fix checkstyle
  • 1063bb4c - Add check for FieldRuleExecutor
  • bc194452 - Add a CEL test with a tags expr in the guard
  • ff0b820b - Ensure specific Avro type is setup during configure
  • 6df56c8f - Optimization: remove known deps from ProtobufSchema
  • 16214b96 - Re-add schema parameter that was dropped during refactor
  • d6095ad3 - Fix checkstyle
  • 343ddff3 - First commit
  • PR-3012 - Minor: Apply service bot changes to earlier branches
  • 306dd5c5 - Add error message for PACKAGE_CHANGED
  • 2861cd88 - Fix checkstyle
  • abbab267 - Ensure package change for Protobuf is incompatible
  • 641a9bab - Fix checkstyle
  • d59f138d - Enhance healthy method
  • aed3b7d5 - Ensure correct GenericData is used
  • 26569386 - Optimization: remove extraneous KekInfo/DekInfo classes
  • be5c571b - Improve parameter handling in FieldEncryptionExecutor
  • 3bd1cdd5 - update common-compress
  • 77f4ed97 - Ensure Vault namespace is being passed
  • 98ffc914 - update azure-identity
  • 2ef767e3 - Fix merge issue for wire upgrade
  • 7684230c - Update pom.xml
  • 177af0d0 - Add Protobuf compat check for moving field into existing oneOf
  • PR-2969 - Add health check for SR and extensions
  • 82f51da1 - Support older versions of Tink library
  • bbe261f4 - exclude okio
  • 28d86634 - Don’t prefix KEK names with the context in the ContextFilter
  • 0cb664f1 - Fix sharedKeys cache to account for multiple use of kms key ID
  • PR-2943 - Update Bouncycastle dependencies
  • d08a0978 - fix NPE when value is nil when print.schema.ids=true is set
  • 7ec6fc8b - Fix breaking change ti ParsedSchema.validate method
  • 9408a386 - Improve err msg
  • df7effb2 - Handle alreadyExists in putKek
  • 87ca438c - Cache shared KEKs in DEK Registry
  • PR-2937 - Added num_keks_shared metric
  • 13848a40 - Improve the performance of invalid name scrubbing
  • 0bdb81e7 - Omit decrypted DEK when registering DEK in IMPORT mode
  • 0eb5a631 - Allow JSONata exprs to be used for CONDITION rules
  • 85a9e53c - Also support allOf to singleton; add tests
  • 26a3423e - Ensure logicalType flag passed to ReflectData
  • 414ce79f - Better handling of JSON Schema singleton combined types

Version 7.5.3

Released December 2023

Community Features

Common

  • e99d8b1a - exclude logback-core and logback-classic in line with the changes in kafka-rest
  • eb200670 - update zookeeper component to match version in ce-kafka
  • 0c034e86 - re-add dependencymanagement sections for old bouncycastle to satisfy downstream validation
  • 33734c5f - add additional version definition to ease the update of the downstream repos
  • 8e71caf2 - update bouncycastle

Kafka

  • PR-14293 - KAFKA-15372: Reconfigure dedicated MM2 connectors after leadership change (#14293)
  • PR-14235 - KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2 (#14235)
  • 1e35277f - CONFLUENT: Bump ZK to 3.8.3 for 7.5.x
  • PR-989 - KSECURITY-2090: Updated bcpkix version to 1.77
  • PR-14635 - KAFKA-15755: LeaveGroupResponse v0 - v2 loses its member under certain error conditions (#14635)
  • PR-14718 - KAFKA-15800: Prevent DataExceptions from corrupting KafkaOffsetBackingStore (#14718)
  • PR-14647 - KAFKA-15693: Immediately reassign lost connectors and tasks when scheduled rebalance delay is disabled (#14647)
  • 79de845b - fix build error
  • PR-14506 - KAFKA-15552 Fix Producer ID ZK migration (#14506)
  • PR-14674 - KAFKA-15771: fix concurrency bug in ProduceRequest#partitionSizes() (#14674)
  • PR-14216 - KAFKA-15319: Upgrade rocksdb (#14216)
  • ad925d25 - Revert “KAFKA-15093: Add 3.5 Streams upgrade system tests (#14602)”
  • PR-13315 - KAFKA-14767: Fix missing commitId build error after git gc (#13315)
  • PR-14602 - KAFKA-15093: Add 3.5 Streams upgrade system tests (#14602)
  • PR-14601 - KAFKA-15664: Add 3.4 Streams upgrade system tests (#14601)
  • PR-14539 - KAFKA-15378: fix streams upgrade system test (#14539)
  • PR-14587 - KAFKA-15607: Fix NPE in MirrorCheckpointTask::syncGroupOffset (#14587)
  • PR-13920 - KAFKA-15106: Fix AbstractStickyAssignor isBalanced predict (#13920)
  • PR-14317 - KAFKA-13973: Fix inflated block cache metrics (#14317)
  • PR-14190 - KAFKA-7438: Replace Easymock & Powermock with Mockito in RocksDBMetricsRecorderGaugesTest (#14190)
  • PR-14126 - Fix a Scala 2.12 compile issue (#14126)
  • PR-14458 - KAFKA-15498: upgrade to snappy 1.1.10.5 (#14458)
  • PR-14434 - KAFKA-15498: bump snappy-java version to 1.1.10.4 (#14434)
  • PR-14519 - KAFKA-15571: StateRestoreListener#onRestoreSuspended is never called because DelegatingStateRestoreListener doesn’t implement onRestoreSuspended (#14519)
  • PR-14500 - HOTIFX: fix Kafka versions for system tests (#14500)
  • PR-14438 - Upgrade Jetty to 9.4.52.v20230823 (#14438)
  • PR-14445 - KAFKA-15502: Update SslEngineValidator to handle large stores (#14445)
  • PR-14367 - KAFKA-15450 Don’t allow ZK migration with JBOD (#14367)
  • PR-14221 - KAFKA-15338: The metric group documentation for metrics added in KAFKA-13945 is incorrect (#14221)
  • PR-14326 - KAFKA-15429: reset transactionInFlight on StreamsProducer close (#14326)
  • PR-14325 - KAFKA-15429: catch+log errors from unsubscribe in streamthread shutdown (#14325)
  • c252e930 - KAFKA-15375: fix broken clean shutdown detection logic in LogManager
  • PR-14236 - KAFKA-15353: make sure AlterPartitionRequest.build() is idempotent (#14236)
  • PR-14039 - KAFKA-15211: Mock InvalidParameterException in DistributedConfigTest (#14039)
  • PR-14244 - KAFKA-15377: Don’t expose externalized secret values in tasks-config API endpoint (#14244)
  • PR-14278 - KAFKA-15393: Improve shutdown behavior in MM2 integration tests (#14278)
  • PR-14280 - KAFKA-15391: Handle concurrent dir rename which makes log-dir to be offline unexpectedly (#14280)
  • PR-14103 - HOTIFX: fix Kafka Streams upgrade path from 3.4 to 3.5 (#14103)
  • PR-14237 - KAFKA-15374 Handle case of default broker in config migration (#14237)
  • PR-14082 - KAFKA-15102: Add replication.policy.internal.topic.separator.enabled property to MirrorMaker 2 (KIP-949) (#14082)
  • PR-14187 - KAFKA-13197: fix GlobalKTable join/left-join semantics documentation. (#14187)
  • PR-14156 - KAFKA-15202: Fix MM2 offset translation when syncs are variably spaced (#14156)
  • PR-14141 - KAFKA-15100; KRaft data race with the expiration service (#14141)
  • PR-14125 - KAFKA-15271: Historicalterator can exposes elements that are too new (#14125)
  • PR-14162 - KAFKA-15312; Force channel before atomic file move (#14162)
  • PR-14115 - KAFKA-15263 Check KRaftMigrationDriver state in each event (#14115)
  • PR-14094 - KAFKA-15243: Set decoded user names to DescribeUserScramCredentialsResponse (#14094)
  • PR-14075 - KAFKA-15235: Fix broken coverage reports since migration to Gradle 8.x (#14075)
  • PR-14079 - KAFKA-15238: Move DLQ reporter setup from the DistributedHerder’s tick thread to the sink task thread (#14079)
  • PR-14044 - KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers (#14044)
  • PR-13948 - KAFKA-15091: Fix misleading Javadoc for SourceTask::commit (#13948)
  • PR-13284 - KAFKA-14718: Wait for MirrorMaker to start before executing test (#13284)
  • PR-13982 - KAFKA-15159: upgrade minor dependencies (#13982)
  • PR-13988 - [KAFKA-15137] Do not log entire request payload in KRaftControllerChannelManager (#13988)
  • PR-13955 - KAFKA-15145: Don’t re-process records filtered out by SMTs on Kafka client retriable exceptions in AbstractWorkerSourceTask (#13955)
  • PR-13950 - KAFKA-15140: Use TestUtils methods and add logs for assertion failure at TopicCommandIntegrationTest (#13950)
  • PR-13968 - KAFKA-15149: Fix handling of new partitions in dual-write mode (#13968)
  • PR-13904 - KAFKA-15114: Update StorageTool help for creating SCRAM credentials to specify name instead of user. (#13904)
  • PR-13831 - KAFKA-15053: Use case insensitive validator for security.protocol config (#13831)
  • PR-13895 - KAFKA-15098 Allow authorizers to be configured in ZK migration (#13895)
  • PR-13865 - KAFKA-15096: Update snappy-java to 1.1.10.1 (#13865)
  • PR-13843 - KAFKA-15080; Fetcher’s lag never set when partition is idle (#13843)
  • c97b88d5 - Bump version to 3.5.0

REST Proxy

  • PR-1230 - KREST-12372 Upgrade dependency to JDK 1.8 with version 1.77
  • PR-1221 - KREST-12630 Exclude logback from zookeeper dependency in tests
  • PR-1219 - KREST-12630 Add missing dependency due to zookeeper upgrade
  • PR-1212 - KREST-12277 semaphore migration to release branches
  • PR-440 - KREST-11812 upgrade jaxb-api dependency
  • ee8dd479 - KREST-12364: update bcpix
  • PR-427 - KRES-12288 migration semaphore release branches

Schema Registry

  • 01cc37a6 - Fix NPE for getAllVersionsById in MockSchemaRegistryClient
  • de9de139 - Handle all exceptions when rule action cannot be performed
  • 30fbe76c - Clean up test
  • e66c5c51 - Fix checkstyle
  • d0a9c3f9 - Fix copyright date
  • d6d92d65 - Add unit tests
  • 78b1947d - DGS-9450 Strip rule param prefix from configs
  • bfda9dd4 - Improve logging
  • 806989d7 - DGS-9450 More minor cleanup
  • ef077a85 - DGS-9450 Minor cleanup
  • 23b859ff - DGS-9451: Fix validateFields value in error message
  • 0b445986 - DGS-9450 Add command line tool to register/auto-rotate DEKs
  • 14936914 - Refactor getAead method so it can be overridden
  • 98a1a4de - update swagger core
  • 12166e07 - DGS-9402 In CEL, treat exception from guard as false
  • 3bc36b7a - DGS-9162: Address review feedback
  • PR-2888 - Apply service bot changes to version branches
  • PR-2887 - DGS-9059 Fix jmx prefix default
  • 1d415181 - DGS-9387 Ensure isKey flag is passed when reusing deserializers
  • ae313e51 - Add logging
  • 6683bd00 - Fix CSFLE dek rotation tests
  • c180f450 - Fix findbugs
  • 92a148cb - Temporarily disable DEK rotation tests
  • e3deb5dd - Fix CSFLE dek rotation test
  • PR-2879 - Update azure-identity and unpin azure-core
  • bafbad8c - DGS-9343 Allow removing enum in JSON Schema to be backward compatible
  • 897e6716 - further cleanup of method calls
  • f98d9a93 - change set.of to immutableset.of in AvroSchemaTest and ProtobufSchemaTest
  • 4d813665 - fix cannot find symbol error
  • eb68e9b2 - fix checkstyle
  • 0d57454c - Add unit tests and introduce validate fields config
  • PR-2874 - DGS-9048: Semaphore Migration of Version Branches
  • 089b1fb7 - Upgrade tink to 1.12.0
  • 899b38a2 - migrate to semaphore
  • 40c364a0 - address comments
  • 002f3b60 - address review feedback
  • 552ae00a - move reserved fields validation out of compatibility checks
  • 53982cac - DGS-9162: Metadata compatibility checks for forward and full
  • 7f7a7ab1 - add metrics-core to fix the build
  • 5aa50c15 - Fix test
  • 02a996b1 - Check multiple actions only valid for WRITEREAD and UPDOWN
  • 2f93432d - Add null check
  • 3f8d16f7 - Add initialized method to dek registry
  • cc24a287 - DGS-9106: Update log level for leader election
  • 5e7bf83d - DGS-9031 Add strings extension lib for CEL executor
  • b19aea83 - DGS-9067 Ensure keys for schema cache are not mutated
  • d3b1912d - upgrade com.squareup.okio:okio to 3.4.0
  • 3e5ecba2 - Add tests
  • a990ce0f - DGS-9031 Add builtin CEL validation functions
  • 828040f3 - DGS-7866: Set default for validateFields to false
  • 4aa97bb8 - DGS-7866: Use 35742 instead of $ as the prefix for reserved fields
  • c8ab3fd0 - Add log if encoding fails
  • 531cc692 - Add test
  • PR-2822 - Add deleted flag to create kek/dek requests
  • d81d5a53 - Encode subject names for Maven download plugin
  • PR-2820 - Cache ProtobufSchema.toDynamicSchema calls (#2818)
  • PR-2817 - Sort list of dek subjects before returning to client
  • PR-2816 - DGS-8960 Fix handling of diamond structure in Protobuf converter
  • PR-2815 - Update DEK Registry spec
  • 55c43766 - cherry-pick: DGS-7865 and DGS-7866

Version 7.5.2

Released October 2023

Community Features

Common

  • 35a1415d - APPSEC-3039: Update jetty and netty
  • 81295738 - update snappy to 1.10.5
  • aea31af8 - APPSEC-2796: add guava to dependencyManagement

Kafka

  • PR-968 - KSECURITY-2013: Upgraded Jetty to 9.4.53.v20231009 and Netty to 4.1.1
  • PR-955 - KAFKA-15498: bump snappy-java version to 1.1.10.5 (#14434)

REST Proxy

  • PR-408 - KREST-11276 - Tighten SSL auto cert reload
  • PR-409 - KREST-11276 - Fixing cert reload with multiple registered listeners

Schema Registry

  • PR-2806 - DGS-8908 Ensure tenant prefix is used for aliases
  • PR-2805 - DGS-8901 Ensure logical type config is applied to Reflect/Specific Avro data types
  • 61f94429 - DGS-5796: Optimize leader election
  • PR-2784 - DGS-8165 Add support for automatic DEK Rotation
  • PR-2790 - DGS-8088 Get the mode in scope rather that just for the subject
  • PR-2789 - DGS-8783 Add rulesToMerge/rulesToRemove to tags API
  • PR-2786 - DGS-8769 Upgrade vault lib; add namespace support
  • PR-2783 - DGS-8708 Add rule config to preserve source fields
  • PR-2773 - DGS-8615 Minor change to DlqAction
  • PR-2772 - DGS-8553: Updated create and delete DEK Registry API documented names
  • PR-2769 - DGS-8522 Support required fields in proto2 in CEL executor

Version 7.5.1

Released September 2023

Community Features

Common

  • 59440c88 - APPSEC-2855: add generation of SBOMs to maven based Java projects
  • PR-542 - update easymock support for java 17 support

Kafka

  • PR-939 - KSECURITY-1789: upgrading requests to v2.31.0 (3.5)

ksqlDB

  • PR-10065 - fix: import not picked back in fix

Schema Registry

  • PR-2748 - Ensure all SR client props are passed to FLE
  • PR-2746 - Create default KMS params in rule based on KMS key id
  • PR-2745 - Log DEK errors
  • PR-2744 - MINOR Added another constructor to DekRegistryConfig
  • PR-2742 - DGS-8169 Handle null compat groups for schemas
  • PR-2741 - Improve error handling during DEK creation
  • PR-2738 - Ensure kek name is prefixed with context
  • PR-2734 - DGS-8112 Allow properties to be omitted during serialization
  • PR-2730 - DGS-7826 DGS-7828 DGS-7830 Add kek/dek timestamp, dek version
  • PR-2764 - DGS-8336 Add ruleType filter to schemas endpoint
  • PR-2758 - DGS-8323 Always populate version metadata prop for tags API

Version 7.5.0

Released August 2023

Community Features

Common

  • b4334b94 - Pin jose4j version
  • 95ed5124 - Include snappy and jetty in the dependency management, cleanup jackson imports
  • 27a3c1ca - APPSEC-2796: update guava
  • PR-530 - Backport log redactor to CP patches.
  • 9cce6f1e - Add log redactor dependency
  • bd393b16 - AUDIT-1600: Updating logredactor version to 1.0.12

Kafka

  • PR-930 - KSECURIYT-1859: upgrading netty to 4.1.96.Final (3.5)
  • PR-13759 - KAFKA-15019: Improve handling of broker heartbeat timeouts (#13759)
  • PR-13735 - KAFKA-15003: Fix ZK sync logic for partition assignments (#13735)
  • PR-13758 - KAFKA-15010: ZK migration failover support (#13758)
  • PR-13757 - KAFKA-15017: Fix snapshot load in dual write mode for ClientQuotas and SCRAM (#13757)
  • PR-13786 - Upgrade snappy to v1.1.10.0 (#13786)
  • PR-13767 - KAFKA-15004: Fix configuration dual-write during migration (#13767)
  • PR-13781 - KAFKA-8713: JsonConverter replace.null.with.default should prevent emitting default for Struct fields (#13781)
  • PR-13742 - KAFKA-14996: Handle overly large user operations on the kcontroller (#13742)
  • PR-13741 - KAFKA-15009: Handle new ACLs in KRaft snapshot during migration (#13741)
  • PR-13732 - KAFKA-15007: Use the correct MetadataVersion in MetadataPropagator (#13732)
  • PR-13729 - KAFKA-14970: Fix SCRAM during migration dual-write (#13729)
  • PR-13745 - KAFKA-15015: Explicit load of reload4j (#13745)
  • PR-13723 - KAFKA-14980: Fix MirrorSourceConnector source consumer configuration (#13723)
  • PR-13717 - KAFKA-14994: Update jose4j (#13717)
  • PR-13690 - KAFKA-14978: ExactlyOnceWorkerSourceTask should remove parent metrics (#13690)
  • PR-13660 - KAFKA-14662: Update the ACL list in the doc (#13660)
  • PR-13688 - KAFKA-14974: Restore backward compatibility in KafkaBasedLog (#13688)
  • PR-13461 - KAFKA-14840: Support for snapshots during ZK migration (#13461)
  • PR-13628 - KAFKA-14859: SCRAM ZK to KRaft migration with dual write (#13628)
  • PR-13653 - KAFKA-14946: Fix NPE when merging the deltatable (#13653)
  • PR-13657 - KAFKA-14876: Document the new ‘PUT /connectors/{name}/stop’ REST API for Connect (#13657)
  • PR-13631 - KAFKA-14909: Check zkMigrationReady tag before migration (#13631)
  • PR-13407 - KAFKA-14805: KRaft controller supports pre-migration mode (#13407)
  • PR-13550 - KAFKA-14639: A single partition may be revoked and assign during a single round of rebalance (#13550)
  • PR-13606 - KAFKA-14918: Only send controller RPCs to migrating ZK brokers (#13606)
  • 33962e8b - KAFKA-14943: Fix ClientQuotaControlManager validation
  • PR-13429 - KAFKA-14666: Add MM2 in-memory offset translation index for offsets behind replication (#13429)
  • PR-13533 - KAFKA-12446: Update change encoding to use varint (#13533)
  • PR-13615 - KAFKA-14834: [12/N] Minor code cleanups relating to versioned stores (#13615)
  • PR-13592 - KAFKA-14862: Outer stream-stream join does not output all results with multiple input partitions (#13592)
  • PR-13587 - KAFKA-14876: Document the new ‘GET /connectors/{name}/offsets’ REST API for Connect (#13587)
  • PR-13575 - KAFKA-14905: Reduce flakiness in MM2 ForwardingAdmin test due to admin timeouts (#13575)
  • PR-13604 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13604)
  • 03b41b54 - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
  • 3f1e78ee - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
  • PR-13579 - KAFKA-14904: Pending state blocked verification of transactions (#13579)
  • PR-13614 - KAFKA-14586: Adding redirection for StreamsResetter (#13614)
  • PR-13609 - KAFKA-14834: [11/N] Update table joins to identify out-of-order records with isLatest (#13609)
  • PR-13513 - KAFKA-14881: Rework UserScramCredentialRecord (#13513)
  • PR-13526 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13526)
  • PR-13511 - KAFKA-14869: Ignore unknown record types for coordinators (KIP-915, Part-1) (#13511)
  • PR-13567 - Update zstd to 1.5.5 (#13567)
  • PR-12830 - KAFKA-8115: Reduce flakiness in Trogdor JsonRestServer shutdown (#12830)
  • PR-13534 - KAFKA-14054: Handle TimeoutException gracefully (#13534)
  • PR-13565 - KAFKA-14834: [9/N] Disable versioned-stores for unsupported operations (#13565)
  • PR-13477 - KAFKA-7499: Handle serialization error in ProductionExceptionHandler (#13477)
  • PR-13566 - KAFKA-14834: [10/N] Reserve repartition topic formats to include isLatest (#13566)
  • PR-13564 - KAFKA-14834: [8/N] Propagate isLatest as part of Change (#13564)
  • PR-13554 - KAFKA-14834: [7/N] Update VersionedKeyValueStore#put() to return validTo (#13554)
  • PR-13552 - KAFKA-14834: [6/N] Add tracking of versioned tables into graph nodes (#13552)
  • PR-13491 - KAFKA-14722: Make BooleanSerde public (an addition of upgrade.html changes) (#13491)
  • PR-13505 - KAFKA-14462: [5/N] Add EventAccumulator (#13505)
  • PR-13525 - KAFKA-14883: Expose observer state in KRaft metrics (#13525)
  • PR-13522 - KAFKA-14834: [5/N] Drop out-of-order records from FK join with versioned tables (#13522)
  • PR-13510 - KAFKA-14834: [4/N] Drop out-of-order records from table-table join with versioned tables (#13510)
  • PR-13391 - KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction (#13391)
  • PR-13509 - KAFKA-14834: [3/N] Timestamped lookups for stream-table joins (#13509)
  • PR-13541 - KAFKA-14894: MetadataLoader must call finishSnapshot after loading a snapshot (#13541)
  • PR-13462 - KAFKA-14857: Fix some MetadataLoader bugs (#13462)
  • PR-13520 - KAFKA-14462: [4/N] Add Group, Record and Result (#13520)
  • PR-13539 - KAFKA-14891: Fix rack-aware range assignor to assign co-partitioned subsets (#13539)
  • PR-13487 - KAFKA-9550: Copying log segments to tiered storage in RemoteLogManager (#13487)
  • PR-13497 - KAFKA-14834: [2/N] Test coverage for out-of-order data in joins (#13497)
  • PR-13496 - KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter (#13496)
  • PR-13424 - KAFKA-14783: (KIP-875): New STOPPED state for connectors (#13424)
  • PR-10747 - KAFKA-12446: Call subtractor before adder if key is the same (#10747)
  • PR-13373 - KAFKA-14420: Use incrementalAlterConfigs API for syncing topic configurations in MirrorMaker 2 (KIP-894) (#13373)
  • PR-13449 - KAFKA-14491: [22/N] Add test for manual upgrade to versioned store (#13449)
  • PR-13345 - KAFKA-13020: Implement reading Snapshot log append timestamp (#13345)
  • PR-13489 - KAFKA-14617: Fill broker epochs to the AlterPartitionRequest (#13489)
  • PR-13269 - KAFKA-12634: enforce checkpoint after restoration (#13269)
  • PR-12992 - KAFKA-14376: Add ConfigProvider to make use of environment variables KIP-887 (#12992)
  • PR-13499 - KAFKA-14880: TransactionMetadata with producer epoch -1 should be expirable (#13499)
  • PR-13369 - KAFKA-14172: Should clear cache when active recycled from standby (#13369)
  • PR-13300 - KAFKA-10199: Add task updater metrics, part 2 (#13300)
  • PR-13442 - KAFKA-14491: [20/N] Add public-facing methods for versioned stores (#13442)
  • PR-13456 - KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint (#13456)
  • PR-13318 - KAFKA-14533: Do not interrupt state-updater thread during shutdown (#13318)
  • PR-13374 - KAFKA-14765: and KAFKA-14776: Support for SCRAM at bootstrap with integration tests (#13374)
  • PR-13470 - KAFKA-14864: Close iterator in KStream windowed aggregation emit on window close (#13470)
  • PR-13431 - KAFKA-14491: [19/N] Combine versioned store RocksDB instances into one (#13431)
  • PR-13364 - KAFKA-14491: [16/N] Add recovery logic for store inconsistency due to failed write (#13364)
  • PR-13480 - Fix typos in security.html (#13480)
  • PR-13325 - KAFKA-14771: Include threads info in ConcurrentModificationException message (#13325)
  • PR-13350 - KAFKA-14452: Make sticky assignors rack-aware if client rack is configured (KIP-881) (#13350)
  • PR-13434 - KAFKA-14785: (KIP-875): Connect offset read REST API (#13434)
  • PR-13486 - Suppress exception in testExceptionInUpdateCoordinator (#13486)
  • PR-13458 - KAFKA-14838: Add flow/connector/task/role information to MM2 Kafka client.id configs (#13458)
  • PR-13474 - KAFKA-14867: Trigger rebalance when replica racks change if client.rack is configured (KIP-881) (#13474)
  • PR-13483 - Use readlock for reading epochs in LeaderEpochFIleCache (#13483)
  • PR-13408 - KAFKA-14617: Update AlterPartitionRequest and enable Kraft controller to reject stale request. (#13408)
  • PR-13482 - Switch to SplittableRandom in ProducerPerformance utility (#13482)
  • PR-13380 - KAFKA-14468: Committed API (#13380)
  • PR-13440 - KAFKA-14829: Consolidate reassignment logic into PartitionReassignmentReplicas (#13440)
  • PR-13472 - KAFKA-14774: the removed listeners should not be reconfigurable (#13472)
  • PR-13460 - KAFKA-14853: the serializer/deserialize which extends ClusterResourceListener is not added to Metadata (#13460)
  • PR-13445 - KAFKA-14843: Include Connect framework properties when retrieving connector config definitions (#13445)
  • PR-13148 - KAFKA-14645: Use plugin classloader when retrieving connector plugin config definitions (#13148)
  • PR-13446 - KAFKA-14837/14842: Avoid the rebalance caused by the addition and deletion of irrelevant groups for MirrorCheckPointConnector (#13446)
  • PR-13127 - KAFKA-14586: Moving StreamResetter to tools (#13127)
  • PR-13423 - KAFKA-14827: Support for StandardAuthorizer benchmark (#13423)
  • PR-13368 - KAFKA-14796: Migrate ACLs from AclAuthorizor to KRaft (#13368)
  • PR-13452 - KAFKA-14848: KafkaConsumer incorrectly passes locally-scoped serializers to FetchConfig (#13452)
  • PR-13326 - KAFKA-14774: the removed listeners should not be reconfigurable (#13326)
  • PR-13441 - Fix log DateTime format unit test (#13441)
  • PR-13393 - KAFKA-10244: An new java interface to replace ‘kafka.common.MessageReader’ (#13393)
  • PR-13425 - KAFKA-14365: Extract common logic from Fetcher (#13425)
  • PR-13438 - KAFKA-14835: Create ControllerMetadataMetricsPublisher (#13438)
  • PR-13382 - KAFKA-14722: Make BooleanSerde public (#13382)
  • PR-13385 - KAFKA-14800: Bump snappy-java version to 1.1.9.1 (#13385)
  • PR-13426 - KAFKA-14814: Skip Connect target state updates when the configs store has same state (#13426)
  • PR-13340 - KAFKA-14491: [15/N] Add integration tests for versioned stores (#13340)
  • PR-13409 - KAFKA-14491: [18/N] Update versioned store to check latest value on timestamped get (#13409)
  • PR-13292 - KAFKA-14491: [14/N] Set changelog topic configs for versioned stores (#13292)
  • PR-13420 - KAFKA-14740: Add source tag to MirrorSourceMetrics - KIP-911 (#13420)
  • PR-13419 - KAFKA-8713: Allow using null for field in JsonConverter (KIP-581) (#13419)
  • PR-13367 - KAFKA-14797: Emit offset sync when offset translation lag would exceed max.offset.lag (#13367)
  • PR-13404 - KAFKA-14812:ProducerPerformance still counting successful sending in console when sending failed (#13404)
  • PR-13398 - KAFKA-6891: send.buffer.bytes should be allowed to set -1 in KafkaConnect (#13398)
  • PR-13362 - KAFKA-14795: Provide message formatter for RemoteLogMetadata (#13362)
  • 361095a1 - KAFKA-14491: [17/N] Refactor segments cleanup logic
  • PR-13415 - KAFKA-14816: Only load SSL properties when issuing cross-worker requests to HTTPS URLs (#13415)
  • PR-13323 - KAFKA-14617: Add ReplicaState to FetchRequest (KIP-903) (#13323)
  • PR-13386 - KAFKA-14809: Fix logging conditional on WorkerSourceTask (#13386)
  • PR-13384 - KAFKA-14801: Encoded configs are not decoded before ZK migration (#13384)
  • PR-13396 - KAFKA-13884: Only voters flush on Fetch response (#13396)
  • PR-13378 - KAFKA-14793: Propagate Topic Ids to the Group Coordinator during Offsets Commit (#13378)
  • PR-13379 - KAFKA-14799: Ignore source task requests to abort empty transactions (#13379)
  • PR-13388 - KAFKA-14804: Update swagger plugin/dependencies to 2.2.8 (#13388)
  • PR-13349 - KAFKA-14452: Refactor AbstractStickyAssignor to prepare for rack-aware assignment (#13349)
  • PR-13258 - KAFKA-14447: remove stale TODO comment (#13258)
  • PR-13301 - KAFKA-14758: Extract inner classes from Fetcher for reuse in refactoring (#13301)
  • PR-13371 - Updating video links to ASF YouTube (#13371)
  • PR-12813 - KAFKA-14317: ProduceRequest timeouts are logged as network exceptions (#12813)
  • PR-13363 - KAFKA-14794: Decode base64 JSON string (#13363)
  • PR-13351 - KAFKA-14781: Downgrade MM2 log message severity when no ACL authorizer is configured on source broker (#13351)
  • PR-13067 - KAFKA-14524: Rewrite KafkaMetricsGroup in Java (#13067)
  • PR-13357 - KAFKA-14462: [3/N] Add onNewMetadataImage to GroupCoordinator interface (#13357)
  • PR-13354 - KAFKA-14753: Improve kafka producer example (#13354)
  • PR-13353 - KAFKA-14752: Improving the existing consumer examples (#13353)
  • PR-13303 - KAFKA-14761: Adding integration test for the prototype consumer (#13303)
  • PR-13359 - KAFKA-14792: Race condition in LazyIndex.get() (#13359)
  • PR-13116 - KAFKA-14351: Controller Mutation Quota for KRaft (#13116)
  • PR-13231 - KAFKA-14402: Update AddPartitionsToTxn protocol to batch and handle verifyOnly requests (#13231)
  • PR-13346 - KAFKA-14770: Allow dynamic keystore update for brokers if string representation of DN matches even if canonical DNs don’t match (#13346)
  • PR-13329 - KAFKA-14462: [2/N] Add ConsumerGroupHeartbeart to GroupCoordinator interface (#13329)
  • PR-13274 - KAFKA-14491: [13/N] Add versioned store builder and materializer (#13274)
  • PR-883 - Update tags to match standards
  • PR-13215 - KAFKA-14578: Move ConsumerPerformance to tools (#13215)
  • PR-13304 - KAFKA-14726: Move/rewrite of LogReadInfo, LogOffsetSnapshot, LogStartOffsetIncrementReason to storage module (#13304)
  • PR-13114 - KAFKA-14084: SCRAM support in KRaft. (#13114)
  • PR-13328 - KAFKA-14745: Cache the ReplicationPolicy instance in MirrorConnectorConfig (#13328)
  • PR-13264 - KAFKA-14491: [12/N] Relax requirement that KTable stores must be TimestampedKVStores (#13264)
  • PR-13182 - KAFKA-14649: Isolate failures during plugin path scanning to single plugin classes (#13182)
  • PR-13322 - KAFKA-14462: [1/N] Add new server configurations (KIP-848) (#13322)
  • PR-13095 - KAFKA-14580: Moving EndToEndLatency from core to tools module (#13095)
  • PR-13172 - KAFKA-14590: Move DelegationTokenCommand to tools (#13172)
  • PR-13270 - KAFKA-14729: The kafakConsumer pollForFetches(timer) method takes up a lot of cpu due to the abnormal exit of the heartbeat thread (#13270)
  • PR-13078 - KAFKA-13999: Add ProducerCount metrics (KIP-847) (#13078)
  • PR-12990 - KAFKA-14451: Rack-aware consumer partition assignment for RangeAssignor (KIP-881) (#12990)
  • PR-13102 - KAFKA-14371: Remove unused clusterId field from quorum-state file (#13102)
  • PR-13190 - KAFKA-12639: Exit upon expired timer to prevent tight looping (#13190)
  • PR-13184 - KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation (#13184)
  • PR-13276 - KAFKA-14732: Use an exponential backoff retry mechanism while reconfiguring connector tasks (#13276)
  • PR-13193 - KAFKA-14659: source-record-write-[rate|total] metrics should exclude filtered records (#13193)
  • PR-13291 - KAFKA-14742: Throttle connectors in ExactlyOnceSourceIntegrationTest to fix flakey OOMEs (#13291)
  • PR-13191 - KAFKA-14060: Replace EasyMock and PowerMock with Mockito in AbstractWorkerSourceTaskTest (#13191)
  • PR-13279 - KAFKA-14295: FetchMessageConversionsPerSec meter not recorded (#13279)
  • PR-13297 - KAFKA-14743: update request metrics after callback (#13297)
  • PR-13021 - KAFKA-14468: Implement CommitRequestManager to manage the commit and autocommit requests (#13021)
  • PR-13206 - KAFKA-14685: Refactor logic to handle OFFSET_MOVED_TO_TIERED_STORAGE error (#13206)
  • PR-13252 - KAFKA-14491: [11/N] Add metered wrapper for versioned stores (#13252)
  • PR-13228 - KAFKA-10199: Add task updater metrics, part 1 (#13228)
  • PR-13205 - KAFKA-14680: Upgrade gradle version from 7.6 to 8.0.1 (#13205)
  • PR-13295 - KAFKA-14744: NPE while converting OffsetFetch from version < 8 to version >= 8 (#13295)
  • PR-13263 - KAFKA-14728: Don’t run ‘spotlessScalaCheck’ in CI (#13263)
  • PR-13052 - KAFKA-14545: Make MirrorCheckpointTask.checkpoint handle null OffsetAndMetadata gracefully (#13052)
  • PR-13181 - KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method (#13181)
  • PR-11818 - KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (#11818)
  • PR-13161 - KAFKA-14128: Kafka Streams does not handle TimeoutException (#13161)
  • PR-13255 - KAFKA 14714: Move/Rewrite RollParams, LogAppendInfo, and LeaderHwChange to storage module. (#13255)
  • PR-13251 - KAFKA-14491: [10/N] Add changelogging wrapper for versioned stores (#13251)
  • PR-13025 - KAFKA-14299: Fix pause and resume with state updater (#13025)
  • PR-13192 - KAFKA-14675: Extract metadata-related tasks from Fetcher into MetadataFetcher 1/4 (#13192)
  • PR-13282 - KAKFA-14733: Added a few missing checks for Kraft Authorizer and updated AclAuthorizerTest to run tests for both zk and kraft (#13282)
  • PR-13273 - KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
  • PR-11976 - KAFKA-13771: Support to explicitly delete delegationTokens that have expired but have not been automatically cleaned up (#11976)
  • PR-13244 - [MINOR] assert the cache size for each operation in RemoteIndexCacheTest (#13244)
  • PR-13196 - KAFKA-14673: Add high watermark listener to Partition/Log layers (#13196)
  • PR-13250 - KAFKA-14491: [9/N] Add versioned bytes store and supplier (#13250)
  • PR-13248 - KAFKA-14717: KafkaStreams can’ get running if the rebalance happens be… (#13248)
  • PR-13234 - KAFKA-14706: Move/rewrite ShutdownableThread to server-common module. (#13234)
  • PR-13253 - KAFKA-14253: More informative logging (#13253)
  • PR-13262 - KAFKA-14727: Enable periodic offset commits for EOS source tasks (#13262)
  • PR-13168 - Kafka 14565: On failure, close AutoCloseable objects instantiated and configured by AbstractConfig (#13168)
  • PR-13238 - KAFKA-14708: Use Java thread instead of kafka library for example purpose (#13238)
  • PR-13208 - KAFKA-5756: Wait for concurrent source task offset flush to complete before starting next flush (#13208)
  • PR-13249 - KAFKA-14491: [8/N] Add serdes for ValueAndTimestamp with null value (#13249)
  • PR-13243 - KAFKA-14491: [7/N] Enforce strict grace period for versioned stores (#13243)
  • PR-13207 - KAFKA-14664: Fix inaccurate raft idle ratio metric (#13207)
  • b9754747 - KAFKA-14653: Use raw properties instead of post-resolution properties for MirrorMaker connectors(#13163)
  • PR-13227 - KAFKA-14693: Kafka node should halt instead of exit (#13227)
  • PR-13230 - KAFKA-14704: Follower should truncate before incrementing high watermark (#13230)
  • PR-13189 - KAFKA-14491: [6/N] Support restoring RocksDB versioned store from changelog (#13189)
  • PR-13241 - KAFKA-14711: kafaka-metadata-quorum.sh does not honor –command-confi… (#13241)
  • PR-13225 - KAFKA-6793: Unused configuration logging appears to be noisy and unnecessary (#13225)
  • PR-12366 - KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector (#12366)
  • PR-13211 - KAFKA-14676: Include all SASL configs in login cache key to ensure clients in a JVM can use different OAuth configs (#13211)
  • PR-13040 - KAFKA-14480: Move/Rewrite ProducerStateManager to storage module. (#13040)
  • PR-13188 - KAFKA-14491: [5/N] Basic operations for RocksDB versioned store (#13188)
  • PR-13202 - KAFKA-14513: Add broker side PartitionAssignor interface (#13202)
  • PR-13167 - KAFKA-14650: Synchronize access to tasks inside task manager (#13167)
  • PR-13183 - KAFKA-14668: Avoid unnecessary UMR during ZK migration (#13183)
  • PR-13137 - KAFKA-15086, KAFKA-9981: Intra-cluster communication for Mirror Maker 2 (#13137)
  • PR-12590 - KAFKA-7109: Close fetch sessions on close of consumer (#12590)
  • PR-12972 - KAFKA-14391: Add ConsumerGroupHeartbeat API (#12972)
  • PR-13203 - KAFKA-14048: Add new __consumer_offsets records from KIP-848 (#13203)
  • PR-12984 - KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic (#12984)
  • PR-13175 - KAFAK-14660: Fix divide-by-zero vulnerability (#13175)
  • PR-13213 - KAFKA-14688: Move package org.apache.kafka.server.log.internals to org.apache.kafka.storage.internals.log (#13213)
  • PR-13179 - KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener (#13179)
  • PR-13046 - KAFKA-14551: Move/Rewrite LeaderEpochFileCache and its dependencies to the storage module. (#13046)
  • PR-13200 - KAFKA-14678: Move __consumer_offsets records from core to group-coordinator (#13200)
  • PR-13107 - KAFKA-13972: Ensure replica state deleted after reassignment cancellation (#13107)
  • PR-13143 - KAFKA-14491: [3/N] Add logical key value segments (#13143)
  • PR-13186 - KAFKA-14491: [4/N] Improvements to segment value format for RocksDB versioned store (#13186)
  • PR-879 - Set Owner and role tags for Vagrant EC2 instances
  • PR-12777 - Replace EasyMock and PowerMock with Mockito - TimeOrderedWindowStoreTest (#12777)
  • PR-880 - KC-2332: Upgrade netty to 4.1.86
  • PR-13136 - KAFKA-14582: Move JmxTool to tools (#13136)
  • PR-13126 - KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store (#13126)
  • PR-13169 - KAFKA-14658: Do not open broker ports until we are ready to accept traffic (#13169)
  • PR-12879 - KAFKA-14409: Clean ProcessorParameters from casting (#12879)
  • 2e1947d2 - Bump version to 3.4.0
  • PR-13142 - KAFKA-14491: [2/N] Refactor RocksDB store open iterator management (#13142)
  • PR-13159 - KAFKA-14656: Send UMR first during ZK migration (#13159)
  • PR-13131 - KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions to tools (#13131)
  • PR-13166 - KAFKA-14623: OAuth’s HttpAccessTokenRetriever potentially leaks secrets in logging (#13119) (#13166)
  • PR-13140 - KAFKA-14644: Process should crash after failure in Raft IO thread (#13140)
  • PR-13151 - KRaft brokers and controllesr should fail on Authorizer failure (#13151)
  • 8c637b71 - Update muckrake version mapping
  • PR-13119 - KAFKA-14623: OAuth’s HttpAccessTokenRetriever potentially leaks secrets in logging (#13119)
  • PR-13156 - KAFKA-14533: re-enable ‘false’ and disable the ‘true’ parameter of SmokeTestDriverIntegrationTest (#13156)
  • PR-13144 - KAFKA-14463: Close ConnectorClientConfigOverridePolicy instances (#13144)
  • PR-13147 - KAFKA-14533: temporarily disable the ‘false’ parameter of SmokeTestDriverIntegrationTest (#13147)
  • PR-12802 - KAFKA-14311: Cleanly cleanly stop connectors/tasks on Connect worker shutdown (#12802)
  • PR-13080 - KAFKA-14575: Move ClusterTool to tools module (#13080)
  • PR-13139 - Update outdated documentation. (#13139)
  • PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
  • PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
  • PR-13112 - KAFKA-14367: Add internal APIs to the new GroupCoordinator interface (#13112)
  • PR-13129 - KAFKA-14638: Elaborate when transaction.timeout.ms resets (#13129)
  • PR-12821 - KAFKA-14132: Replace PowerMock and EasyMock with Mockito in streams tests (#12821)
  • PR-13130 - KAFKA-14637: Fix upgrade compatibility issue from older versions to 3.4 (#13130)
  • PR-12818 - Replace EasyMock with Mockito in streams tests (#12818)
  • PR-13117 - KAFKA-14621: Disallow authorizers during ZK migration (#13117)
  • PR-12902 - KAFKA-14367: Add OffsetDelete to the new GroupCoordinator interface (#12902)
  • PR-13115 - Update ProducerConfig.java (#13115)
  • PR-13108 - KAFKA-14618: Fix off by one error in snapshot id (#13108)
  • PR-13106 - KAFKA-13709: (follow-up): Avoid mention of ‘exactly-once delivery’ or ‘delivery guarantees’ in Connect (#13106)
  • PR-12901 - KAFKA-14367: Add TxnOffsetCommit to the new GroupCoordinator interface (#12901)
  • PR-13085 - KAFKA-14568: Move FetchDataInfo and related to storage module (#13085)
  • PR-13104 - KAFKA-14612: Make sure to write a new topics ConfigRecords to metadata log iff the topic is created (#13104)
  • 8478bbb5 - KAFKA-14601: Improve exception handling in KafkaEventQueue #13089
  • PR-12886 - KAFKA-14367: Add OffsetCommit to the new GroupCoordinator interface (#12886)
  • PR-13017 - KAFKA-14530: Check state updater more often (#13017)
  • PR-13103 - KAFKA-14304: Use boolean for ZK migrating brokers in RPC/record (#13103)
  • PR-12301 - KAFKA-14003: Kafka Streams JUnit4 to JUnit5 migration part 2 (#12301)
  • PR-13092 - KAFKA-14607: Move Scheduler/KafkaScheduler to server-common (#13092)
  • PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
  • PR-13058 - KAFKA-14557: Lock metadata log dir (#13058)
  • PR-12870 - KAFKA-14367: Add OffsetFetch to the new GroupCoordinator interface (#12870)
  • PR-13032 - KAFKA-14540: Fix DataOutputStreamWritable#writeByteBuffer (#13032)
  • PR-13087 - KAFKA-14600: Reduce flakiness in ProducerIdExpirationTest (#13087)
  • PR-13077 - KAFKA-14279: Add 3.3.x streams system tests (#13077)
  • PR-12998 - KAFKA-14493: Introduce Zk to KRaft migration state machine STUBs in KRaft controller. (#12998)
  • PR-13072 - KAFKA-14570: Fix parenthesis in verifyFullFetchResponsePartitions output (#13072)
  • PR-13043 - KAFKA-14558: Move/Rewrite LastRecord, TxnMetadata, BatchMetadata, ProducerStateEntry, and ProducerAppendInfo to the storage module (#13043)
  • PR-13075 - KAFKA-9087: Replace log high watermark by future log high watermark wh… (#13075)
  • 347042cc - KAFKA-14458: Introduce RPC support during ZK migration #13028
  • PR-13001 - KAFKA-14446: Code style improvements for broker-to-controller forwarding (#13001)
  • PR-13020 - KAFKA-14529: Use the MetadataVersion from ClusterConfig in ZK tests (#13020)
  • PR-13076 - KAFKA-14279: Add 3.3.x to core compatibility tests (#13076)
  • PR-13065 - KAFKA-14482: Move LoadedLogOffsets to storage module (#13065)
  • 0e51a202 - KAFKA-14458: Introduce RPC support during ZK migration #13028
  • PR-13073 - KAFKA-14571: Include rack info in ZkMetadataCache.getClusterMetadata (#13073)
  • PR-13049 - KAFKA-14478: Move LogConfig/CleanerConfig and related to storage module (#13049)
  • PR-13070 - KAFKA-14564: Upgrade netty to 4.1.86
  • PR-13053 - KAFKA-13881: Add Storage package info (#13053)
  • PR-13039 - KAFKA-14550: Move SnapshotFile and CorruptSnapshotException to storage module (#13039)
  • PR-13061 - [MINOR] Update upgrade documentation for 3.4 (#13061)
  • PR-13016 - KAFKA-14498: reduce the startup nodes to avoid timeout error (#13016)
  • PR-12008 - KAFKA-13439: move upgrade note to stream upgrade doc (#12008)
  • PR-12937 - KAFKA-13881: Add Connect package infos (#12937)
  • PR-12613 - MINOT: Fix punctuation marks (#12613)
  • PR-12166 - KAFKA-13817: Always sync nextTimeToEmit with wall clock (#12166)
  • PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
  • PR-13048 - KAFKA-14554: Move ClassLoaderAwareRemoteStorageManagerTest to storage module (#13048)
  • PR-13038 - KAFKA-14543: Move LogOffsetMetadata to storage module (#13038)
  • PR-12936 - KAFKA-13881: Add Streams package infos (#12936)
  • PR-12979 - KAFKA-14544: The “is-future” should be removed from metrics tags after future log becomes current log (#12979)
  • PR-13042 - KAFKA-14476: Move OffsetMap and related to storage module (#13042)
  • PR-13041 - KAFKA-14549: Move LogDirFailureChannel to storage module (#13041)
  • PR-13036 - KAFKA-14534: Reduce flakiness in TransactionsExpirationTest (#13036)
  • PR-13012 - KAFKA-14477: Move LogValidator and related to storage module (#13012)
  • PR-12947 - KAFKA-6586: Refactor ConnectDistributed and ConnectStandalone to re-use shared logic (#12947)
  • PR-12735 - KAFKA-14132: Replace EasyMock and PowerMock with Mockito in connect/runtime/ErrorHandlingTaskTest (#12735)
  • PR-13014 - KAFKA-14461: Move StoreQueryIntegrationTest to junit5 and fixing logic in a couple of tests for finding active streams (#13014)
  • PR-13023 - KAFKA-14532: Correctly handle failed fetch when partitions unassigned (#13023)
  • PR-13026 - KAFKA-14299: Avoid interrupted exceptions during clean shutdown (#13026)
  • PR-13010 - KAFKA-14475: Move TimeIndex/LazyIndex to storage module (#13010)
  • PR-13019 - KAFKA-14531: Fix controller snapshot interval (#13019)
  • PR-13011 - KAFKA-14521: Replace BrokerCompressionCodec with BrokerCompressionType (#13011)
  • PR-13009 - KAFKA-14474: Move OffsetIndex to storage module (#13009)
  • PR-12994 - KAFKA-14457: Controller metrics should only expose committed data (#12994)
  • PR-12801 - KAFKA-14343: Upgrade tests for state updater (#12801)
  • PR-13007 - KAFKA-14473: Move AbstractIndex to storage module (#13007)
  • PR-12996 - KAFKA-14472: Move TransactionIndex and related to storage module (#12996)
  • PR-12862 - KAFKA-14264: New logic to discover group coordinator (#12862)
  • PR-12968 - KAFKA-14417: Address incompatible error code returned by broker from InitProducerId (#12968)
  • PR-13013 - KAFKA-14466: Move ClassloaderAwareRemoteStorageManager to storage module (#13013)
  • PR-12993 - Move IndexEntry and related to storage module (#12993)
  • PR-11390 - KAFKA-13369: Follower fetch protocol changes for tiered storage. (#11390)
  • PR-12961 - KAFKA-14446: API forwarding support from zkBrokers to the Controller (#12961)
  • PR-12965 - KAFKA-14448: Let ZK brokers register with KRaft controller (#12965)
  • PR-12856 - KAFKA-14392: Fix overly long request timeouts in BrokerToControllerChannelManager (#12856)
  • PR-12946 - KAFKA-14427: ZK client support for migrations (#12946)
  • PR-13000 - KAFKA-14496: Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler (#13000)
  • PR-12977 - Removing Multicasting partitioner for IQ (#12977)
  • PR-12985 - KAFKA-13602: Remove unwanted logging in RecordCollectorImpl.java (#12985)
  • PR-12858 - KAFKA-14367: Add DeleteGroups to the new GroupCoordinator interface (#12858)
  • PR-12944 - KAFKA-14395: Add config to configure client supplier (#12944)
  • PR-12729 - KAFKA-14285: Delete quota node in zookeeper when configs are empty (#12729)
  • PR-12855 - KAFKA-14367: Add DescribeGroups to the new GroupCoordinator interface (#12855)
  • PR-12956 - KAFKA-14379: Consumer should refresh preferred read replica on update metadata (#12956)
  • PR-12971 - KAFKA-14454: Making unique StreamsConfig for tests (#12971)
  • PR-12962 - KAFKA-14318: KIP-878, Introduce partition autoscaling configs (#12962)
  • PR-12932 - KAFKA-14425: The Kafka protocol should support nullable structs (#12932)
  • c6590ee2 - KAFKA-14435: Fix allow.everyone.if.no.acl.found config behavior for StandardAuthorizer
  • PR-12935 - KAFKA-14432: RocksDBStore relies on finalizers to not leak memory (#12935)
  • PR-12955 - KAFKA-14443: Close topic creation Admin clients in MM2 connectors (#12955)
  • PR-12954 - KAFKA-14352: Rack-aware consumer partition assignment protocol changes (KIP-881) (#12954)
  • PR-12893 - KAFKA-14260: Add synchronized to prefixScan method (#12893)

ksqlDB

  • d3f0e149 - fix: fix compile error introduced by merge conflict resolution
  • PR-10000 - fix: implement default method for schema registry client
  • PR-9853 - feat: expose compaction and compression configs for RocksDB
  • PR-9916 - docs: use https for repository
  • PR-9911 - docs: upgrade pymdown-extensions and Pygments version
  • PR-9880 - feat: add MD5 hash UDF
  • e4d0bd7e - Upgrade netty to 4.1.87.Final
  • PR-9810 - Bump minimist and mkdirp
  • 4ce2d5f3 - fix: Replace regex in CommandParser with a split by space
  • da83d1c6 - fix: Replace regex in CommandParser with a split by space
  • PR-9876 - fix: check for tombstone before forwarding to sink
  • PR-9868 - KSE-911: allow proxy protocol listeners in KSQL
  • PR-9865 - bugfix: NPE for BigDecimal in CSV serializer
  • PR-9856 - feat: Add metrics for pull query thread pools
  • a5e1c5b5 - fix: Simplify regex in DdlDmlRequestValidators to avoid catastrophic backtracking
  • PR-9851 - fix: Avoid calling context.fail() twice
  • PR-9823 - fix: Set timeout for pull query forwarded requsts to 20s
  • PR-9818 - fix: default to get authHeader if using basic
  • PR-9812 - fix: catch TopicAuthorizationExceptions when getting topic config
  • PR-9770 - Upgrade netty to 4.1.86 #2
  • PR-9711 - fix: use args map instead of method args in generated java code
  • f1feb1ab - SSEO-3393 - Add metatag for Zendesk search crawler
  • PR-9772 - fix: complete transition to 7.5.x series

REST Proxy

  • 3274c17d - KREST-11122: - Add note about empty response from endpoint
  • a18ee85d - KREST-11122: - Add note about empty response from endpoint
  • PR-1171 - Remove the use of non-public API MockTime
  • PR-1170 - KREST-10646: Build break as a result of MockTime moving in Apache Kafka
  • PR-1169 - KREST-10639: Tidy up batch produce
  • PR-1166 - KREST-10244: Fix produce-request-rate metric
  • PR-1165 - Disable integration test temporarily
  • PR-1162 - KREST-10313: Prototype of batch REST Produce
  • PR-1163 - KREST-10286: Example python code for produce streaming V3 API
  • PR-1161 - Remove verbose log line
  • PR-1158 - KREST-8294: Permit charset for v3 produce
  • PR-1157 - KREST-4467: First part of reworking produce tests
  • PR-1156 - KREST-10318: Topic create without DESCRIBE_CONFIGS
  • PR-1154 - KREST-9968: Aggregated OpenAPI improvements
  • PR-1142 - KREST-9691: Add integration tests for the produce rate-limits for a single tenant.
  • PR-1153 - KREST-9559: Update lifecycle for Produce v3
  • PR-1146 - KREST-5471: Tidy up descriptions of API operations
  • PR-1144 - KREST-8221: Dynamic broker config improvements
  • PR-1140 - KREST-2594: Fix principal format in OpenAPI spec
  • PR-1139 - KREST-9401: Support STRING type in v3 produce API
  • PR-1138 - KREST-6569: - Cope with partitions disappearing
  • PR-1137 - KREST-1701: - Blank message for missing topic
  • PR-1136 - KREST-5711: - Non-negative partition id for produce
  • PR-1134 - KREST-8332: - Better error messages for query parameters
  • PR-1135 - KREST-2863: - Validate topic name in createTopic
  • PR-1132 - OpenAPI specification validation fixes
  • PR-1131 - Error response improvements
  • PR-1130 - Tidy up multi-record produce examples
  • 24efcbe1 - Integration test to send larger message size data in produce
  • PR-1126 - Required query parms and errors for ACL operations
  • PR-1129 - Update readme for v3 API
  • PR-1125 - KREST-7599: - add authorized_operations
  • PR-1124 - KREST-4757: Fix example for replicas_assignments
  • PR-1052 - Update README.md
  • PR-1119 - KREST-1797: - CreateTopic returns partitions and repl factor
  • PR-1114 - KREST-8943: Add an integration test with multiple consumers in parallel.
  • a54dde7e - CNC-366: Update Jetty
  • PR-395 - Don’t use DosFilter.Listener.onRequestOverLimit default to avoid verbosity
  • PR-393 - KREST 8913 Jetty 429 metrics
  • PR-384 - AUTHN-1586 Add optional latency SLO/SLA metrics in annotation
  • PR-380 - Insert RequestLogHandler to the ServletContextHandler instead of putting in Handler collection
  • PR-379 - Remove stacktrace in response when server throws BadMessageException
  • 1ec7602c - KREST 9773 Handle InvalidTopicException correctly
  • 8b251aa1 - Defer loading Conscrypt library
  • PR-376 - KREST-8331: - Remove class names from exceptions
  • PR-374 - Make SNI host check explicit
  • 54cde302 - Add BoringSSL TLS provider with Conscrypt
  • 5ae7042a - Add max buffer header size configs

Schema Registry

  • ca7a941c - Fix merge issue
  • PR-2719 - DGS-7826 DGS-7828 DGS-7830: Add DEK registry for CSFLE
  • 3f0e1198 - DGS-7927: fix for io.grpc_grpc-protobuf
  • a2933052 - throw IOException in BearerAuthCredentialProvider
  • c67de210 - Fix exception handling
  • 8cf3c48f - Update client/src/main/java/io/confluent/kafka/schemaregistry/client/SchemaRegistryClient.java
  • 38bbe23f - Close SerDes
  • 843787b4 - DSG-7861: Make BearerAuthCredentialProvider Closeable
  • 9ef14fcd - Fix exception handling
  • 049ba7bc - Update schema-serializer/src/main/java/io/confluent/kafka/formatter/SchemaMessageSerializer.java
  • a6672805 - Update schema-serializer/src/main/java/io/confluent/kafka/formatter/SchemaMessageDeserializer.java
  • dc8e8914 - Update client/src/main/java/io/confluent/kafka/schemaregistry/client/security/bearerauth/BearerAuthCredentialProvider.java
  • 0388c541 - Update client/src/main/java/io/confluent/kafka/schemaregistry/client/SchemaRegistryClient.java
  • PR-2717 - DGS-7976: Add tagging support for composite json schema
  • PR-2716 - Upgrade wire to 4.8.0
  • 0cf27170 - DGS-7797: Fix checkstyle errors
  • 47296719 - DGS-7797: Introduce a new optional host.port config for inter instance communication
  • PR-2699 - Minor: Fix entity order after converting list to map
  • PR-2696 - DGS-7772: Impl POST /subjects/{subject}/versions/{version}/tags
  • PR-2695 - do not reinstantiate objectmapper as it’s expensive (#2686)
  • PR-2693 - DGS-7746: Fix perf regression due to DGS-6331
  • 48101a91 - Fix merge issue
  • PR-2692 - DGS-7723: Fix for qualifySubjectWithParent method
  • PR-2690 - DGS-7697: Ensure version 0 is not returned in register response
  • PR-2685 - DGS-7565: Add rule name validation
  • 1e57cba0 - Fix merge issue
  • PR-2684 - DGS-7675: Fix NPE when print.schema.ids=true
  • PR-2683 - DGS-7682: Ensure correct tenant is passed when resolving refs
  • PR-2679 - DGS-7668: DGS-7669: Ensure invalid defaults are dropped; also handle slash in aliases
  • PR-2674 - DGS-7564: When registering, return schema if changed on server
  • PR-2673 - Exclude metadata population during import mode
  • PR-2670 - DGS-7488: Add X-Forward=true if forwarding request to leader
  • 4a833e6e - reorder
  • 1bf961b7 - Update json-schema
  • PR-2665 - DGS-7208: DGS-7474: Fix some regressions
  • PR-2664 - DGS-7208: DGS-7474: Add alias/normalize configs
  • PR-2658 - DGS-7457: Cache schemas in client when getting by subject/version
  • PR-2663 - Ensure compat check returns mutable lists
  • 963677cd - Fix conflicts and delete duplicate methods
  • 0f13c16e - Fix checkstyle
  • 20526ee5 - Solve conflicts
  • 119fcae0 - Fix duplicate variable httpConnectTimeoutMs definition
  • e0626f64 - Update the test to make it clear to customer
  • dc6f2096 - Update the doc
  • 33659feb - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
  • 3ffd8fc7 - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
  • 1ca2a39d - Update the codes with checking string for http connect timeout
  • ffba37fe - Change the config names
  • PR-2635 - DGS-7289: Adding CustomBearerAuthCredentialProvider
  • 7155f0fb - * Fix checkstyle error
  • 7e2dec9c - * Remove unused import
  • 9ab1b0ec - * Add defaults
  • 89aa28be - * Made SR client http connect and read timeout configurable.
  • PR-2655 - Enable use of service loader for CSFLE
  • PR-2653 - Make DEK algorithm configurable for CSFLE
  • PR-2644 - Add JSONata integration test
  • PR-2654 - DGS-7432: Fix ClassCastException when getting params option
  • PR-2652 - DGS-7430: Add auto.flush config for DlqAction
  • PR-2647 - DGS-7422: Optimization: omit compat check during IMPORT
  • PR-2646 - DGS-7412: Fix for registering references in non-default context
  • PR-2645 - DGS-7407: Fix issues for numeric types in CEL_FIELD exprs
  • 6e014ed0 - Fix merge issue
  • PR-2643 - DGS-7375: DGS-7376: DlqAction enhancements, support ServiceLoader
  • PR-2642 - DGS-7367: Propagate error messages even if not JSON
  • PR-2640 - DGS-7363: Handle JsonNode objects in CEL executors
  • PR-2614 - DGS-7103: Optimize instantiation of lists of schemas
  • PR-2629 - RCCA-12410: Fix exception due to addition to immutable list
  • PR-2639 - DGS-7353: Allow message references in CEL_FIELD exprs
  • 7572f96c - Made classes public
  • PR-2632 - DGS-7243: Add initialized method to determine extension readiness
  • PR-2630 - Add a local SR client for internal use to SR
  • dec6d308 - Made MutableHttpServletRequest case-insensitive
  • 01b5d127 - Use a handler instead of filter for adding request ID
  • 3e4e14f3 - Updated unittest
  • 535615db - Added X-Request-ID header to all request and response headers
  • PR-2625 - Static token provider: adding optional lsrc-id and pool-id
  • PR-2617 - Update RestClientException.java
  • PR-2556 - DGS-6065: Use HOST_NAME config for host even if inter.instance.listener.name is set
  • PR-2554 - Remove kafka-server-common dependency
  • e4f254af - Update README.md
  • PR-2545 - Fix due to KAFKA-14706