Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Version 7.8.0

Released December 2024

Community Features

Common

  • PR-702 - add okio-jvm to dependencyManagement to avoid regressions from ce-kafka
  • PR-684 - define version of commons-io and add it to dependencyManagement
  • PR-25 - Update zkclient to 0.9
  • PR-663 - disable CI gating workflow for master branch
  • 948d59f6 - Upgrade Netty
  • PR-660 - update protobuf to 3.25.5 (#16) (#654) (#17) (#657) (#18)
  • PR-657 - update protobuf to 3.25.5 (#16) (#654) (#17)
  • PR-654 - update protobuf to 3.25.5
  • PR-646 - update bouncycastle to version 2.0 (#11)
  • PR-651 - update bouncycastle to 2.0.0 7.8.x branch (#14)
  • PR-637 - adding cp-jar-build ci gating
  • PR-631 - Fix pint merge 7.7.x to 7.8.x
  • PR-629 - Bump protobuf version to 3.25.4
  • PR-2 - Remove IntelliJ and build output and add it to .gitignore.
  • ab0d4be5 - Update pom.xml
  • 2add2049 - pin classgraph
  • 1825f6c2 - update azure-identity to 1.12.2
  • 14541b03 - add cc-cluster-upgrader as downstream
  • 57dce6aa - Remove bouncy property
  • 3753b015 - Remove bcprov-jdk15on
  • a0c60665 - update zookeeeper to 3.8.4
  • 9ee11d1c - update
  • a19b74bd - Bump Zookeeper to 3.8.4
  • PR-588 - Add CodeArtifact package paths to service.yml
  • 8cd1f56b - revert bc.tls-fips version to 1.0.13
  • a7a6f359 - revert bc upgrade
  • 335ff618 - rename okio -> okio-jvm

Kafka

  • PR-1452 - Bump commonsIo to version 2.16.0
  • PR-17517 - upgrade Jetty (#17517)
  • 86380a21 - Align ducktape version in tests/docker/Dockerfile and tests/setup.py (#17482)”
  • PR-17501 - Document that control.plane.listener should be removed before ZK migration is finished (#17501)
  • 0bc10f33 - Ignore failing quota_test
  • PR-17508 - Pin all external GitHub Actions to the specific git hash (#16960) (#17461) (#17508)
  • PR-17482 - Align ducktape version in tests/docker/Dockerfile and tests/setup.py (#17482)
  • PR-17453 - Update protobuf and commons-io dependencies (#17453)
  • PR-17430 - Fix Throttler metrics name (#17430)
  • PR-17421 - release.py must not use home.apache.org (#17421)
  • PR-17431 - Removed timed waiting signal for client telemetry close (#17431)
  • PR-17359 - handle dangling “copy_segment_start” state when deleting… (#17359)
  • 596bb90c - Bump protobuf to 3.25.5
  • PR-1438 - disable flakey test
  • PR-17278 - Fix incorrect synonym handling for dynamic log configurations (#17258) (#17278)
  • PR-16337 - fix synchronization of streams threadState (#16337)
  • PR-17250 - Stablize reassign_partitions_test.py (#17250)
  • PR-17115 - Do not handle failed tasks as tasks to assign (#17115)
  • PR-17147 - KRaftMigrationDriver initialization race (#17147)
  • PR-17101 - Backport to 3.8 and Enable KRaft for BaseAdminIntegrationTest and SaslSslAdminIntegrationTest (#15175) (#17101)
  • PR-17119 - Don’t allow ZK migration to start without transactions (#17094) (#17119)
  • PR-16995 - local log retention ms/bytes “-2” is not treated correctly (#16995)
  • PR-16858 - Fix the behavior of delegation tokens that expire immediately upon creation in KRaft mode (#16858)
  • PR-16672 - AssignmentsManager gets stuck retrying on deleted topics (#16672)
  • PR-16811 - Refactor compression code to only load codecs when used (#16782) (#16811)
  • PR-16763 - Update zstd-jni lib (#16763)
  • PR-16678 - Fix MirrorMaker2 worker config does not pass config.provi… (#16678)
  • PR-16287 - Clean up and refactor release.py (#16287)
  • 771b9576 - Bump version to 3.8.0
  • PR-16641 - Use NoOpScheduler to rewrite LogManagerTest#testLogRecoveryMetrics (#16641)
  • PR-16614 - Fix deadlock caused by LogManagerTest#testLogRecoveryMetrics (#16614)
  • PR-16617 - Remove print MetaPropertiesEnsemble from kafka-storage tool (#16607) (3.8) (#16617)
  • PR-16608 - Use Utils.loadClass instead of Class.forName to resolve aliases correctly (#16608)
  • PR-16249 - Fixing error in completing future (#16249)
  • PR-16217 - Fix blocking DescribeCluster call in AdminClient DescribeTopics (#16217)
  • PR-16570 - Re-add task to state updater if transit to RUNNING fails (#16570)
  • PR-16565 - explicitly register Afterburner module in JsonSerializer and JsonDeserializer (#16565)
  • PR-16555 - Handle tasks in state updater before tasks in task registry (#16555)
  • PR-1393 - 3.7 fixed downstream trigger
  • PR-1394 - 3.8 fixed downstream trigger
  • PR-16545 - Close pending active tasks to init on partitions lost (#16545)
  • PR-16533 - Update LATEST_STABLE_METADATA_VERSION in system tests (#16533)
  • PR-16400 - mark MV 3.8-IV0 as the latest production (#16400)
  • PR-1367 - 3.7 removed mk-include and make init-ci from semaphore
  • PR-1368 - 3.8 removed mk-include and make init-ci from semaphore
  • PR-16478 - Revert group.version from 3.8 (#16478)
  • PR-16409 - Revert “Update kraft_upgrade_test.py to support KIP-848’s group protocol config (#16409)
  • PR-1316 - Update refs to confluent-packaging-tools 3.7
  • PR-16380 - use the default dsl store supplier for fkj subscriptions (#16380)
  • e2494e6f - Bump version to 3.7.1
  • PR-15473 - Fix the documentation and adjust the format (#15473)
  • PR-16379 - add 1 more node for test_exactly_once_source system test (#16379)
  • PR-16376 - fix consumer close to release assignment in background (#16376)
  • PR-16353 - Reapply dynamic remote configs after broker restart (#16353)
  • PR-16366 - Log error if config conficts with MV (#16366)
  • PR-16369 - [MINOR] Add a note for JBOD support for tiered storage (#16369)
  • PR-16338 - Add documentation for the native docker image (#16338)
  • PR-15986 - Revert new configurations from Dynamic KRaft network manager and channel (#15986)
  • c29260f3 - Implement AddVoter, RemoveVoter, UpdateVoter RPCs”
  • PR-16310 - AsyncKafkaConsumer removes offset fetch responses from cache too aggressively (#16310)
  • PR-16319 - Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port (#16319)
  • PR-16272 - New consumer unsubscribe close commit fixes (#16272)
  • PR-16321 - Reset tier lag metrics on becoming follower (#16321)
  • PR-16331 - (KIP-924 pt. 26) default standby task assignment nit (#16331)
  • PR-16322 - (KIP-924 pt. 25) Rename old internal StickyTaskAssignor to LegacyStickyTaskAssignor (#16322)
  • PR-16229 - FenceProducers API returns “unexpected error” when succes… (#16229)
  • PR-16318 - (KIP-924 pt. 24) internal TaskAssignor rename to LegacyTaskAssignor (#16318)
  • PR-16237 - Compute valid log-start-offset when deleting overlapping remote segments (#16237)
  • PR-16288 - Automatically wait for cluster startup in embedded Connect integration tests (#16288)
  • PR-16292 - (KIP-924 pt. 23) More TaskAssignmentUtils tests (#16292)
  • PR-13277 - KIP-899: Allow client to rebootstrap (#13277)
  • PR-16136 - Add IncludeTopicAuthorizedOperations option for DescribeTopicPartitionsRequest (#16136)
  • PR-16078 - Dynamic broker configs for remote fetch/copy quotas (#16078)
  • PR-16215 - [2/2] Coalesce records into bigger batches (#16215)
  • PR-16107 - Enable state updater by default (#16107)
  • PR-16294 - (KIP-924 pt. 22) Add RackAwareOptimizationParams and other minor TaskAssignmentUtils changes (#16294)
  • PR-15820 - Integrate RLMQuotaManager for throttling copies to remote storage (#15820)
  • PR-15993 - Fix potential leader-epoch checkpoint file corruption (#15993)
  • PR-16269 - (KIP-924 pt. 21) UUID to ProcessId migration (#16269)
  • PR-16283 - UniformHeterogeneousAssignmentBuilder throws NPE when one member has no subscriptions (#16283)
  • PR-16209 - Metric to measure the latency of remote read requests (#16209)
  • PR-16281 - KIP-1028: Modfiying download url for kafka dockerfile (#16281)
  • PR-16267 - KIP-1028: Adding 3.7.0 docker official images static assets (#16267)
  • PR-16203 - Support added to update remote.fetch.max.wait.ms dynamically (#16203)
  • PR-16256 - Renamed the enableRemoteStorageSystem to isRemoteStorageSystemEnabled (#16256)
  • PR-16199 - Refactor RemoteLogManagerConfig with AbstractConfig (#16199)
  • PR-14778 - Introduce remote.fetch.max.timeout.ms to configure DelayedRemoteFetch timeout (#14778)
  • PR-16180 - Migrate RemoteLogSegmentLifecycleTest to ClusterInstance infra (#16180)
  • PR-16173 - Update equals and hashcode methods for two attributes (#16173)
  • PR-16154 - Adding two thread pools (#16154)
  • PR-16153 - Cleanup check if tiered storage is enabled (#16153)
  • PR-16170 - Migrate TopicBasedRemoteLogMetadataManagerRestartTest to new test infra (#16170)
  • PR-16152 - Used the right constant in RemoteLogManagerTest#testFetchQuotaManagerConfig (#16152)
  • PR-16053 - Restart tasks on runtime-only connector config changes (#16053)
  • PR-16261 - [MINOR] KIP-1028: Modifying dockerfile comments (#16261)
  • PR-16226 - modifying Readme for Docker official images (#16226)
  • PR-16158 - Handle includeAuthorizedOperations in ConsumerGroupDescribe API (#16158)
  • PR-16245 - (KIP-924 pt. 20) Custom task assignment configuration fix (#16245)
  • PR-16201 - (KIP-924 pt. 18) Better assignment testing (#16201)
  • PR-16232 - Detect replica demotion in AssignmentsManager (#16232)
  • 7879f1c0 - Implement AddVoter, RemoveVoter, UpdateVoter RPCs
  • PR-15869 - Avoid unneeded subscription regex check if metadata version unchanged (#15869)
  • PR-15834 - Gate JBOD configuration on 3.7-IV2 (#15834)
  • PR-16031 - Enforce that RequestManager implementations respect user-provided timeout (#16031)
  • PR-16214 - Remove old assignment strategy usage in new consumer (#16214)
  • PR-16219 - (KIP-924 pt. 19) Update to new AssignmentConfigs (#16219)
  • PR-16222 - Consider produce error of different task (#16222)
  • PR-16198 - Move PartitionAssignor to new group-coordinator-api module (#16198)
  • PR-16079 - Fix high-watermark calculation to not assume the leader is in the voter set (#16079)
  • PR-16165 - Fix KRaft bug regarding config (#16165)
  • PR-16194 - (KIP-924 pt. 17) State store computation fixed (#16194)
  • PR-16147 - (KIP-924 pt. 16) TaskAssignor.onAssignmentComputed handling (#16147)
  • PR-15689 - Update FK-left-join documentation (#15689)
  • PR-15790 - Specify node and store where serdes are needed (#15790)
  • PR-16161 - Add exception regarding schema (#16161)
  • PR-16071 - Integrate RLMQuotaManager for throttling fetches from remote storage (#16071)
  • PR-16186 - Fix failed StorageToolTest.testFormatSucceedsIfAllDirectoriesAreAvailable and StorageToolTest.testFormatEmptyDirectory (#16186)
  • PR-16156 - The background thread should try to process the remaining task until the shutdown timer is expired. (#16156)
  • PR-16118 - Handle PartitionChangeRecord without directory IDs (#16118)
  • PR-16122 - KAFKA-16837, Ignore task configs for deleted connectors, and compare raw task configs before publishing them (#16122)
  • PR-16068 - Member Subscription Spec Interface (#16068)
  • PR-16151 - Use REQUEST_TIMEOUT_MS_CONFIG in AdminClient.fenceProducers (#16151)
  • PR-16144 - Re-add EventAccumulator.poll(long, TimeUnit) (#16144)
  • PR-15165 - Reset read offsets when seeking to beginning in TBRLMM (#15165)
  • PR-16163 - Don’t convert to group to classic if the size is larger than group max size. (#16163)
  • PR-16175 - InitialState type leaks into the Connect REST API OpenAPI spec (#16175)
  • PR-16042 - DescribeLogDirsResponseData#results#topics have unexpected topics having empty partitions (#16042)
  • PR-15945 - Fix broker re-registration issues around MV 3.7-IV2 (#15945)
  • 327b696b - Fix broker re-registration issues around MV 3.7-IV2
  • PR-16017 - Ensure HeartbeatRequestManager generates leave request regardless of in-flight heartbeats. (#16017)
  • PR-16088 - Optimize uniform (homogenous) assignor (#16088)
  • PR-16149 - [2/2] Introduce group.version feature flag (#16149)
  • PR-16120 - [1/2] Introduce group.version feature flag (#16120)
  • PR-16048 - Utils.getHost and Utils.getPort do not catch a lot of invalid host and ports. (#16048)
  • PR-16125 - offset fetch timeout exception in new consumer consistent with legacy (#16125)
  • PR-16142 - Revert “Add ProcessingExceptionHandler interface and implementations (#16090)” (#16142)
  • PR-16146 - RemoteLogManagerTest.testCopyQuotaManagerConfig failing (#16146)
  • PR-16129 - (KIP-924 pt. 15) Implement #defaultStandbyTaskAssignment and finish rack-aware standby optimization (#16129)
  • PR-16130 - [2/N]: Allow unstable feature versions and rename unstable metadata config (#16130)
  • PR-16062 - Fixing PartitionInfo and Cluster equals and hashCode (#16062)
  • PR-16135 - Move java versions inside java block to resolve deprecation (#16135)
  • PR-16046 - Integrate kafka native-image with system tests (#16046)
  • PR-15625 - Add Remote Log Manager quota manager (#15625)
  • PR-16141 - Revert “Add ProcessingExceptionHandler in Streams configuration (#16092)” (#16141)
  • PR-16101 - Add ByteBuffer support for Connect ByteArrayConverter (#16101)
  • PR-15779 - Mirgrate ResetConsumerGroupOffsetTest to new test infra (#15779)
  • PR-16092 - Add ProcessingExceptionHandler in Streams configuration (#16092)
  • PR-16005 - Update RemoteLogManager configuration in broker server (#16005)
  • PR-16075 - Improve producer ID expiration performance (#16075)
  • PR-15685 - [1/N]: Create FeatureVersion interface and add –feature flag and handling to StorageTool (#15685)
  • PR-16033 - (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks (#16033)
  • PR-16123 - (KIP-924 pt. 14) Callback to TaskAssignor::onAssignmentComputed (#16123)
  • PR-16010 - First log directory printed twice when formatting storage (#16010)
  • PR-15998 - Introduce ConsumerGroupPartitionAssignor interface (#15998)
  • PR-15946 - the flag “started” of RaftClusterInstance is false even though the cluster is started (#15946)
  • PR-16090 - Add ProcessingExceptionHandler interface and implementations (#16090)
  • PR-15601 - Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide (#15601)
  • PR-15690 - Add JBOD support in tiered storage (#15690)
  • PR-16057 - LeaveGroup API for upgrading ConsumerGroup (#16057)
  • PR-16114 - (KIP-924 pt. 13) AssignmentError calculation added (#16114)
  • PR-16074 - (KIP-924 pt. 12) Wiring in new assignment configs and logic (#16074)
  • PR-16052 - (KIP-924 pt. 11) Implemented StickyTaskAssignor (#16052)
  • PR-16041 - Add oldest-iterator-open-since-ms metric (#16041)
  • PR-15691 - Add KeyDeserializationException and ValueDeserializationException with record content (#15691)
  • PR-16064 - Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder (#16064)
  • PR-15947 - Make sure to update highestOffsetInRemoteStorage after log dir change (#15947)
  • PR-16040 - Move general configs out of KafkaConfig (#16040)
  • PR-16081 - Stop using a ClosureBackedAction to configure Spotbugs reports (#16081)
  • PR-15951 - abortAndPauseCleaning only when future log is not existed (#15951)
  • PR-16072 - fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE (#16072)
  • PR-15825 - Don’t throw OOORE when converting the offset to metadata (#15825)
  • PR-16076 - Use LongAdder instead of AtomicInteger (#16076)
  • PR-15974 - Reverse lookup map from topic partitions to members (#15974)
  • PR-16059 - CoordinatorRuntime should initialize MemoryRecordsBuilder with max batch size write limit (#16059)
  • PR-15910 - Restarts of MirrorCheckpointTask should not permanently i… (#15910)
  • 4f55786a - Fix the ZK Metadata cache confusion between brokers and controllers
  • 90892ae9 - Fix the controller node provider for broker to control channel
  • PR-16027 - KIP-1028: Adding code to support Apache Kafka Docker Official Images (#16027)
  • PR-16047 - Handle FencedInstanceId in HB response (#16047)
  • PR-15970 - Lazily convert subscribed topic names to topic ids (#15970)
  • PR-16045 - Integrate Native Docker Image with github actions (#16045)
  • PR-16044 - RackAwareTaskAssignorTest failed (#16044)
  • PR-16034 - (KIP-924 pt. 10) Topic partition rack annotation simplified (#16034)
  • PR-16020 - Fix broken compatibility in kafka.tools.NoOpMessageFormatter, kafka.tools.DefaultMessageFormatter, and kafka.tools.LoggingMessageFormatter (#16020)
  • PR-16016 - Replace archivesBaseName with archivesName (#16016)
  • PR-15697 - Handle directory failure timeout (#15697)
  • PR-16038 - Update netty/jetty/jackson/zstd dependencies (#16038)
  • PR-15988 - Heartbeat API for upgrading ConsumerGroup (#15988)
  • PR-16028 - Add iterator-duration metrics (#16028)
  • PR-16024 - (KIP-924 pt. 8) Added TopicPartitionAssignmentInfo (#16024)
  • PR-16004 - (KIP-924 pt. 7) Simplify requirements for rack aware graphs (#16004)
  • PR-15972 - (KIP-924 pt. 5) Add rack information to ApplicationState (#15972)
  • PR-15985 - Deprecate KafkaLog4jAppender (#15985)
  • PR-15640 - Enforce that event processing respects user-provided timeout (#15640)
  • PR-15975 - Add num-open-iterators metric (#15975)
  • PR-15927 - Native docker image for Apache Kafka (KIP-974) (#15927)
  • PR-15983 - Migrate RemoteLogMetadataManagerTest to new test infra (#15983)
  • PR-16002 - (KIP-924 pt. 6) Post process new assignment structure (#16002)
  • PR-15564 - Add unit tests of foreignKeyJoin classes (#15564)
  • PR-15516 - Support Compression Levels (KIP-390) (#15516)
  • PR-15992 - Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to use ClusterTestExtensions (#15992)
  • PR-15916 - Refactor kafka.test.annotation.Type and ClusterTestExtensions (#15916)
  • b5a013e4 - Add test for WriteTxnMarkers with AlterCluster permission
  • PR-15965 - Refactored and new rebalance callbacks integration tests (#15965)
  • PR-15839 - [4/N] ConfigCommandTest rewritten in java (#15839)
  • PR-15964 - [1/N] Coalesce records into bigger batches (#15964)
  • PR-15305 - Print Connect worker specific logs on poll timeout expiry (#15305)
  • PR-15997 - A bit cleanup of FeatureControlManager (#15997)
  • PR-15971 - InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka (#15971)
  • PR-15984 - Fix thread leak detection for event handler threads (#15984)
  • PR-15933 - Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (#15933)
  • PR-15980 - KRaft support in ListOffsetsRequestTest (#15980)
  • PR-15979 - DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE (#15979)
  • PR-15958 - Upgrade to scala 2.12.19 and scala 2.13.14 (#15958)
  • PR-15954 - SyncGroup API for upgrading ConsumerGroup (#15954)
  • PR-15981 - Remove TRACE level logging from AsyncKafkaConsumer hot path (#15981)
  • PR-1232 - Update aws-init.sh
  • PR-15956 - (KIP-924 pt. 4) Generify rack graph solving utilities (#15956)
  • PR-15859 - Quorum state data version 1 (#15859)
  • PR-15930 - [3/N] ConfigCommandTest rewritten in java (#15930)
  • PR-15861 - Add tags support in ClusterTestExtension (#15861)
  • PR-15953 - Fix IncrementalAlterConfigs during ZK migration (#15744) (#15953)
  • PR-15962 - Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest (#15962)
  • PR-15963 - Migrate tests in remaining packages in o.a.k.streams (#15963)
  • PR-15957 - Handle telemetry push response while terminating (#15957)
  • PR-1234 - Updated netty version to 4.1.108.Final
  • PR-1235 - Updated netty version to 4.1.108.Final
  • PR-15784 - New consumer fails with stop within allotted timeout in consumer_test.py system test (#15784)
  • PR-15408 - Implement example tests for common rebalance callback (#15408)
  • PR-15896 - Remove queue-based remove from state updater (#15896)
  • PR-15885 - Wait for given offset in TopicBasedRemoteLogMetadataManagerTest (#15885)
  • PR-15868 - enable test for ensureInternalEndpointIsSecured (#15868)
  • PR-15944 - (KIP-924 pt. 3) Implement KafkaStreamsAssignment (#15944)
  • PR-15920 - (KIP-924 pt. 2) Implement ApplicationState and KafkaStreamsState (#15920)
  • PR-13965 - Fix rack-aware assignment in AbstractStickyAssignor (#13965)
  • PR-15416 - AbstractStickyAssignor - Sort owned TopicPartitions by partition when reassigning (#15416)
  • PR-15919 - Have Streams treat InvalidPidMappingException like a ProducerFencedException (#15919)
  • PR-11023 - Update doSend doc about possible blocking (#11023)
  • PR-15922 - Fix producer leaks in KafkaLog4jAppenderTest (#15922)
  • PR-15903 - Remove Rack Awareness Code from the Server Side Assignors (#15903)
  • PR-15923 - Close serverChannel in SocketServer if unable to bind to a port (#15923)
  • PR-15909 - Improve expired poll logging (#15909)
  • PR-1229 - added specific paths to code artifact
  • PR-1228 - Make pipeline only trigger on branches in the kafkaMuckrakeVersionMap
  • PR-15727 - [3/4] Add integration test for consumerGroupDescribe API (#15727)
  • PR-15931 - Deprecate offsets.commit.required.acks (#15931)
  • PR-15894 - Shutdown with new remove operation in state updater (#15894)
  • PR-15873 - [2/N] ConfigCommandTest rewritten in java (#15873)
  • PR-15911 - Removed the in-memory implementation of RSM and RLMM (#15911)
  • PR-15785 - Add subscription model information to group state (#15785)
  • PR-15882 - Handle assignment with new remove operation in state updater (#15882)
  • PR-15902 - Cancel write timeout TimerTask on successful event completion (#15902)
  • PR-15897 - Replace ClusterType#ALL and ClusterType#DEFAULT by Array (#15897)
  • PR-15884 - merge unit test down to the class of integration test (#15884)
  • PR-15836 - Reduce contention for Fetch requests (#15836)
  • PR-15837 - Allowing WriteTxnMarkers API to run with AlterCluster permissions (#15837)
  • PR-14729 - Close SocketServer channels when calling shutdown before enableRequestProcessing (#14729)
  • PR-1223 - changed package paths for codeartifact
  • PR-1219 - added codeartifact to service.yml
  • PR-15886 - Use helper method to shutdown ExecutorService (#15886)
  • PR-15907 - reduce the check interval to speedup DelegationTokenRequestsTest (#15907)
  • PR-14716 - Move StreamTaskTest to Mockito (#14716)
  • PR-14360 - update/note deprecated configs (#14360)
  • PR-15573 - MissingSourceTopicException should include topic names (#15573)
  • PR-15715 - Support to define per broker/controller property by ClusterConfigProperty (#15715)
  • PR-15887 - (KIP-924) New interfaces and stubbed utility classes for pluggable TaskAssignors. (#15887)
  • PR-6934 - Add PATCH method for connector config (#6934)
  • PR-15149 - Fix spurious REST-related warnings on Connect startup (#15149)
  • PR-15880 - Add parent exception to RLMTask warning logs (#15880)
  • PR-15881 - Replace TestUtils#resource by scala.util.Using (#15881)
  • PR-15871 - Revoke tasks from state updater with new remove (#15871)
  • PR-15817 - Fix the leaking tiered segments during segment deletion (#15817)
  • PR-15879 - Remove variable “unimplementedquorum” (#15879)
  • PR-13801 - Write connector tombstone offsets to secondary store before primary store (#13801)
  • PR-15379 - Introduce FieldPath abstraction and nested path support for ExtractField SMT (#15379)
  • PR-14309 - KAFKA-13328, (2): Add custom preflight validation support for connector header, key, and value converters (#14309)
  • PR-15835 - Fix coordinator thread idle ratio (#15835)
  • PR-15870 - Remove lost tasks in state updater with new remove (#15870)
  • PR-15856 - Allow to initialize newly assigned partition’s positions without allowing fetching while callback runs (#15856)
  • PR-15798 - JoinGroup API for upgrading ConsumerGroup (#15798)
  • PR-15766 - Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions (#15766)
  • PR-15652 - kafka-dump-log –offsets-decoder should support new records (#15652)
  • PR-15744 - Fix IncrementalAlterConfigs during ZK migration (#15744)
  • PR-15803 - Honour interrupted thread state on KafkaConsumer.poll (#15803)
  • PR-15620 - Remove class-name dispatch in RemoteLogMetadataSerde (#15620)
  • PR-15831 - Don’t run scan job on forks (#15831)
  • PR-15571 - read/write sequence of buffers correctly (#15571)
  • PR-15850 - [3/N] ConfigCommandTest rewritten in java (#15850)
  • PR-15852 - Add remove operation with future to state updater (#15852)
  • PR-15841 - Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (2/3) (#15841)
  • PR-15853 - KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER (#15853)
  • PR-15671 - KRaft’s internal log listener to update voter set (#15671)
  • PR-15843 - KafkaConsumer#position() does not respect timeout when group protocol is CONSUMER (#15843)
  • 1fd39150 - Deflake ZKMigrationIntegrationTest.testDualWrite #15845
  • PR-15849 - Accept only one task per element in output queue for failed tasks (#15849)
  • PR-15745 - allow defining number of disks per broker in ClusterTest (#15745)
  • PR-15810 - Don’t generate useless PartitionChangeRecord on older MV (#15810)
  • PR-15832 - UserScramCredentialsCommandTest rewritten in Java (#15832)
  • PR-15833 - Remove setMetadataDirectory from BrokerNode/ControllerNode (#15833)
  • PR-15800 - Disallow @ClusterTemplate(“”) (#15800)
  • PR-15773 - broker shutdown hangs when log.segment.delete.delay.ms is zero (#15773)
  • PR-15735 - add more tests to TopicImageNodeTest (#15735)
  • PR-15723 - Consumer’s RequestState has incorrect logic to determine if inflight (#15723)
  • PR-15824 - Remove ClusterConfig parameter in BeforeEach and AfterEach (#15824)
  • PR-15828 - fix kafka.server.ControllerRegistrationManagerTest#testWrongIncarnationId (#15828)
  • PR-15645 - ConfigCommandIntegrationTest rewritten in java (#15645)
  • PR-15822 - Move metrics configs out of KafkaConfig (#15822)
  • PR-15778 - Fix consumer sys test revocation validation (#15778)
  • PR-15732 - retry pollEvent in KRaftMigrationDriver for retriable errors (#15732)
  • PR-15761 - Refactor/cleanup BrokerNode/ControllerNode/ClusterConfig (#15761)
  • PR-15796 - Enable DynamicBrokerReconfigurationTest.testDefaultTopicConfig (#15796)
  • PR-15775 - Move KRAFT configs out of KafkaConfig (#15775)
  • PR-15742 - Ensure rebalance listener exceptions are propagated to the user on consumer poll (#15742)
  • PR-15737 - IncrementalAssignmentConsumerEventHandler throws error when attempting to remove a partition that isn’t assigned (#15737)
  • PR-15698 - Client HB timing fix (#15698)
  • PR-15755 - cherrypick Convert NETWORK_EXCEPTIONs from KIP-890 transaction verification (#15755)
  • PR-15717 - JMH Benchmarks for Server Side Rebalances (#15717)
  • PR-15721 - Online downgrade triggering and group type conversion (#15721)
  • PR-15799 - Update parse_describe_topic to support new topic describe output (#15799)
  • PR-15795 - Replace “Map#entrySet#forEach” by “Map#forEach” (#15795)
  • PR-15739 - reassign_partitions_test.bounce_brokers should wait for messages to be sent to every partition (#15739)
  • PR-15794 - remove TestUtils#subscribeAndWaitForRecords (#15794)
  • PR-15792 - Stop the abort transaction try loop when closing producers (#15541) (#15792)
  • PR-15762 - Add a new constructor which invokes the existing constructor with default value for alternativeString (#15762)
  • PR-15361 - Do not flush global state store at each commit (#15361)
  • PR-15679 - migrate DeleteOffsetsConsumerGroupCommandIntegrationTest to use ClusterTestExtensions (#15679)
  • PR-15616 - remove truncated logs after alter dir (#15616)
  • PR-15774 - Move quota configs into server-common package (#15774)
  • PR-1195 - add service.yml to run service bot (moving from jenkinsfile to semaphore)
  • PR-15772 - Move socket configs into org.apache.kafka.network.SocketServerConfigs (#15772)
  • PR-1172 - update tags and added seamphore tags as part of semaphore migration 3.7
  • PR-15777 - Avoid resuming future replica if current replica is in the same directory (#15777)
  • PR-15746 - New consumer fails to consume records in security_test.py system test (#15746)
  • PR-15701 - Log exception message for non-fault errors in QuorumController (#15701)
  • PR-15776 - allow defining number of disks per broker in TestKitNodes (#15776)
  • PR-15765 - Avoid decompressing/collecting all records when all we want to do is to find a single matched record from remote storage (#15765)
  • PR-15767 - suppress the warnings from RemoteLogManager (#15767)
  • PR-15613 - AsyncConsumer should await pending async commits on commitSync and close (#15613)
  • PR-15770 - Refactor KafkaConfig to use PasswordEncoderConfigs (#15770)
  • PR-15763 - Clear all admins after close all them. (#15763)
  • PR-15719 - Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests (#15719)
  • PR-15136 - Avoid resuming future replica if current replica is in the same directory (#15136)
  • PR-15682 - Integrate KIP-1019 measurability changes (KIP-714) (#15682)
  • PR-15594 - Adding new coordinator related changes for connect_distributed.py (#15594)
  • PR-15753 - Revert Consumer Rolling Upgrade (#15753)
  • PR-15738 - Fix consumer static membership system test with new protocol (#15738)
  • PR-15681 - Expose method to determine metric measurability (KIP-1019) (#15681)
  • PR-15470 - DescribeTopicPartitions client side change. (#15470)
  • PR-15747 - Increment the local-log-start-offset before deleting segments in memory table (#15631) (#15747)
  • PR-15022 - Always create the /config/topics ZNode even for topics without configs (#15022)
  • PR-15631 - Increment the local-log-start-offset before deleting segments in memory table (#15631)
  • PR-15684 - Move consumer group and group coordinator configs out of core (#15684)
  • PR-15662 - Online upgrade triggering and group type conversion (#15662)
  • PR-15733 - Storage tool crashes if dir is unavailable (#15733)
  • PR-15730 - allow defining number of disks per broker in TestKitNodes (#15730)
  • PR-15720 - add test for TopicImageNodeTest (#15720)
  • PR-15724 - SubscriptionState should not prematurely reset ‘pending’ partitions (#15724)
  • PR-15558 - Improve MirrorMaker logging in case of authorization errors (#15558)
  • PR-15397 - Notify listener of latest leader and epoch (#15397)
  • PR-15716 - Upgrade gradle from 8.6 to 8.7 (#15716)
  • PR-15489 - Add KRaft support in GetOffsetShellTest (#15489)
  • PR-15695 - CurrentControllerId metric is unreliable in ZK mode (#15695)
  • PR-15648 - Delete metadata log on ZK broker startup (#15648)
  • PR-15670 - Move transactions configs out of core (#15670)
  • PR-15658 - Use correct cluster ID when formatting log dir. (#15658)
  • PR-15410 - Fix CoreUtils.swallow() test gaps (#15410)
  • PR-15676 - Eliminate the IDE warnings of accepting ClusterConfig in BeforeEach (LeaderElectionCommandTest and ProducerIdsIntegrationTest) (#15676)
  • PR-14995 - Allow restricting File and Directory ConfigProviders to specific paths (#14995)
  • PR-15411 - Add group protocol migration enabling config (#15411)
  • PR-15661 - ConsumerEventHandler does not support incremental assignment changes causing failure in system test (#15661)
  • PR-15656 - Move Sasl and SSL configs out of core (#15656)
  • PR-15557 - Race condition while promoting future replica (#15557)
  • PR-15688 - BeginningAndEndOffsets and OffsetsForTimes should send an event and return empty with zero timeout provided (#15688)
  • PR-15585 - Client support for java regex based subscription (#15585)
  • PR-15621 - ListOffsets doesn’t report the offset with maxTimestamp a… (#15621)
  • PR-15687 - Support to define server properties by ClusterTestDefaults (#15687)
  • PR-12174 - Fix hanging ServerShutdownTest.testCleanShutdownWithKRaftControllerUnavailable (#12174)
  • PR-15680 - Broker metrics to follow kebab/hyphen case (KIP-714) (#15680)
  • PR-15659 - Check partition exists before send reassignments to server in ReassignPartitionsCommand (#15659)
  • PR-15668 - Detect thread leaked client-metrics-reaper in tests (#15668)
  • PR-15525 - beginningOrEndOffsets does not need to build an OffsetAndTimestamps object upon completion (#15525)
  • PR-15409 - Test log directory failure in Kraft (#15409)
  • PR-15335 - Log directory failure re-creates partitions in another logdir automatically (#15335)
  • PR-15663 - Fix integration tests in Java with parameter name (#15663)
  • PR-15434 - Avoid optimisation in handshakeUnwrap (#15434)
  • PR-15655 - invoke SSLEngine::closeInbound on SslTransportLayer close (#15655)
  • PR-14706 - Clean shutdown detection - server side (#14706)
  • PR-15532 - Corrected manifest file for kafka-clients (#15532)
  • PR-15629 - Update replication_replica_failure_test.py to support KIP-848’s group protocol config (#15629)
  • PR-15628 - Update security_test.py to support KIP-848’s group protocol config (#15628)
  • PR-15627 - Update consumer_test.py’s static tests to support KIP-848’s group protocol config (#15627)
  • PR-15626 - Update kraft_upgrade_test.py to support KIP-848’s group protocol config (#15626)
  • PR-15624 - add FileLockTest (#15624)
  • PR-15646 - Useless import class org.apache.kafka.common.config.ConfigDef.Type (#15646)
  • PR-15510 - move outerJoinBreak-flags out of the loop (#15510)
  • PR-15446 - Implement GroupMetadataManager#onUnloaded (#15446)
  • PR-15521 - AssignmentsManager callback handling issues (#15521)
  • PR-15636 - Avoid empty remote metadata snapshot file in partition dir (#15636)
  • PR-15635 - Add test (#15635)
  • PR-14760 - disconnect from controller on AuthenticationException (#14760)
  • PR-15463 - fix testRemoteFetchExpiresPerSecMetric (#15463)
  • PR-15630 - Fix failed ReplicaManagerTest (#15630)
  • PR-15572 - [1/2] Move kafka.security package from core to server module (#15572)
  • PR-15541 - Stop the abort transaction try loop when closing producers (#15541)
  • PR-15520 - Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (1/3) (#15520)
  • PR-15423 - Add upgrade notes to 3.5.0, 3.5.2, and 3.7.0 about MM2 offset translation (#15423)
  • PR-15589 - Use ByteBufferOutputStream to avoid array copy (#15589)
  • PR-15484 - Prevent race conditions in Exit class from stopping test JVM (#15484)
  • fc646f92 - Revert “fix getOffsetByMaxTimestamp for compressed records (#15474)”
  • bd5989dd - Revert “fix the LogValidator for non-compressed type (#15476)”
  • PR-15414 - Reprocessing on user added global stores restore (#15414)
  • PR-15612 - Split consumer commit tests (#15612)
  • PR-15549 - kafka hangs forever in the starting process if the authorizer future is not returned (#15549)
  • PR-15608 - Fix bug where config change notification znode may not get created during migration (#15608)
  • PR-15584 - Correctly migrate default client quota entities (#15584)
  • PR-15075 - ZK configuration moved to ZkConfig (#15075)
  • PR-15576 - Update connect_distributed_test.py to support KIP-848’s group protocol config (#15576)
  • PR-15492 - Offline protocol migration integration tests (#15492)
  • PR-15568 - remove .lock file when FileLock#destroy (#15568)
  • PR-15588 - Enhance documentation for retention.ms and retention.bytes configurations (#15588)
  • PR-15389 - Simplify and deduplicate standalone herder test mocking (#15389)
  • PR-15596 - Use NetworkClientTest to replace RequestResponseTest to be the example of log4j output (#15596)
  • PR-15505 - Unify metadata.version format in log and error message (#15505)
  • PR-1102 - update tags and added seamphore tags as part of semaphore migration
  • PR-15592 - Fix handling of “–version” option in ConsumerGroupCommand (#15592)
  • PR-15586 - DeleteRecordsCommand should use standard exception handling (#15586)
  • PR-15559 - Convert NETWORK_EXCEPTIONs from KIP-890 transaction verification (#15559)
  • PR-15581 - Do not retry committing if topic or partition deleted (#15581)
  • PR-14903 - Serialize heartbeat requests (#14903)
  • PR-15535 - Splitting consumer integration test (#15535)
  • PR-15562 - Stop emitting warning log message when parsing source connector offsets with null partitions (#15562)
  • PR-15579 - Fix for rejoin while reconciling (#15579)
  • PR-15013 - Add nightly docker image scan job (#15013)
  • PR-15534 - High watermark updates should have a higher priority (#15534)
  • PR-15591 - kafka-leader-election / LeaderElectionCommand doesn’t set exit code on error (#15591)
  • PR-15583 - kafka-get-offsets / GetOffsetShell doesn’t handle –version or –help (#15583)
  • PR-15563 - add production-ready test of 3.3 - 3.6 release to MetadataVersionTest.testFromVersionString (#15563)
  • PR-15550 - use volatile to guarantee KafkaMetric#config visibility across threads (#15550)
  • PR-15486 - Introducing the AbortableTransactionException (#15486)
  • PR-15567 - Update transactions_test.py to support KIP-848’s group protocol config (#15567)
  • PR-15577 - Update replica_scale_test.py to support KIP-848’s group protocol config (#15577)
  • PR-15578 - Upgrade consumer_rolling_upgrade_test.py (#15578)
  • PR-15254 - Move stateDirectory mock in TaskManagerTest to Mockito (#15254)
  • PR-1013 - master update bcprov-jdk15on
  • PR-1064 - Fix for jetty
  • PR-15548 - Update consumer_bench_test.py to use consumer group protocol (#15548)
  • PR-14206 - Fix unnecessary topic config deletion during ZK migration (#14206)
  • PR-15483 - : add javafoc for kafka metric (#15483)
  • PR-15546 - Offline group protocol migration (#15546)
  • PR-14471 - ConsumerGroupCommand rewritten in java (#14471)
  • PR-15481 - desanitize entity name when migrate client quotas (#15481)
  • PR-15512 - replace assertTrue(obj instanceof X) with assertInstanceOf (#15512)
  • PR-15533 - Full ConsumerGroupHeartbeat response must be sent when full request is received (#15533)
  • PR-15476 - fix the LogValidator for non-compressed type (#15476)
  • PR-15524 - Txn may get get stuck in PrepareCommit or PrepareAbort state (#15524)
  • PR-15511 - KAFKA-16312, Local epochs in reconciliation (#15511)
  • PR-15530 - Broker may not shut down when SocketServer fails to bind as Address already in use (#15530)
  • PR-15464 - upgrade jline from 3.22.0 to 3.25.1 (#15464)
  • PR-15401 - Member should send full heartbeat when rejoining (#15401)
  • PR-15538 - Update snapshot_test.py to support KIP-848’s group protocol config (#15538)
  • PR-15540 - Update reassign_partitions_test.py to support KIP-848’s group protocol config (#15540)
  • PR-15539 - Update fetch_from_follower_test.py to support KIP-848’s group protocol config (#15539)
  • PR-15537 - Update consumer_group_command_test.py to support KIP-848’s group protocol config (#15537)
  • PR-15330 - Update consumer_test.py to support KIP-848’s group protocol config (#15330)
  • PR-15499 - Sort transformations by name in documentation; add missing transformations to documentation; add hyperlinks (#15499)
  • PR-15474 - fix getOffsetByMaxTimestamp for compressed records (#15474)
  • PR-15506 - Cleanup WorkerSinkTaskTest (#15506)
  • PR-15493 - Reduce synchronization between producer threads (#15323) (#15493)
  • PR-11982 - Adjustments for jacoco, coverage reporting (#11982)
  • PR-15364 - Improve reconciliation state machine (#15364)
  • PR-15491 - Avoid IllegalStateException during fetch initialization (#15491)
  • PR-15497 - Move consumer mock in TaskManagerTest to Mockito - part 3 (#15497)
  • PR-15201 - Checkpoint log-start-offset for remote log enabled topics (#15201)
  • 4cd36f93 - TRIVIAL: fix typo
  • 861fe68c - TRIVIAL: fix typo
  • PR-15465 - [4/4] Tests of ConsoleGroupCommand rewritten in java (#15465)
  • PR-12742 - Shared Readonly State Stores ( revisited ) (#12742)
  • PR-15466 - Add missing producer metrics to documentatio (#15466)
  • PR-15296 - Extra dot in error message in producer (#15296)
  • PR-15088 - Fix orphaned locks when rebalancing and store cleanup race on unassigned task directories (#15088)
  • PR-15479 - Divide DeleteTopics requests by leader node (#15479)
  • PR-15455 - Add timeout to all the CompletableApplicationEvents (#15455)
  • PR-15261 - Move consumer mock in TaskManagerTest to Mockito - part 2 (#15261)
  • PR-15457 - Cleanups in ConsoleConsumer (#15457)
  • PR-15363 - [2/4] Tests of ConsoleGroupCommand rewritten in java (#15363)
  • PR-15316 - Migrate WorkerSinkTaskTest to Mockito (3/3) (#15316)
  • PR-1064 - Fix for jetty
  • PR-15365 - [3/4] Tests of ConsoleGroupCommand rewritten in java (#15365)
  • PR-15480 - Upgrade zookeeper 3.8.3 -> 3.8.4 (#15480)
  • PR-14426 - flip joinSpuriousLookBackTimeMs and emit non-joined items (#14426)
  • PR-15444 - : fetchSnapshot might return null if topic is created before v2.8 (#15444)
  • PR-15387 - Log cleaner configuration move to CleanerConfig (#15387)
  • PR-15437 - FencedException in commitAsync not propagated without callback (#15437)
  • PR-15399 - KAFKA-16288, Fix Values convertToDecimal exception and parseString corruption (#15399)
  • PR-15395 - record discarded FK join subscription responses (#15395)
  • PR-15426 - Make group metadata available when a new assignment is set (#15426)
  • PR-14818 - KIP-768 - Extend support for opaque (i.e. non-JWT) tokens in SASL/OAUTHBEARER (#14818)
  • PR-15440 - updateSubscription fails if already empty subscription (#15440)
  • PR-15445 - Disable wakeups during autocommit on close (#15445)
  • PR-15438 - Clean up of consumer client internal events (#15438)
  • PR-15150 - Add Group Type Filter for List Group to the Admin Client (#15150)
  • PR-15213 - Broker returns offset for LATEST_TIERED_TIMESTAMP (#15213)
  • PR-15339 - Rebalance Metrics for AsyncKafkaConsumer (#15339)
  • PR-15433 - fix GroupCoordinatorService logger (#15433)
  • PR-15424 - migrate StreamedJoinTest to Mockito (#15424)
  • PR-15385 - Add test for concurrently updatingMetadata and fetching snapshot/cluster (#15385)
  • PR-15415 - callback to release assignment when stale member leaves group (#15415)
  • PR-15384 - KIP-994 (Part 1) Minor Enhancements to ListTransactionsRequest (#15384)
  • PR-15419 - Fix PlaintextConsumerTest.testStaticConsumerDetectsNewPartitionCreatedAfterRestart (#15419)
  • PR-15392 - Fix for not sending heartbeat while fenced (#15392)
  • PR-15422 - Remove check for log message that is no longer present (fix builds) (#15422)
  • PR-15369 - Do not return records from poll if group metadata unknown (#15369)
  • PR-15398 - Missing license for scala related dependencies (#15398)
  • PR-15357 - Commit retry logic fixes (#15357)
  • PR-15317 - Additional fix on the rollback migration documentation (#15317)
  • PR-15393 - Fix performance regression in RocksDB (#15393)
  • PR-15219 - IQv2 must return immutable position (#15219)
  • PR-15372 - Make sure that we do not exceed max poll interval inside poll (#15372)
  • PR-15391 - ducker-ak should fail fast when gradlew systemTestLibs fails (#15391)
  • PR-15383 - Fix PlaintextConsumerTest. testMaxPollIntervalMsDelayInRevocation (#15383)
  • PR-15359 - Enforce partition reassignment should complete when all target replicas are in ISR (#15359)
  • PR-14631 - add “inter.broker.listener.name” config in KRaft controller config (#14631)
  • PR-15328 - Update verifiable_consumer.py to support KIP-848’s group protocol config (#15328)
  • PR-15375 - Fix invalid transition on poll timer expiration (#15375)
  • PR-15358 - re-enable PlaintextConsumerTest.testAutoCommitOnCloseAfterWakeup (#15358)
  • PR-15323 - Reduce synchronization between producer threads (#15323)
  • PR-15334 - Re-enable testAutoCommitIntercept (#15334)
  • be6653c8 - [1/N]: Set metadata.log.dir to broker in KRAFT mode in integration test
  • PR-15274 - Move ConsoleConsumer to tools (#15274)
  • PR-1013 - master update bcprov-jdk15on
  • PR-15256 - [1/3] Tests of ConsoleGroupCommand rewritten in java (#15256)
  • PR-15313 - Migrate WorkerSinkTaskTest to Mockito (2/3) (#15313)
  • PR-15275 - Trigger client reconciliation based on manager poll (#15275)
  • PR-15352 - Clean up references to non-existent IntegrationTestHelper (#15352)
  • PR-15351 - Avoid the keyword var for a variable declaration (#15351)
  • PR-15311 - Fix member not rejoining after error (#15311)
  • PR-13909 - Update-Description-String (#13909)
  • PR-15324 - Fix slow expired producer id deletion (#15324)
  • PR-15347 - [MINOR] Fix docker image build by introducing bash (#15347)
  • PR-15346 - Fix ConnectRestApiTest system test (#15346)
  • PR-14645 - Allow git push to fail gracefully (#14645)
  • PR-15121 - Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders (#15121)
  • PR-15325 - Update VerifiableConsumer to support KIP-848’s group protocol config (#15325)
  • PR-15051 - Update Streams API broker compat table for AK 3.7 (#15051)
  • PR-15243 - Remove more leaky NamedTopologyIntegrationTest tests (#15243)
  • PR-14663 - Migrate WorkerSinkTaskTest to Mockito (1/3) (#14663)
  • PR-15225 - Added KRaft support in LeaderEpochIntegrationTest (#15225)
  • PR-15152 - Add group type filter to List Groups API (#15152)
  • PR-15263 - fix topic recreation handling with offline disks (#15263)
  • PR-15293 - Fix UMR and LAIR handling during ZK migration (#15293)
  • PR-15262 - ignore metadata.log.dir failure in ZK mode (#15262)
  • PR-14736 - KRaft support in DescribeUserScramCredentialsRequestNotAuthorizedTest (#14736)
  • PR-15307 - Increase timeout in KTableKTableForeignKeyInnerJoinCustomPartitionerIntegrationTest (#15307)
  • PR-15216 - Adding missing heartbeat metrics (#15216)
  • PR-9351 - Unnecessary widening of (int to long) scope in FloatSerializer (#9351)
  • b8ad6095 - Revert “Additional fixes on KRaft migration documentation (#15287)”
  • a1addb56 - Revert “Additional fixes on KRaft migration documentation (#15287)”
  • 945f4b91 - Reduce batch size for initial metadata load during ZK migration
  • 12ce9c7f - Reduce batch size for initial metadata load during ZK migration
  • PR-15180 - Begin enforcing ‘tasks.max’ property for connectors (#15180)
  • PR-15285 - [2/N]: Fix inverted args & more tests (#15285)
  • PR-15249 - Improve worker liveness check during Connect integration tests (#15249)
  • PR-15253 - Extend admin to support ConsumerGroupDescribe API (#15253)
  • PR-15237 - Implement GroupCoordinator.onPartitionsDeleted (#15237)
  • PR-15271 - Reconciliation may lose partitions when topic metadata is delayed (#15271)
  • PR-15167 - Move AuthorizerUtils and its dependencies to server module (#15167)
  • PR-14693 - KRaft support in LogRecoveryTest (#14693)
  • PR-15282 - Fix bug on auto-commit empty offsets blocking inflight (#15282)
  • PR-15176 - [6/N] Avoid recheduling callback in request thread (#15176)
  • PR-15287 - Additional fixes on KRaft migration documentation (#15287)
  • PR-15251 - Broker metrics for client telemetry (KIP-714) (#15251)
  • PR-15292 - Revert “MINOR: Fix “No suitable checks publisher found” message during CI build” (#15292)
  • PR-15246 - Move PasswordEncoder to server-common (#15246)
  • cf6defb8 - resend broker registration on metadata update to IBP 3.7-IV2
  • 4c6f975a - resend broker registration on metadata update to IBP 3.7-IV2
  • PR-15288 - Create partition dir for mockLog (#15288)
  • PR-15284 - Update jose4j to 0.9.4 (#15284)
  • 5a861075 - Fix stray replica of recreated topics in KRaft mode
  • f7feb43a - Fix stray replica of recreated topics in KRaft mode
  • PR-14846 - KRaft support in DeleteTopicTest (#14846)
  • PR-15193 - Fix KRaft migration documentation (#15193)
  • b7098722 - Fix ZK migration controller race #15238
  • 16ed7357 - Fix ZK migration controller race #15238
  • PR-15000 - Implement ConsumerInterceptors in the async consumer (#15000)
  • PR-15211 - Update list group state type filter to include the states for the new consumer group type (#15211)
  • PR-15133 - Add metric value consolidated for topics on a broker for tiered storage. (#15133)
  • PR-15151 - Add version 3.6 to Kafka Streams system tests (#15151)
  • PR-15266 - Handle retriable errors in TxnOffsetCommits (#15266)
  • PR-15248 - ConsumerGroupServiceTest rewritten in java (#15248)
  • PR-15087 - Refactor ReplicaManager code for transaction verification (#15087)
  • PR-15202 - Fixes for commit/fetch error handling (#15202)
  • PR-15267 - Handle invalid whole value casts in the Cast transform gracefully (#15267)
  • PR-15221 - [8/8] Update offset delete paths (#15221)
  • PR-15259 - Add groupType enum for KIP-848 (#15259)
  • PR-15147 - Enable testMetricsReporterUpdate (#15147)
  • PR-15162 - fix overlap of streams-config sections & describe additional parameters (#15162)
  • PR-14612 - Implement DescribeTopicPartitions RPC on broker (#14612)
  • PR-15186 - Fix “Unable to find FetchSessionHandler for node X” bug (#15186)
  • PR-15215 - Stop fetching while onPartitionsAssign completes (#15215)
  • PR-14797 - Delete unused classes (#14797)
  • PR-15234 - Evict client instances from cache (KIP-714) (#15234)
  • PR-15139 - [1/2]: Prevent transaction coordinator reloads on already loaded leaders (#15139)
  • PR-15183 - [7/N] Always materialize the most recent committed offset (#15183)
  • PR-15235 - skip checkQuorum for only 1 voter case (#15235)
  • PR-15233 - Generify RetryWithToleranceOperator, ErrorReporter, and WorkerTask (#15233)
  • PR-14856 - ConsumerGroupCommand options and case classes rewritten (#14856)
  • PR-15214 - Add missing RPC field descriptions (#15214)
  • PR-15158 - Move KafkaConfig.Defaults to server module (#15158)
  • PR-15212 - Partition is assigned to two members at the same time (#15212)
  • PR-15236 - Note that Java 11 support for broker and tools is deprecated for removal in 4.0 (#15236)
  • PR-14763 - Prevent KafkaBasedLog subclasses from leaking passed-in clients (#14763)
  • PR-15190 - Enhance request context with client socket port information (KIP-714) (#15190)
  • PR-15217 - Fix StreamsStandbyTask system test (#15217)
  • PR-15210 - Add committed and commit sensor to record metrics (#15210)
  • PR-15206 - Enable additional PlaintextConsumerTest tests for new consumer (#15206)
  • PR-15228 - MINOR - Correcting logs to debug telemetry reporter (#15228)
  • PR-15154 - Avoid data race which drops wrong record when errors.tolerance=all (#15154)
  • PR-13294 - Avoid NPE when RestClient calls expecting no-content receive content. (#13294)
  • PR-15159 - Move ClientQuotaManagerConfig outside of core (#15159)
  • 3c923302 - Be more consistent about getting the latest MetadataVersion
  • 7bf7fd99 - Be more consistent about getting the latest MetadataVersion
  • PR-15197 - Only update directoryIds if the metadata version supports DirectoryAssignment (#15197)
  • PR-15148 - Added support for compression of metrics (KIP-714) (#15148)
  • PR-15207 - Fix StreamsUpgradeTest (#15207)
  • PR-15204 - Disable state updater in trunk (#15204)
  • PR-15199 - Fix StreamsUpgradeTest (#15199)
  • b4036833 - Kcontroller dynamic configurations may fail to apply at startup
  • 0015d0f0 - Kcontroller dynamic configurations may fail to apply at startup
  • PR-15185 - Remove NamedTopologyIntegrationTest case which leaks clients (#15185)
  • PR-15191 - KRaft support in ReplicationQuotasTest (#15191)
  • PR-15182 - Coordinator unloading fails when replica is deleted (#15182)
  • PR-15194 - - Reconciliation auto-commit fix (#15194)
  • PR-14707 - KRaft support in KafkaMetricReporterExceptionHandlingTest (#14707)
  • PR-14669 - KRaft support in DeleteOffsetsConsumerGroupCommandIntegrationTest (#14669)
  • 4280df48 - Fix partition reassignment during ZK migration
  • d0f845a5 - Fix partition reassignment during ZK migration
  • PR-15060 - Handling remote segment read in case of log compaction (#15060)
  • PR-15080 - Fixed deadlock in StandaloneHerder (#15080)
  • PR-14754 - Fix leaked sockets in core tests (#14754)
  • PR-15155 - [5/N] Add UNSTABLE_OFFSET_COMMIT error support (#15155)
  • PR-15160 - Move ReplicationQuotaManagerConfig to server module (#15160)
  • PR-15171 - Adding KRaft support in ConsumerWithLegacyMessageFormatIntegrationTest (#15171)
  • PR-14917 - Increase timeout for Trogdor CoordinatorTest (#14917)
  • PR-15174 - Fix memory leak in RocksDBStore (#15174)
  • PR-15142 - [4/N] Wire transaction verification (#15142)
  • PR-15161 - Move OffsetConfig to group-coordinator module (#15161)
  • PR-15130 - Exclude throttle time when expiring inflight requests on a connection (#15130)
  • PR-15166 - Move ProcessRole to server module (#15166)
  • PR-15168 - Verify pending recycle action when standby is re-assigned (#15168)
  • PR-15028 - Add KRaft support in DynamicConnectionQuotaTest (#15028)
  • PR-15112 - Migrate consumer mock in TaskManagerTest to Mockito (#15112)
  • PR-15077 - close leaking threads in replica manager tests (#15077)
  • PR-15163 - Add suspended tasks back to the state updater when reassigned (#15163)
  • PR-15156 - KRaft support in SaslMultiMechanismConsumerTest (#15156)
  • PR-15153 - BrokerRegistrationRequest.logDirs field must be ignorable (#15153)
  • PR-14966 - Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite (#14966)
  • PR-14668 - KRaft support in DescribeConsumerGroupTest (#14668)
  • PR-15124 - KRaft support in DeleteTopicsRequestWithDeletionDisabledTest (#15124)
  • PR-15145 - Revert “Better Rocks column family management” (#15145)
  • PR-15146 - Disable state updater in 3.7 (#15146)
  • PR-15049 - Add KRaft support in OffsetsForLeaderEpochRequestTest (#15049)
  • PR-15048 - Add docker wrapper in core and remove docker utility script (#15048)
  • PR-14868 - Move ConfigEntityName to server-common (#14868)
  • PR-15132 - close more kafkaApis instances (#15132)
  • PR-15117 - Streams with state updater fails to close task upon fencing (#15117)
  • PR-15135 - Fix memory leak in RocksDBStore (#15135)
  • PR-15122 - fix threads leak threads in LocalLeaderEndPointTest and other tests (#15122)
  • PR-15125 - Migrate remaining mocks in StoreChangelogReaderTest to Mockito (#15125)
  • PR-15116 - Migrate storeMetadata mock in StoreChangelogReaderTest to Mockito (#15116)
  • PR-15105 - Decouple RocksDB access from CF (#15105)
  • PR-14599 - fix exception thrown in Admin#describeTopics for unknown ID (#14599)
  • PR-15086 - KRaft support in GroupCoordinatorIntegrationTest (#15086)
  • PR-14771 - Fix leaked sockets in trogdor tests (#14771)
  • PR-15106 - Migrate activeStateManager and standbyStateManager mocks in StoreChangelogReaderTest to Mockito (#15106)
  • PR-15073 - also fix stores for outer join (#15073)
  • PR-15104 - Disable shutdownhook in MiniKdc (used for testing) (#15104)
  • PR-15096 - Fix IllegalStateException in updateLags (#15096)
  • PR-15093 - Fix thread leak KafkaAPIsTest (#15093)
  • PR-14929 - Migrate stateManager mock in StoreChangelogReaderTest to Mockito (#14929)
  • PR-14764 - Fix leaked sockets in runtime tests (#14764)
  • PR-15094 - Save heap in AbstractCoordinatorConcurrencyTest by creating real ReplicaManager (#15094)
  • PR-15091 - Improve ControllerApiTest (#15091)
  • PR-15089 - Upgrade mockito to 5.8.0 (#15089)
  • PR-15090 - close DelayedFuturePurgatory in DelayedOperationTest (#15090)
  • PR-15030 - Fix custom timeouts overwritten by defaults in LeaderElectionCommand (#15030)
  • PR-15084 - close controllerApi instance to avoid thread leaks (#15084)
  • PR-15079 - Fix memory leaks due to KDC server in tests (#15079)
  • PR-15068 - Fix ConfigCommandTest.shouldNotSupportAlterClientMetricsWithZookeeper (#15068)
  • PR-14992 - Improve JsonConverter performance (#14992)
  • PR-15024 - Eagerly look up StringSerializer encoding during configure (#15024)
  • PR-15023 - Ensure new leader information merged correctly with the current metadata (#15023)
  • PR-14867 - Move ConfigType to server-common (#14867)
  • PR-15061 - fix stream-stream-join store types (#15061)
  • PR-15035 - Send Poll event to the background thread (#15035)
  • PR-15056 - add tests for remoteLogSizeComputationTime/remoteFetchExpiresPerSec metrics (#15056)
  • PR-15050 - Implement RemoteLogSizeBytes (#15050)
  • PR-15059 - Rename Generic to Classic (#15059)
  • PR-14985 - [3/N] Wire WriteTxnMarkers API (#14985)
  • PR-14557 - Commit/Fetch error handling improvements and V9 support (#14557)
  • PR-15043 - new group coordinator should check if partition goes offline during load (#15043)
  • PR-12228 - Fix resource leak in error scenarios (#12228)
  • PR-15053 - Add group.coordinator.rebalance.protocols and publish all new configs (#15053)
  • PR-15044 - Checkpoint restored offsets instead of written offsets (#15044)
  • PR-15015 - Throw an exception in DelayedRemoteFetch for follower fetch replicas. (#15015)
  • PR-15005 - Implement RemoteCopyLagSegments, RemoteDeleteLagBytes and RemoteDeleteLagSegments (#15005)
  • PR-14937 - Refactor closing consumer (#14937)
  • PR-14930 - Migrate async consumer tests to mocks (#14930)
  • PR-15032 - Add metric and request log attribute for deprecated request api versions (KIP-896) (#15032)
  • PR-15026 - Add RemoteLogMetadataCount metric (#15026)
  • PR-14848 - Move new group coordinator metrics from Yammer to Metrics (#14848)
  • PR-14977 - Modify LogDirFailureTest for KRaft (#14977)
  • PR-14853 - Generalise over RocksDB WriteBatch (#14853)
  • PR-15039 - Update upgrade guide for KIP-988 (#15039)
  • PR-13662 - Upgrade Jackson dependencies to version 2.16.0 (#13662)
  • PR-14375 - Add metrics for RemoteDelete and BuildRemoteLogAuxState (#14375)
  • PR-15042 - Replay BrokerRegistrationChangeRecord.logDirs (#15042)
  • PR-14998 - Replay BrokerRegistrationChangeRecord.logDirs (#14998)
  • PR-15021 - Add RemoteLogSizeComputationTime metric (#15021)
  • PR-15014 - Add metric for expiration rate of delayed remote fetch (#15014)
  • PR-14832 - Implement RemoteCopyLagBytes (#14832)
  • PR-15029 - Revert “Missing Tests for Transactions (#14702)” (#15029)
  • PR-15007 - Merge batch records during ZK migration (#15007)
  • PR-14702 - Missing Tests for Transactions (#14702)
  • PR-14925 - Re-enable consumer integration tests for new consumer (#14925)
  • PR-14962 - New consumer listTopics and partitionsFor (#14962)
  • PR-14640 - Implement event plumbing for ConsumerRebalanceListener callbacks (#14640)
  • PR-14996 - [MINOR]: Fix backward-compatibility bug (#14996)
  • PR-14984 - Bump MetadataVersion to support JBOD with KRaft (#14984)
  • PR-14774 - Ensure atomicity of in memory update and write when transactionally committing offsets (#14774)
  • PR-14981 - Implement write operation timeout (#14981)
  • PR-14988 - update Group size only when groups size changes (#14988)
  • PR-14293 - Reconfigure dedicated MM2 connectors after leadership change (#14293)
  • PR-14936 - Add support for clientInstanceIds in Kafka Stream (#14936)
  • PR-14649 - Update the local log start offset of a log after rebuilding the auxiliary state (#14649)

ksqlDB

  • PR-10477 - Update version
  • PR-10442 - Ensure Deterministic UDF Loading by Avoiding Context ClassLoader Interference
  • 70e7ffbf - Update netty version from 4.1.108.Final to 4.1.113.Final.
  • PR-140 - Delimited mode deserializer now can handle null values.
  • PR-139 - Ksql 230 sanity check for persistent queries
  • PR-138 - Added props to the sanity check.
  • f923a1ea - Fix git commit id plugin version for Java 8
  • ec48c835 - Upgrade git commit id plugin version from 2.2.6 to 9.0.1 and include git.commit.id property only.
  • PR-10413 - Fix for select struct->* for Pull Queries
  • 789394d9 - Fix CP Jar Build
  • 06ffd440 - Fix CP Jar Build
  • PR-10443 - updating cp-jar-build block in semaphore pipeline
  • c7adca7f - some changes
  • PR-120 - Now we can read avro content compatible with Schema Registry and also
  • 36247b8f - fix build
  • 7e97b9c2 - fix build
  • 4034cab4 - fix build
  • 4f37472e - fix build
  • 6f54ef1a - fix build
  • df548cd9 - fix build
  • c89dfafc - KSQL OAuth Client Side Integration
  • PR-80 - Fix ‘PRINT TOPIC’ statement’s tabular output
  • PR-63 - Checkstyle fixes
  • PR-62 - Remove unnecessary exclusions
  • PR-61 - Remove unused imports
  • PR-60 - Removed the imlicit topic generation from th estatement executor.
  • PR-59 - Added partition by clause to CSAS statement
  • PR-52 - Inherit from common POM, clean up dependencies
  • PR-50 - Ksql 47 json processing
  • PR-49 - Fixed the issue with COUNT(*).
  • PR-47 - Make tabular output for ‘DESCRIBE …’ statements a little prettier
  • PR-10403 - Add a null check in the response object to avoid NPE while accessing the connection status.
  • PR-37 - Remove custom JSON (de)serialization, use Jackson’s ObjectMapper instead
  • PR-10401 - SystemTime is refactored to a singleton class. Fix the ref in the class MetricCollectors, MetricAppender.
  • PR-10400 - SystemTime is refactored to a singleton class. Fix the ref in the class KafkaEmbedded.
  • PR-10398 - Package kafka streams in api client
  • PR-10397 - Increase time limit for the pipeline
  • PR-9 - Leverage ARG (Avro Random Generator) in DataGen class
  • PR-10381 - MINOR: Fix typo (#1)
  • PR-10380 - Use range version instead of pinned nano-versions
  • 3116084b - Use range version instead of pinned nano-versions
  • 88704d16 - Use range version instead of pinned nano-versions
  • ea2c84c9 - Use range version instead of pinned nano-versions
  • af2b0440 - Use range version instead of pinned nano-versions
  • e65fd80d - Use range version instead of pinned nano-versions
  • ddcda7d2 - Use range version instead of pinned nano-versions(to fix build failure)
  • PR-10369 - Add kafka streams dependency on ksqldb api client
  • PR-10371 - Fix ksql build failure
  • PR-10368 - MINOR: Fix compilation failures due to kafka changes
  • 21f1621d - Add property name in the ksql codebase
  • PR-10354 - Migrate configs to ksql repo
  • 78700556 - Fix build failure
  • PR-10343 - Fix compilation errors due to visibility change in Kafka properties.
  • PR-10326 - Move private kafka configs inside ksql
  • PR-10320 - Merge Conflict Resolution (from 7.7.x to master)
  • PR-10306 - Migrate master to ranges
  • PR-10286 - docs: update create-stream.md
  • PR-10282 - Fix schema registry version
  • ea42ac57 - add service.yml file
  • PR-10270 - Fix version pump that has gone wrong by bot

REST Proxy

  • PR-1290 - adding cp jar build workflow as PR ci gating
  • PR-1277 - Fix SystemTime and revert pin ak version
  • e0da9c6e - checkstyle again
  • 8ad64614 - fix checkstyle
  • 329a19d8 - Remove uses of KafkaConfig…Prop
  • ff025ec5 - Uncaught exception while closing input stream is causing clean up failures
  • PR-1262 - Fix for ZkConnectProp that is moved in https://github.com/apache/kafka/pull/15075
  • PR-1261 - Upscale Semaphore CI instance to avoid occasionally crash
  • PR-504 - Update bcfips modules (#17)
  • c843858b - Changing filter order cpsec 482
  • 753141ef - changing order for filter configuration
  • PR-500 - skipping cp jar builds on master branch PRs
  • PR-8 - Remove the single static ConfigDef from RestConfig.
  • PR-492 - adding cp-jar-build block for PR builds
  • PR-477 - Support HTTP/2 for BouncyCastle FIPS enabled (#472)

Schema Registry

  • PR-3345 - Revert change to default rule actions (#3334)
  • PR-3316 - Return encrypted value instead of throwing in exception
  • PR-3311 - Fix 7.1.x merge conflicts
  • PR-3264 - Add performanceMetric to list schema refs API
  • PR-3263 - Add performanceMetric to global delete config API
  • PR-3307 - Fix pint merge from 7.3.x to 7.4.x
  • PR-3244 - Handle 401/403 errors appropriately
  • e39cf350 - Fix typo in scope for JSON schema else clause
  • e5853d1b - Reenable Protobuf test after properly tracking scopes
  • PR-72 - Fixed the wait timeout in the KafkaStoreReaderThread. Fixes #54
  • PR-68 - Updated classpath logic in shell script.
  • PR-70 - Api update
  • PR-71 - Docs
  • 41e967fc - inor fix
  • 66a9487a - inor fix
  • PR-3262 - remove azure-identity definition, update azure-keyvault version (#49)
  • PR-3278 - Use concurrent hash maps in serde members (#61)
  • PR-3276 - Add format query param to APIs that return a schema (#46)
  • PR-3275 - Scrub refs in arrays when searching for inline tags (#59)
  • PR-3273 - Dedupe imported schemas with older versions (#58)
  • PR-3266 - Ensure register schema request is forwarded correctly (#53)
  • PR-3252 - Add CSFLE tests with pre-canned data (#42)
  • PR-3250 - Allow oneofs to be flattened in Protobuf converter (#41)
  • PR-3249 - Bound recursion when finding tagged entities (#40)
  • PR-63 - Integration tests
  • PR-3256 - adding cp jar build CI gating block (#38)
  • 593bad88 - Clean up pom
  • b478e3d1 - revert more AK 3.9 related changes
  • a86e9db8 - Revert breaking changes
  • PR-33 - Moved key serialization to a JSON serializer.
  • PR-30 - APIs changed to refer to subjects. Retrieval by globally unique schema id
  • PR-3235 - Fix Pint Merge Error
  • PR-25 - Issue 5
  • PR-11 - Moves hard-coded configs
  • 51f62d2f - Dedupe schemas during IMPORT mode
  • 3cdc0109 - fix license typo
  • 8e113e93 - Less ambiguous error message when unable to parse body from schema registry
  • dabb309c - checkstyle
  • bc09ecc1 - fix err
  • a4b184c7 - Upgrade tink to 1.13.0
  • 1808547b - Fix license in io.confluent.connect.schema.AbstractDataConfig.java under schema-converter
  • e9aad7d6 - Upgrade cel-java to 0.4.4
  • 01ee0565 - Fix incorrect license in client/ and schema-serializer/
  • 9db52d5a - fix compilation errors
  • d6812bb9 - fix kafka config
  • 01d024a5 - add back getMetricsTags() method
  • PR-3155 - Fix SystemTime that change visibility and compilation issue of KafkaGroupLeaderElector
  • 14f98ec9 - remove unused imports
  • 5677e636 - organize test
  • b5550ae6 - Fix checkstyle
  • 00b6bca7 - Fix removal of Utils.join
  • c24ba5a7 - test if missing schema issue can be reproduced
  • a55efeba - fix missed references
  • d6af16aa - fix
  • b43331ba - Update service.yml
  • fc51118e - missing whitespace
  • 1d75d999 - Added testing for nested schema in Json Union
  • c8c3fbca - Consider nested schemas in matching the correct Union schema
  • da9cfe26 - Unused
  • 46eba525 - Updating emitting one metric for keystore and one for truststore
  • aa7f2078 - Logic to emit metrics for certificate expiration times, to create alerts and dashboards for observability and monitoring
  • PR-3405 - Added kafka-tools dependency