Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Version 7.0.1

Released November 2021

Community Features

Common

  • PR-383 - Upgrade Netty to 4.1.68.Final

Kafka

  • PR-11439 - KAFKA-13406: Skip assignment validation for built-in cooperativeStickyAssignor (#11439)
  • PR-11323 - KAFKA-12226: Commit source task offsets without blocking on batch delivery (#11323)
  • PR-10563 - KAFKA-12487: Add support for cooperative consumer protocol with sink connectors (#10563)
  • PR-11429 - KAFKA-13396: Allow create topic without partition/replicaFactor (#11429)
  • PR-11463 - KAFKA-13430: Remove broker-wide quota properties from the documentation (#11463)
  • PR-8844 - KAFKA-9887: Fix failed task or connector count on startup failure (#8844)
  • PR-11426 - KAFKA-13391: Don’t fsync directory on Windows OS (#11426)
  • 5b7824d2 - Upgrade Netty version to 4.1.68.final
  • PR-10649 - KAFKA-12762: Use connection timeout when polling the network for new connections (#10649)
  • PR-11174 - KAFKA-9747: Creating connect reconfiguration URL safely (#11174)
  • PR-11337 - KAFKA-13309: Fix InMemorySessionStore#fetch/backwardFetch order issue (#11337)
  • PR-11347 - KAFKA-13296: Warn if previous assignment has duplicate partitions (#11347)
  • PR-11351 - KAFKA-13315: Log layer exception during shutdown that caused an unclean shutdown (#11351)
  • PR-11344 - KAFKA-13312; NetworkDegradeTest#test_rate should wait until iperf server is listening (#11344)
  • PR-11080 - KAFKA-13149; Fix NPE when handling malformed record data in produce requests (#11080)
  • PR-11283 - KAFKA-13249: Always update changelog offsets before writing the checkpoint file (#11283)
  • PR-11319 - KAFKA-13288; Include internal topics when searching hanging transactions (#11319)

ksqlDB

  • PR-8389 - Fix: Upgrade Netty version to 4.1.68.final

REST Proxy

  • PR-924 - KREST-1936: Remove mask filtering of producer configs.
  • PR-292 - Upgrade Netty Version

Schema Registry

  • PR-2090 - DGS-2415: Reject subjects with control chars
  • PR-2085 - Fix widening -> narrowing issue in casting
  • PR-2082 - DGS-2487: Adding missing cache to schema registry client
  • PR-2083 - DGS-2428: Minor rename to normalizeSchemas
  • PR-2079 - DGS-2507: Use fullName if enhanced.protobuf.schema.support=true
  • PR-2080 - DGS-2506: Add reference.lookup.only for Protobuf serializer
  • PR-2064 - DGS-2428: Add config to ignore ordering when registering/looking up schema
  • PR-2075 - DGS-2491: Ensure ProtobufSchema.name returns qualified name
  • PR-2070 - DGS-2401: Add converter flag to allow old doc/default behavior
  • PR-2039 - Fixes #1930: Support Jakarta EE 9 by eliminating dependencies on jersey ey-common and jakarta.ws.rs in schema-registry-client. (#1933)
  • PR-2036 - Fix default value for array of records
  • PR-2068 - Demonstrate wrong cache hit for union type reader schema
  • PR-2065 - DGS-2293 Add serializer config to specify schema ID (#2048)
  • PR-2061 - DGS-2417 Change moving Protobuf messages as compatible
  • PR-2058 - DGS-2393: Fix special chars in options for oneof/enum

Commercial Features

Confluent Server

  • KGLOBAL-863: Fix –pending-stopped-only to print topics only in the PENDING_STOPPED state and with partitions in the PENDING_STOPPED state (#4933)
  • KGLOBAL-890: Add dedicated admin client for dest reverse connections to avoid timeouts with unavailable links
  • KGLOBAL-891: Only display topic names in exception message while dele
  • KENGINE-49: Cache ReplicaManager.fetchPartitionPruningEnabled to avoid lock contention
  • KGLOBAL-923: Don’t add mirror partitions to transactions, fail as invalid request
  • KGLOBAL-913: Disable auto-topic creation in cluster link tests to workaround license timing issues
  • KGLOBAL-912: Filter out excluded groups when committing offsets for mirror promotion
  • KGLOBAL-712: Fixing bug that didn’t update task ms configs
  • KGLOBAL-845: Close cluster link network client if admin client creation fails due to invalid config
  • KGLOBAL-721: Fix output error for kafka-cluster-links –list
  • KGLOBAL-827: Fix MirrorCommand output for partitions without leader, source offset for stopped mirror
  • KGLOBAL-215: Source origination link system test
  • KGLOBAL-818: Handle NOT_CONTROLLER error for initiateReverseConnection
  • KGLOBAL-801: Don’t include topic filters from source links for validating auto-mirroring overlap
  • KGLOBAL-709: Receive buffer auto tuning for cluster linking connections
  • KGLOBAL-499: Add a link config to specify the topic configs to sync
  • SEC-2731: Improve Ldap handling for stale connections (#4497)

Control Center

  • (MINOR): Fix SR test due to SR method signature change
  • MMA-11217: Use test containers for integration test
  • (MINOR): MMA-11236: Clean and reorganize checkstyle suppressions

Metadata Service

  • SEC-2805: Manually tested fix for MDS forwarding logic to handle unicode and request with query params

Replicator

  • Upgrade jakarta.el:jakarta.el-api
  • REPL-1843: Track uncommited records given to Connect framework.

Confluent Platform Docker Images

  • Confluent Platform Docker images formerly shipped with world writable data and configuration directories (chmod a+w). Those permissions are now removed, and adjustments made to make those directories writable from the image’s appuser USER.
  • Confluent Platform Docker images are now built with yum --nodocs ... to prevent installing Red Hat Documentation. This slims down the images slightly. Confluent documentation from Confluent packages are still installed.

Version 7.0.0

Released October 2021

Community Features

Common

  • PR-381 - DGS-2161: Update netty version to 4.1.65.Final
  • PR-376 - Upgrade Jackson to 2.12.3.
  • PR-361 - Upgrade bcprov-jdk15on to 1.68
  • PR-365 - Upgrade log4j2 to 2.13.2
  • PR-371 - KAFKA-12756: Update ZooKeeper to v3.6.3
  • PR-309 - SEC-1789: Upgrade bouncycastle to 1.68

Kafka

  • d3cebd79 - DGS-2161: Update netty version to 4.1.65.Final
  • PR-11301 - KAFKA-13276: Prefer KafkaFuture in admin Result constructors (#11301)
  • PR-11294 - KAFKA-13266; InitialFetchState should be created after partition is removed from the fetchers (#11294)
  • PR-11300 - KAFKA-13258/13259/13260: Fix error response generation (#11300)
  • PR-11308 - KAFKA-13277; Fix size calculation for tagged string fields in message generator (#11308)
  • PR-11307 - KAFKA-13262: Remove final from MockConsumer.close() and delegate implementation (#11307)
  • PR-11295 - KAFKA-13270: Set JUTE_MAXBUFFER to 4 MB by default (#11295)
  • PR-11267 - KAFKA-13234; Transaction system test should clear URPs after broker restarts (#11267)
  • PR-11264 - KAFKA-13231; TransactionalMessageCopier.start_node should wait until the process if fully started (#11264)
  • PR-11164 - KAFKA-13155; Fix concurrent modification in consumer shutdown (#11164)
  • PR-11231 - KAFKA-13214; Consumer should not reset state after retriable error in rebalance (#11231)
  • PR-11239 - KAFKA-13219: BrokerState metric not working for KRaft clusters (#11239)
  • PR-11083 - KAFKA-13010: Retry getting tasks incase of rebalance for TaskMetadata tests (#11083)
  • PR-11184 - KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
  • PR-11216 - KAFKA-13198: Stop replicas when reassigned (#11216)
  • PR-11221 - KAFKA-13207: Skip truncation on fetch response with diverging epoch if partition removed from fetcher (#11221)
  • 17e2b249 - KAFKA-13209: Upgrade jetty-server
  • PR-11191 - KAFKA-13173; Ensure KRaft controller handles concurrent broker expirations correctly (#11191)
  • PR-11189 - KAFKA-13161; Update replica partition state and replica fetcher state on follower update (#11189)
  • PR-11194 - KAFKA-12779: rename namedTopology in TaskId to topologyName (#11194)
  • PR-11179 - KAFKA-13165: Validate KRaft node id, process role and quorum voters (#11179)
  • PR-11171 - KAFKA-13132; Upgrading to topic IDs in LISR requests has gaps introduced in 3.0 (part 2) (#11171)
  • PR-11177 - KAFKA-13167; KRaft broker should send heartbeat immediately after starting controlled shutdown (#11177)
  • PR-11178 - KAFKA-13168: KRaft observers should not have a replica id (#11178)
  • PR-11168 - KAFKA-13160: Fix bug in BrokerMetadataPublisher handling of default resources (#11168)
  • PR-10602 - KAFKA-12724: Add 2.8.0 to system tests and streams upgrade tests. (#10602)
  • PR-11166 - KAFKA-13159 Enable additional transaction system tests in KRaft (#11166)
  • PR-11165 - Enable transactions_test.py system test in kraft mode (#11165)
  • PR-10909 - KAFKA-12158; Better return type of RaftClient.scheduleAppend (#10909)
  • PR-11116 - KAFKA-13114; Revert state and reregister raft listener (#11116)
  • PR-11145 - KAFKA-13151; Disallow policy configs in KRaft since they are not yet supported (#11145)
  • PR-11126 - KAFKA-13132; Ensure topicId is updated on replicas even when the leader epoch is unchanged (#11126)
  • PR-11131 - KAFKA-13137; KRaft Controller Metric MBean names incorrectly quoted (#11131)
  • PR-10693 - KAFKA-12625: Fix the NOTICE file (#10693)
  • PR-11135 - KAFKA-13143; Remove Metadata handling from ControllerApis (#11135)
  • PR-10915 - KAFKA-13041: Enable connecting VS Code remote debugger (#10915)
  • PR-11136 - KAFKA-13141; Skip follower fetch offset update in leader if diverging epoch is present (#11136)
  • PR-11098 - KAFKA-13099; Transactional expiration should account for max batch size (#11098)
  • PR-11134 - KAFKA-12851: Fix Raft partition simulation (#11134)
  • PR-11132 - KAFKA-13139: Empty response after requesting to restart a connector without the tasks results in NPE (#11132)
  • PR-11002 - KAFKA-13026: Idempotent producer (KAFKA-10619) follow-up testings (#11002)
  • PR-11109 - KAFKA-13113; Support unregistering Raft listeners (#11109)
  • PR-11057 - KAFKA-13008: Try to refresh end offset when partitionLag returns empty (#11057)
  • PR-11053 - KAFKA-13015: Ducktape System Tests for Metadata Snapshots (#11053)
  • PR-11113 - KAFKA-13128: wait for all keys to be fully processed in #shouldQueryStoresAfterAddingAndRemovingStreamThread (#11113)
  • PR-11118 - KAFKA-13127; Fix stray topic partition deletion for kraft (#11118)
  • PR-11108 - KAFKA-13116: Fix message_format_change_test and compatibility_test_new_broker_test failures (#11108)
  • PR-11115 - KAFKA-13129: replace describe topic via zk with describe users (#11115)
  • PR-10921 - KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread (#10921)
  • b894e9ec - KAFKA-13104: Controller should notify raft client when it resigns #11082
  • PR-11084 - KAFKA-13100: Create KRaft controller snapshot during promotion (#11084)
  • PR-10811 - KAFKA-12598: ConfigCommand should only support communication via ZooKeeper for a reduced set of cases (#10811)
  • PR-11078 - KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724) (#11036) (#11078)
  • PR-11016 - KAFKA-13058; AlterConsumerGroupOffsetsHandler does not handle partition errors correctly. (#11016)
  • 637c0b0a - KAFKA-13090: Improve kraft snapshot integration test
  • d375c309 - KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • PR-11071 - KAFKA-13098: Fix NoSuchFileException during snapshot recovery (#11071)
  • PR-11026 - KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11026)
  • PR-11065 - Fix perf regression on LISR requests by asynchronously flushing the partition.metadata file (#11056) (#11065)
  • PR-10952 - KAFKA-12257; Consumer mishandles topics deleted and recreated with the same name (#10952)
  • PR-11059 - KAFKA-12930,KAFKA-12929: Deprecate Java 8 and Scala 2.12 (#11059)
  • PR-10753 - KAFKA-12803: Support reassigning partitions when in KRaft mode (#10753)
  • PR-10941 - KAFKA-10847: Remove internal config for enabling the fix (#10941)
  • PR-11035 - KAFKA-13072: Make RemoveMembersFromConsumerGroupHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11035)
  • PR-11021 - KAFKA-13062: Make DeleteConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11021)
  • PR-11022 - KAFKA-13063: Make DescribeConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11022)
  • PR-11019 - KAFKA-13059: Make DeleteConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11019)
  • PR-10877 - KAFKA-12925: adding presfixScan operation for missed implementations (#10877)
  • 2abe6851 - KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • PR-8259 - KAFKA-7421: Ensure that the Connect PluginClassLoader and DelegatingClassLoader are truly parallel capable and resolve deadlock occurrences (#8259)
  • PR-11007 - KAFKA-10589; Rename kafka-replica-verification CLI command line arguments for KIP-629 (#11007)
  • PR-11008 - KAFKA-10588; Rename kafka-console-consumer CLI command line arguments for KIP-629 (#11008)
  • 0707a8d3 - KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • PR-10985 - KAFKA-12984: make AbstractStickyAssignor resilient to invalid input, utilize generation in cooperative, and fix assignment bug (#10985)
  • PR-11032 - KAFKA-13073: Fix MockLog snapshot implementation (#11032)
  • PR-11031 - KAFKA-13067 Add internal config to lower the metadata log segment size (#11031)
  • PR-11040 - KAFKA-13078: Fix a bug where we were closing the RawSnapshotWriter to early (#11040)
  • PR-11041 - KAFKA-13080: Direct fetch snapshot request to kraft controller (#11041)
  • PR-10986 - KAFKA-12983: reset needsJoinPrepare flag before rejoining the group (#10986)
  • 72159ea8 - Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
  • PR-11011 - KAFKA-13051; Require principal builders implement KafkaPrincipalSerde and set default (#11011)
  • PR-11003 - KAFKA-12360: Document new time semantics (#11003)
  • PR-10935 - KAFKA-13003: In kraft mode also advertise configured advertised port instead of socket port (#10935)
  • PR-10794 - KAFKA-12677: Return not_controller error in envelope response itself in KRaft mode (#10794)
  • PR-11013 - KAFKA-13056; Do not rely on broker for snapshots if controller is co-resident (#11013)
  • PR-11012 - KAFKA-13057; Add KRaft “broker” to several RPC’s listeners (#11012)
  • PR-11000 - KAFKA-13045: Adding a test for batched offsetFetch requests with one group repeating (#11000)
  • PR-9541 - KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message (#9541)
  • PR-11010 - KAFKA-13053; Bump kraft frame version for incompatible changes from 2.8 (#11010)
  • PR-10560 - KAFKA-12660; Do not update offset commit sensor after append failure (#10560)
  • PR-10903 - KAFKA-13023: make “range, cooperative-sticky” as the default assignor in V3.0 (#10903)
  • PR-8571 - KAFKA-7613: Enable -Xlint:rawtypes for connect, fixing warnings (#8571)
  • PR-10962 - KAFKA-12234: Implement request/response for offsetFetch batching (KIP-709) (#10962)
  • PR-10931 - KAFKA-12998: Implement broker-side KRaft snapshots (#10931)
  • 258ed5a7 - KAFKA-12155: Metadata log and snapshot cleaning #10864
  • PR-10975 - KAFKA-13035 updated documentation for connector restart REST API to … (#10975)
  • PR-10974 - KAFKA-12979; Implement command to find hanging transactions (#10974)
  • PR-10960 - KAFKA-12981; Ensure LogSegment.maxTimestampSoFar and LogSegment.offsetOfMaxTimestampSoFar are read/updated in sync (#10960)
  • PR-10971 - KAFKA-12992; Make kraft configuration properties public (#10971)
  • PR-9878 - KAFKA-9687: KIP-707: Add KafkaFuture.toCompletionStage() (#9878)
  • PR-10965 - KAFKA-13029; Set appropriate fields for FindCoordinatorRequest based on version (#10965)
  • PR-10932 - KAFKA-12958: add an invariant that notified leaders are never asked to load snapshot (#10932)
  • PR-10919 - KAFKA-12985: Upgrade jetty to 9.4.42 (#10919)
  • PR-10805 - KAFKA-12436: Deprecate MirrorMaker v1 (KIP-720) (#10805)
  • PR-10854 - KAFKA-12717: Remove internal Connect converter properties (KIP-738) (#10854)
  • 51796bcd - KAFKA-10587; Rename kafka-mirror-maker CLI command line arguments for KIP-629
  • PR-10946 - KAFKA-12997: Expose the append time for batches from raft (#10946)
  • PR-10917 - KAFKA-10847: improve throughput of stream-stream join with spurious left/outer join fix (#10917)
  • PR-10813 - KAFKA-9559: Change default serde to be null (#10813)
  • PR-10896 - KAFKA-12964: Collect and rename snapshot files prior to async deletion. (#10896)
  • PR-10940 - KAFKA-13007; KafkaAdminClient getListOffsetsCalls reuse cluster snapshot (#10940)
  • PR-10743 - KAFKA-12663: Update FindCoordinator to support batch lookups (KIP-699) (#10743)
  • PR-9497 - KAFKA-10619: Configure producer with idempotence and acks all by default (KIP-679) (#9497)
  • PR-10652 - KAFKA-9726: Add IdentityReplicationPolicy to MirrorMaker2 (#10652)
  • PR-10949 - KAFKA-13019: Add MetadataImage and MetadataDelta classes for KRaft Snapshots (#10949)
  • PR-10936 - KAFKA-13002: listOffsets must downgrade immediately for non MAX_TIMESTAMP specs (#10936)
  • PR-10892 - KAFKA-13011; Update deleteTopics Admin API (#10892)
  • PR-10822 - KAFKA-4793: Connect API to restart connector and tasks (KIP-745) (#10822)
  • PR-10812 - KAFKA-12863: Configure controller snapshot generation (#10812)
  • PR-10926 - KAFKA-8613: New APIs for Controlling Grace Period for Windowed Operations (#10926)
  • PR-10918 - KAFKA-12756: Update ZooKeeper to v3.6.3 (#10918)
  • PR-10939 - Avoid increasing app ID when test is executed multiple times (#10939)
  • PR-10221 - KAFKA-12379: Allow configuring the location of the offset-syncs topic with MirrorMaker2 (#10221)
  • PR-10899 - KAFKA-12952 Add header and footer records for raft snapshots (#10899)
  • PR-10763 - KAFKA-12520: Ensure log loading does not truncate producer state unless required (#10763)
  • PR-10930 - KAFKA-12996; Return OFFSET_OUT_OF_RANGE for fetchOffset < startOffset even for diverging epochs (#10930)
  • PR-10858 - KAFKA-12926: ConsumerGroupCommand’s java.lang.NullPointerException at negative offsets while running kafka-consumer-groups.sh (#10858)
  • PR-10913 - KAFKA-12631; Implement resign API in KafkaRaftClient (#10913)
  • PR-10824 - KAFKA-12718: SessionWindows are closed too early (#10824)
  • PR-10893 - KAFKA-12909: add missing tests (#10893)
  • PR-10894 - KAFKA-12951: restore must terminate for tx global topic (#10894)
  • PR-10923 - KAFKA-12976; Remove UNSUPPORTED_VERSION error from delete topics call (#10923)
  • PR-10916 - KAFKA-12938: Fix and reenable testChrootExistsAndRootIsLocked test (#10916)
  • PR-10922 - KAFKA-12790: Remove SslTransportLayerTest.testUnsupportedTlsVersion (#10922)
  • PR-10840 - KAFKA-12849: KIP-744 TaskMetadata ThreadMetadata StreamsMetadata as API (#10840)
  • PR-10760 - KAFKA-12541; Extend ListOffset to fetch offset with max timestamp (KIP-734) (#10760)
  • PR-10900 - KAFKA-12967; KRaft broker should forward DescribeQuorum to controller (#10900)
  • PR-10190 - KAFKA-12336 Custom stream naming does not work while calling stream[K… (#10190)
  • PR-10879 - KAFKA-12991; Fix unsafe access to AbstractCoordinator.state (#10879)
  • PR-10878 - KAFKA-12898; Owned partitions in the subscription must be sorted (#10878)
  • PR-10841 - KAFKA-12482 Remove deprecated rest.host.name and rest.port configs (#10841)
  • PR-10862 - KAFKA-12928: Add a check whether the Task’s statestore is actually a directory (#10862)
  • PR-10883 - KAFKA-12949; Match null when eventQueue poll hits eventTimeoutMs in TestRaftServer (#10883)
  • PR-10814 - KAFKA-12888; Add transaction tool from KIP-664 (#10814)
  • PR-10869 - KAFKA-10546: Deprecate old PAPI (#10869)
  • PR-10834 - KAFKA-12904: Corrected the timeout for config validation REST API resource (#10834)
  • PR-10336 - KAFKA-12483: Enable client overrides in connector configs by default (KIP-722) (#10336)
  • PR-10335 - KAFKA-12484: Enable Connect’s connector log contexts by default (KIP-721) (#10335)
  • PR-10377 - KAFKA-12515; Feature support should be ignorable in ApiVersion response (#10377)
  • PR-10818 - KAFKA-12889: log clean relative index range check of group consider empty log segment to avoid too many empty log segment left (#10818)
  • PR-10880 - KAFKA-12870; Flush in progress not cleared after transaction completion (#10880)
  • PR-10902 - KAFKA-12837; Process entire batch reader in the BrokerMetadataListener commit handler (#10902)
  • PR-10733 - KAFKA-12816 Added tiered storage related configs including remote log manager configs. (#10733)
  • PR-10754 - KAFKA-12835: Topic IDs can mismatch on brokers (after interbroker protocol version update) (#10754)
  • PR-10836 - KAFKA-12906; Added RecordDeserializationException containing partition and offset (#10836)
  • PR-10872 - KAFKA-12945: Remove port, host.name and related configs in 3.0 (#10872)
  • PR-10588 - KAFKA-12662: add unit test for ProducerPerformance (#10588)
  • PR-10863 - KAFKA-12890; Consumer group stuck in CompletingRebalance (#10863)
  • PR-10701 - KAFKA-10437: Update WordCount examples to use new PAPI (#10701)
  • PR-10888 - KAFKA-12955: Fix LogLoader to pass materialized view of segments for deletion (#10888)
  • PR-10861 - KAFKA-12909: disable spurious left/outer stream-stream join fix for old JoinWindows API (#10861)
  • PR-10867 - KAFKA-12931: KIP-746: Revise KRaft Metadata Records (#10867)
  • PR-10804 - KAFKA-12877: Make flexibleVersions mandatory (#10804)
  • PR-10786 - KAFKA-12787; Integrate controller snapshoting with raft client (#10786)
  • PR-10749 - KAFKA-12773; Use UncheckedIOException when wrapping IOException (#10749)
  • PR-10584 - KAFKA-12701: NPE in MetadataRequest when using topic IDs (#10584)
  • PR-10882 - KAFKA-12948: Remove node from ClusterConnectionStates.connectingNodes when node is removed (#10882)
  • PR-10846 - KAFKA-12914: StreamSourceNode should return null topic name for pattern subscription (#10846)
  • PR-10871 - KAFKA-8940: decrease session timeout to make test faster and reliable (#10871)
  • PR-10847 - KAFKA-12921: Upgrade zstd-jni to 1.5.0-2 (#10847)
  • PR-10820 - KAFKA-12892: Disable testChrootExistsAndRootIsLocked (#10820)
  • PR-10865 - KAFKA-12934: Move some controller classes to the metadata package (#10865)
  • PR-10736 - KAFKA-9295: revert session timeout to default value (#10736)
  • PR-10850 - KAFKA-12924 Replace EasyMock and PowerMock with Mockito in streams metrics tests (#10850)
  • PR-9414 - KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup (#9414)
  • PR-10849 - KAFKA-12922: MirrorCheckpointTask should close topic filter (#10849)
  • PR-10835 - KAFKA-12905: Replace EasyMock and PowerMock with Mockito for NamedCacheMetricsTest (#10835)
  • PR-10665 - KAFKA-9009: increase replica.lag.time.max.ms to make the test reliable (#10665)
  • PR-10803 - KAFKA-12874; Increase default consumer session timeout to 45s (#10803)
  • PR-10609 - KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure (#10609)
  • PR-10823 - KAFKA-12897: KRaft multi-partition placement on single broker (#10823)
  • PR-10793 - KAFKA-12338; Remove unused MetadataParser (#10793)
  • PR-9441 - KAFKA-10614: Ensure group state (un)load is executed in the right order (#9441)
  • PR-10821 - KAFKA-12892: Use dedicated root in ZK ACL test (#10821)
  • PR-580 - CONFLUENT: Complete version regex check in system tests
  • PR-10646 - KAFKA-8897 Follow-up: Consolidate the global state stores (#10646)
  • PR-10471 - KAFKA-12597: Remove deprecated –zookeeper option in ReassignPartitionsCommand (#10471)
  • PR-10752 - KAFKA-12620 Allocate Producer IDs in KRaft controller (#10752)
  • PR-10664 - KAFKA-12749: Changelog topic config on suppressed KTable lost (#10664)
  • PR-10787 - KAFKA-12864: Move KafkaEventQueue into server-common. #10787 (#10787)
  • PR-10552 - KAFKA-12675: improve the sticky general assignor scalability and performance (#10552)
  • PR-10808 - KAFKA-12880: Remove deprecated Count and SampledTotal in 3.0 (#10808)
  • PR-10797 - KAFKA-12867: Fix ConsumeBenchWorker exit behavior for maxMessages config (#10797)
  • PR-10616 - KAFKA-12709; Add Admin API for ListTransactions (#10616)
  • PR-10457 - KAFKA-12596: remove –zookeeper option from topic command (#10457)
  • PR-10795 - KAFKA-12866: Avoid root access to Zookeeper (#10795)
  • PR-10765 - KAFKA-12519: Remove built-in Streams metrics for versions 0.10.0-2.4 (#10765)
  • PR-10790 - KAFKA-12865 : Fix doc error in Admin.describeAcls API (#10790)
  • PR-10744 - KAFKA-8410: KTableProcessor migration groundwork (#10744)
  • PR-10704 - KAFKA-12791: ConcurrentModificationException in AbstractConfig use by KafkaProducer (#10704)
  • PR-10778 - KAFKA-12856: Upgrade Jackson to 2.12.3 (#10778)
  • PR-10710 - KAFKA-12796: Removal of deprecated classes under streams-scala (#10710)
  • PR-10762 - KAFKA-12819: Add assert messages to MirrorMaker tests plus other quality of life improvements (#10762)
  • PR-10017 - KAFKA-12260: Avoid hitting NPE for partitionsFor (#10017)
  • PR-10717 - KAFKA-12800: Configure generator to fail on trailing JSON tokens (#10717)
  • PR-577 - KC-1749: Upgrade Jetty, commons-codec, etc
  • PR-576 - KC-1748: Upgrade Jetty and commons-codec
  • PR-10482 - KAFKA-12499: add transaction timeout verification (#10482)
  • PR-10504 - KAFKA-12620 Allocate producer ids on the controller (#10504)
  • PR-10724 - KAFKA-12808: Remove Deprecated Methods under StreamsMetrics (#10724)
  • PR-10375 - KAFKA-12522: Cast SMT should allow null value records to pass through (#10375)
  • 57c14224 - KAFKA-12820: Upgrade maven-artifact dependency
  • PR-10572 - KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller (#10572)
  • PR-10705 - KAFKA-12342: Remove MetaLogShim and use RaftClient directly (#10705)
  • PR-10735 - KAFKA-12779: KIP-740, Clean up public API in TaskId and fix TaskMetadata#taskId() (#10735)
  • PR-10737 - KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs (#10737)
  • PR-10729 - KAFKA-12809: Remove deprecated methods of Stores factory (#10729)
  • PR-10730 - KAFKA-12813: Remove deprecated schedule method in ProcessorContext (#10730)
  • PR-10720 - KAFKA-12815: Preserve context for KTable.transformValues when getting value from upstream state store (#10720)
  • PR-10727 - KAFKA-12810: Remove deprecated TopologyDescription.Source#topics (#10727)
  • PR-564 - APPSEC-60: Upgrade org.apache.maven:apache-maven
  • PR-10700 - KAFKA-12789: Remove Stale comments for meta response handling logic (#10700)
  • PR-10719 - Fix compile errors from KAFKA-12543 (#10719)
  • PR-10431 - KAFKA-12543: Change RawSnapshotReader ownership model (#10431)
  • PR-10561 - KAFKA-12686 AlterIsr and LeaderAndIsr race condition (#10561)
  • PR-10494 - KAFKA-12788: improve KRaft replica placement (#10494)
  • PR-10633 - KAFKA-12751: Reset AlterIsr in-flight state for duplicate update requests (#10633)
  • PR-10675 - KAFKA-12574: remove internal Producer config and auto downgrade logic (#10675)
  • PR-10707 - KAFKA-12792: Fix metrics bug and introduce TimelineInteger (#10707)
  • PR-10542 - KAFKA-12313: KIP-725: Streamlining configs for Windowed Deserialisers (#10542)
  • PR-10709 - KAFKA-12794: Fix trailing json tokens in DescribeProducersRequest.json (#10709)
  • PR-10620 - KAFKA-12736: KafkaProducer.flush holds onto completed ProducerBatch(s) until flush completes (#10620)
  • PR-10606 - KAFKA-12728: Upgrade gradle to 7.0.2 and shadow to 7.0.0 (#10606)
  • PR-10688 - KAFKA-12778: Fix QuorumController request timeouts and electLeaders (#10688)
  • PR-10634 - KAFKA-12754: Improve endOffsets for TaskMetadata (#10634)
  • PR-10680 - Rework on KAFKA-3968: fsync the parent directory of a segment file when the file is created (#10680)
  • PR-6592 - KAFKA-8326: Introduce List Serde (#6592)
  • PR-10679 - KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller (#10679)
  • PR-10676 - KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata (#10676)
  • PR-10667 - KAFKA-12772: Move all transaction state transition rules into their states (#10667)
  • PR-10638 - KAFKA-12758 Added server-common module to have server side common classes. (#10638)
  • PR-10657 - KAFKA-5876: KIP-216 Part 4, Apply InvalidStateStorePartitionException for Interactive Queries (#10657)
  • PR-10643 - KAFKA-12747: Fix flakiness in shouldReturnUUIDsWithStringPrefix (#10643)
  • PR-10645 - KAFKA-12464: minor code cleanup and additional logging in constrained sticky assignment (#10645)
  • PR-10390 - KAFKA-12536: Add Instant-based methods to ReadOnlySessionStore (#10390)
  • PR-10640 - KAFKA-10847: Set StreamsConfig on InternalTopologyDriver before writing topology (#10640)
  • PR-3547 - KC-1618: Adapt the KRaft README and example configs for CP (#3547)
  • PR-10568 - KAFKA-8897: Upgrade RocksDB to 6.19.3 (#10568)
  • PR-10296 - KAFKA-12451: Remove deprecation annotation on long-based read operations in WindowStore (#10296)
  • PR-10636 - KAFKA-12752: Bump Jersey deps to 2.34 (#10636)
  • PR-10509 - KAFKA-12464: enhance constrained sticky Assign algorithm (#10509)
  • PR-10532 - KAFKA-8531: Change default replication factor config (#10532)
  • PR-10014 - KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes (#10014)
  • PR-10613 - KAFKA-10847: Set shared outer store to an in-memory store when in-memory stores are supplied (#10613)
  • PR-10271 - KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage. (#10271)
  • PR-9779 - KAFKA-10767: Adding test cases for all, reverseAll and reverseRange for ThreadCache (#9779)
  • PR-10294 - KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore (#10294)
  • PR-10612 - KAFKA-10847: Add internal flag to disable KAFKA-10847 fix (#10612)
  • PR-10597 - KAFKA-5876: KIP-216 Part 3, Apply StreamsNotStartedException for Interactive Queries (#10597)
  • PR-10446 - KAFKA-12661 ConfigEntry#equal does not compare other fields when value is NOT null (#10446)
  • PR-10085 - KAFKA-12154; Raft Snapshot Loading API (#10085)
  • PR-10557 - KAFKA-12683: Remove deprecated UsePreviousTimeOnInvalidTimestamp (#10557)
  • PR-10615 - KAFKA-12648: basic skeleton API for NamedTopology (#10615)
  • PR-10548 - KAFKA-12396: added null check for state stores key (#10548)
  • PR-555 - fix: Upgrade Jersey to 2.34
  • PR-10480 - KAFKA-12265; Move the BatchAccumulator in KafkaRaftClient to LeaderState (#10480)
  • PR-10611 - KAFKA-12730; Avoid duplicate logout if Kerberos login fails (#10611)
  • PR-10462 - KAFKA-10847: Fix spurious results on left/outer stream-stream joins (#10462)
  • PR-10573 - KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2 (#10573)
  • PR-554 - fix: Upgrade jersey to 2.34
  • PR-10466 - KAFKA-12417: streams copyDependentLibs should not copy testRuntime configuration jars (#10466)
  • PR-10547 - KAFKA-12284: increase request timeout to make tests reliable (#10547)
  • PR-10599 - KAFKA-12716; Add Admin API to abort transactions (#10599)
  • PR-8923 - KAFKA-6435: KIP-623 Add internal topics option to streamResetter (#8923)
  • PR-10519 - KAFKA-12344 Support SlidingWindows in the Scala API (#10519)
  • PR-10558 - KAFKA-12684: Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand (#10558)
  • PR-9627 - KAFKA-10746: Change to Warn logs when necessary to notify users (#9627)
  • PR-9821 - KAFKA-5876: Apply UnknownStateStoreException for Interactive Queries (#9821)
  • PR-9640 - KAFKA-10283; Consolidate client-level and consumer-level assignment within ClientState (#9640)
  • PR-10483 - KAFKA-12586; Add DescribeTransactions Admin API (#10483)
  • PR-10580 - KAFKA-12704: Improve cache access during connector class instantiation in config validations (#10580)
  • PR-10574 - KAFKA-12700: override toString method to show correct value in doc (#10574)
  • PR-10575 - KAFKA-12702: Fix NPE in networkListeners from BrokerServer (#10575)
  • PR-10565 - KAFKA-12691: Add case where task can be considered idling (#10565)
  • PR-10478 - KAFKA-12553: Refactor recovery logic to introduce LogLoader (#10478)
  • PR-10537 - KAFKA-10847: Delete Time-ordered duplicated records using deleteRange() internally (#10537)
  • PR-10508 - KAFKA-12633: Remove deprecated APIs in TopologyTestDriver (#10508)
  • PR-9549 - KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader (#9549)
  • PR-10545 - KAFKA-12672: Added config for raft testing server (#10545)
  • PR-10470 - KAFKA-12612: Remove checksum from ConsumerRecord/RecordMetadata for 3.0 (#10470)
  • PR-10536 - KAFKA-12667: Fix incorrect error log on StateDirectory close (#10536)
  • PR-10529 - KAFKA-12650: fix NPE in InternalTopicManagerTest (#10529)
  • PR-10409 - KAFKA-9295: improve KTableKTableForeignKeyInnerJoinMultiIntegrationTest (#10409)
  • PR-10343 - KAFKA-12471: Implement createPartitions in KIP-500 mode (#10343)
  • PR-10218 - KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager. (#10218)
  • PR-10511 - KAFKA-12584; Remove deprecated Sum and Total classes (#10511)
  • PR-10526 - KAFKA-12655 Update Jetty: 9.4.38.v20210224 → 9.4.39.v20210325 (#10526)
  • PR-10469 - KAFKA-12611: Fix using random payload in ProducerPerformance incorrectly (#10469)
  • PR-10531 - KAFKA-12658: Include kafka-shell jar and dependencies in release tar (#10531)
  • PR-10512 - KAFKA-12637: Remove deprecated PartitionAssignor interface (#10512)
  • PR-10411 - KAFKA-7606: Remove deprecated options from StreamsResetter (#10411)
  • PR-10042 - KAFKA-9527: fix NPE when using time-based argument for Stream Resetter Tool (#10042)
  • PR-10503 - KAFKA-9988: Suppress uncaught exceptions in log messages during Connect task shutdown (#10503)
  • PR-10258 - KAFKA-12408: Document omitted ReplicaManager metrics (#10258)
  • PR-10510 - KAFKA-12607; Test case for resigned state vote granting (#10510)
  • PR-10381 - KAFKA-8410: Migrate KStream Stateless operators to new Processor API (#10381)
  • PR-10301 - KAFKA-9831: increase max.poll.interval.ms to avoid unexpected rebalance (#10301)
  • PR-10293 - KAFKA-12449: Remove deprecated WindowStore#put (#10293)
  • PR-10427 - KAFKA-12591; Remove deprecated quota.producer.default and quota.consumer.default configurations (#10427)
  • PR-10486 - KAFKA-12492: Fix the formatting of example RocksDBConfigSetter (#10486)
  • PR-10481 - KAFKA-12619; Raft leader should expose hw only after committing LeaderChange (#10481)
  • PR-10254 - KAFKA-12406 Integrate client quotas with KRaft broker (#10254)
  • PR-10492 - KAFKA-12457; Add sentinel ID to metadata topic (#10492)
  • PR-10502 - KAFKA-12630: Remove deprecated KafkaClientSupplier#getAdminClient in Streams (#10502)
  • PR-10421 - KAFKA-12568: Remove deprecated APIs in KStream, KTable and Joined (#10421)
  • PR-10380 - KAFKA-12527: Remove deprecated PartitionGrouper annotation (#10380)
  • PR-10302 - KAFKA-7785: move internal DefaultPartitionGrouper (#10302)
  • PR-10331 - KAFKA-10847: Add new RocksDBTimeOrderedWindowStore that persists (time-key)-value records (#10331)
  • PR-9851 - KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup… (#9851)
  • PR-10389 - KAFKA-12384: stabilize ListOffsetsRequestTest#testResponseIncludesLeaderEpoch (#10389)
  • PR-10131 - KAFKA-5146: remove Connect dependency from Streams module (#10131)
  • PR-10474 - KAFKA-12602: Fix LICENSE file (#10474)
  • PR-10366 - KAFKA-12467: Implement QuorumController snapshot generation (#10366)
  • PR-10438 - KAFKA-12579: Remove various deprecated clients classes/methods for 3.0 (#10438)
  • PR-10142 - KAFKA-12294; forward auto topic request within envelope on behalf of clients (#10142)
  • PR-10445 - KAFKA-12548; Propagate record error messages to application (#10445)
  • PR-10393 - KAFKA-12539; Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity (#10393)
  • PR-9992 - KAFKA-10759 Add ARM build stage (#9992)
  • PR-10477 - KAFKA-12615: Fix Selector.clear() javadoc typo (#10477)
  • PR-10450 - KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0 (#10450)
  • PR-10405 - KAFKA-3968: fsync the parent directory of a segment file when the file is created (#10405)
  • PR-10325 - MONOR: Remove redudant LocalLogManager (#10325)
  • PR-10447 - KAFKA-12587 Remove KafkaPrincipal#fromString for 3.0 (#10447)
  • PR-10396 - KAFKA-12474: Handle failure to write new session keys gracefully (#10396)
  • PR-10282 - KAFKA-12426: Missing logic to create partition.metadata files in RaftReplicaManager (#10282)
  • PR-10452 - KAFKA-12593: Fix Apache License headers (#10452)
  • PR-10458 - KAFKA-12600: Remove deprecated config value default for client config client.dns.lookup (#10458)
  • PR-10451 - KAFKA-12952: Remove deprecated LogConfig.Compact (#10451)
  • PR-10454 - Initial commit (#10454)
  • PR-10430 - KAFKA-12575: Eliminate Log.isLogDirOffline boolean attribute (#10430)
  • PR-10443 - KAFKA-8405; Remove deprecated kafka-preferred-replica-election command (#10443)
  • 6703cc20 - KAFKA-12583: Upgrade netty to 4.1.62.Final
  • PR-10436 - KAFKA-12577; Remove deprecated ConfigEntry constructor for 3.0 (#10436)
  • PR-10440 - KAFKA-12581: Remove deprecated Admin.electPreferredLeaders (#10440)
  • PR-10435 - KAFKA-12578: Remove deprecated security classes/methods for 3.0 (#10435)
  • PR-10426 - KAFKA-12571: Eliminate LeaderEpochFileCache constructor dependency on logEndOffset (#10426)
  • PR-10342 - KAFKA-12288: remove task-level filesystem locks (#10342)
  • PR-10418 - KAFKA-12509 Tighten up StateDirectory thread locking (#10418)
  • PR-10401 - KAFKA-12552: Introduce LogSegments class abstracting the segments map (#10401)
  • PR-10404 - KAFKA-12557: Fix hanging KafkaAdminClientTest (#10404)
  • PR-10425 - KAFKA-12573; Remove deprecated Metric#value (#10425)
  • PR-10416 - KAFKA-8784: remove default close for RocksDBConfigSetter (#10416)
  • PR-10407 - KAFKA-12523: handle TaskCorruption and TimeoutException during handleCorruption and handleRevocation (#10407)
  • PR-10378 - KAFKA-7106: remove deprecated Windows APIs (#10378)
  • PR-10412 - KAFKA-12562: Remove deprecated APIs in KafkaStreams and returned state classes (#10412)
  • PR-10387 - KAFKA-12537: fix application shutdown corner case with only one thread (#10387)
  • PR-10300 - KAFKA-12452: Remove deprecated overloads of ProcessorContext#forward (#10300)
  • PR-10397 - KAFKA-12508: Disable KIP-557 (#10397)
  • PR-10275 - KAFKA-12434; Admin support for DescribeProducers API (#10275)
  • PR-10392 - KAFKA-12435: Fix javadoc errors (#10392)
  • PR-10281 - KAFKA-12432: AdminClient should time out nodes that are never ready (#10281)
  • PR-9858 - KAFKA-12173 Migrate streams:streams-scala module to JUnit 5 (#9858)
  • PR-10379 - KAFKA-12524: Remove deprecated segments() (#10379)
  • PR-10374 - (Cherry-pick) KAFKA-9274: handle TimeoutException on task reset (#10000) (#10374)
  • PR-10371 - KAFKA-12479: Batch partition offset requests in ConsumerGroupCommand (#10371)
  • PR-10309 - KAFKA-12181; Loosen raft fetch offset validation of remote replicas (#10309)
  • PR-10372 - (Cherry-pick) KAFKA-9274: handle TimeoutException on task reset (#10000) (#10372)
  • PR-10360 - KAFKA-12508: Emit records with same value and same timestamp (#10360)
  • PR-10220 - KAFKA-12383: Get RaftClusterTest.java and other KIP-500 junit tests working (#10220)
  • PR-10369 - KAFKA-12514: Fix NPE in SubscriptionState (#10369)
  • PR-10150 - KAFKA-3745: Add access to read-only key in value joiner (#10150)
  • PR-10355 - KAFKA-12500: fix memory leak in thread cache (#10355)
  • PR-10286 - KAFKA-12318: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper (#10286)
  • PR-10361 - KAFKA-9189: Use MetadataCache instead of ZooKeeper during controlled shutdown to avoid hang (#10361)
  • PR-10289 - KAFKA-12440; ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot (#10289)
  • PR-10276 - KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog’s validateOffsetAndEpoch (#10276)
  • PR-10299 - KAFKA-10070: parameterize Connect unit tests to remove code duplication (#10299)
  • PR-10356 - KAFKA-12503: inform threads to resize their cache instead of doing so for them (#10356)
  • PR-10332 - KAFKA-10697: Remove ProduceResponse.responses (#10332)
  • PR-10227 - KAFKA-12382: add a README for KIP-500 (#10227)
  • PR-10317 - KAFKA-10357: Add setup method to internal topics (#10317)
  • 9ffa6d6d - KAFKA-12293: Remove JCenter from buildscript and delete buildscript.gradle
  • PR-10341 - KAFKA-12491: Make rocksdb an api dependency for streams (#10341)
  • PR-10304 - KAFKA-12454: Add ERROR logging on kafka-log-dirs when given brokerIds do not exist in current kafka cluster (#10304)
  • PR-10323 - KAFKA-12459; Use property testing library for raft event simulation tests (#10323)
  • PR-10322 - KAFKA-12455: Fix OffsetValidationTest.test_broker_rolling_bounce failure with Raft (#10322)
  • PR-10318 - KAFKA-12330; FetchSessionCache may cause starvation for partitions when FetchResponse is full (#10318)
  • PR-10267 - KAFKA-12427: Don’t update connection idle time for muted connections (#10267)
  • PR-10320 - revert stream logging level back to ERROR (#10320)
  • PR-10232 - KAFKA-12352: Make sure all rejoin group and reset state has a reason (#10232)
  • PR-10135 - KAFKA-10348: Share client channel between forwarding and auto creation manager (#10135)
  • PR-10285 - KAFKA-12442: Upgrade ZSTD JNI from 1.4.8-4 to 1.4.9-1 (#10285)
  • PR-10311 - KAFKA-12462: proceed with task revocation in case of thread in PENDING_SHUTDOWN (#10311)
  • PR-10310 - KAFKA-12460; Do not allow raft truncation below high watermark (#10310)
  • PR-10305 - KAFKA-10357: Add missing repartition topic validation (#10305)
  • PR-10021 - KAFKA-12205; Delete snapshots less than the snapshot at the log start (#10021)
  • PR-10266 - KAFKA-10357: Add validation method for internal topics (#10266)
  • PR-10092 - KAFKA-12287: Add WARN logging on consumer-groups when reset-offsets by timestamp or duration can’t find an offset and defaults to latest (#10092)
  • PR-10284 - KAFKA-12441: remove deprecated method StreamsBuilder#addGlobalStore (#10284)
  • PR-9744 - KAFKA-10062: Add a methods to retrieve the current timestamps as known by the Streams app (#9744)
  • PR-10118 - KAFKA-10192: Increase max time to wait for worker to start in some integration tests (#10118)
  • PR-528 - KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes
  • PR-10252 - KAFKA-12403; Ensure local state deleted on RemoveTopicRecord received (#10252)
  • PR-10240 - KAFKA-12381: remove live broker checks for forwarding topic creation (#10240)
  • PR-10211 - KAFKA-12347: updating TaskMetadata (#10211)
  • PR-525 - Enable downstream validation
  • PR-10260 - Remove ReassigningPartitions metric when shutting down ReplicaManager (#10260)
  • PR-10253 - KAFKA-12376: Apply atomic append to the log (#10253)
  • PR-10203 - KAFKA-12415 Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies (#10203)
  • PR-10263 - KAFKA-12393: Document multi-tenancy considerations (#334) (#10263)
  • PR-10257 - KAFKA-12407: Document Controller Health Metrics (#10257)
  • PR-9950 - KAFKA-12170: Fix for Connect Cast SMT to correctly transform a Byte array into a string (#9950)
  • PR-10185 - KAFKA-12284: Wait for mirrored topics to be created (#10185)
  • PR-10173 - KAFKA-9548 Added SPIs and public classes/interfaces introduced in KIP-405 for tiered storage feature in Kafka. (#10173)
  • b77deece - KAFKA-12400: Upgrade jetty
  • PR-10215 - KAFKA-12375: don’t reuse thread.id until a thread has fully shut down (#10215)
  • PR-10217 - KAFKA-12254: Ensure MM2 creates topics with source topic configs (#10217)
  • PR-10216 - KAFKA-12177: apply log start offset retention before time and size based retention (#10216)
  • PR-10206 - KAFKA-12369; Implement ListTransactions API (#10206)
  • PR-10163 - KAFKA-10357: Extract setup of changelog from Streams partition assignor (#10163)
  • PR-10228 - KAFKA-10251: increase timeout for consuming records (#10228)
  • PR-10223 - KAFKA-12394; Return TOPIC_AUTHORIZATION_FAILED in delete topic response if no describe permission (#10223)
  • PR-9717 - KAFKA-10766: Unit test cases for RocksDBRangeIterator (#9717)
  • PR-10052 - KAFKA-12289: Adding test cases for prefix scan in InMemoryKeyValueStore (#10052)
  • PR-10137 - KAFKA-12268: Implement task idling semantics via currentLag API (#10137)
  • PR-10205 - KAFKA-12323 Follow-up: Refactor the unit test a bit (#10205)
  • PR-10016 - KAFKA-10340: Proactively close producer when cancelling source tasks (#10016)
  • PR-10141 - KAFKA-12329; kafka-reassign-partitions command should give a better error message when a topic does not exist (#10141)
  • PR-10199 - KAFKA-12374: Add missing config sasl.mechanism.controller.protocol (#10199)
  • PR-8812 - KAFKA-10101: Fix edge cases in Log.recoverLog and LogManager.loadLogs (#8812)
  • PR-10213 - KAFKA-12375: fix concurrency issue in application shutdown (#10213)
  • PR-9990 - KAFKA-12235: Fix a bug where describeConfigs does not work on 2+ config keys (#9990)
  • PR-9235 - KAFKA-10449: Add some important parameter desc in connect-distributed.properties (#9235)
  • PR-10194 - KAFKA-12365; Disable APIs not supported by KIP-500 broker/controller (#10194)
  • PR-10059 - KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while building SslTransportLayer (#10059)
  • PR-10200 - KAFKA-12367; Ensure partition epoch is propagated to Partition state (#10200)
  • PR-10183 - KAFKA-12267; Implement DescribeTransactions API (#10183)
  • PR-10165 - KAFKA-12350: Correct the default value in doc (#10165)
  • PR-10170 - KAFKA-12323: Set timestamp in record context when punctuate (#10170)
  • PR-10178 - KAFKA-12361; Use default request.timeout.ms value for Connect producers (#10178)
  • PR-10191 - KAFKA-12341: Ensure consistent versions for javassist (#10191)
  • PR-10174 - KAFKA-12357: Do not inline methods from the scala package by default (#10174)
  • PR-10024 - KAFKA-12273 InterBrokerSendThread#pollOnce throws FatalExitError even… (#10024)
  • 9e452950 - KAFKA-12324: Upgrade jetty
  • PR-10152 - KAFKA-12339: Add retry to admin client’s listOffsets (#10152)
  • PR-10153 - KAFKA-12340: Fix potential resource leak in Kafka*BackingStore (#10153)
  • PR-10145 - KAFKA-12335 Upgrade junit from 5.7.0 to 5.7.1 (#10145)
  • PR-10070 - KAFKA-12276: Add the quorum controller code (#10070)
  • PR-9631 - KAFKA-9672: Leader with ISR as a superset of replicas (#9631)
  • 690f72dd - KAFKA-12334: Add the KIP-500 metadata shell
  • PR-10129 - KAFKA-10817; Add clusterId validation to raft Fetch handling (#10129)
  • PR-10158 - KAFKA-12343: Handle exceptions better in TopicAdmin, including UnsupportedVersionException (#10158)
  • PR-10072 - KAFKA-9274: Throw TaskCorruptedException instead of TimeoutException when TX commit times out (#10072)
  • PR-10143 - KAFKA-12332; Error partitions from topics with invalid IDs in LISR requests (#10143)
  • PR-10063 - KAFKA-12258; Add support for splitting appending records (#10063)
  • PR-10091 - KAFKA-9524: increase retention time for window and grace periods longer than one day (#10091)
  • PR-10138 - KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch (#10138)
  • PR-10666 - KAFKA-12278; Ensure exposed api versions are consistent within listener (#10666)
  • PR-10133 - Fixed README and added clearer error message. (#10133)
  • PR-10073 - KAFKA-12303: Fix handling of null values by Flatten SMT (#10073)
  • PR-9906 - KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of… (#9906)
  • PR-10102 - KAFKA-12272: Fix commit-interval metrics (#10102)
  • PR-10123 - KAFKA-12327: Remove MethodHandle usage in CompressionType (#10123)
  • PR-10122 - KAFKA-12326: Corrected regresion in MirrorMaker 2 executable introduced with KAFKA-10021 (#10122)
  • PR-9780 - KAFKA-10021: Changed Kafka backing stores to use shared admin client to get end offsets and create topics (#9780)
  • b3313b86 - KAFKA-12297: Make MockProducer return RecordMetadata with values as per contract
  • PR-10098 - KAFKA-12321 the comparison function for uuid type should be ‘equals’ rather than ‘==’ (#10098)
  • PR-9430 - KAFKA-5235: GetOffsetShell: Support for multiple topics and consumer configuration override (KIP-635) (#9430)
  • PR-9986 - JUnit extensions for integration tests (#9986)
  • PR-9865 - Fill in the 2.8 release note for Authorizer (#9865)

ksqlDB

  • PR-8051 - chore: commons-codec override
  • PR-7936 - refactor: support BYTES on QTT for Protobuf
  • PR-7932 - fix: dont use default package for version.properties
  • PR-7921 - fix: increase default command response timeout
  • PR-7909 - feat: enable BYTES for LPAD and RPAD
  • PR-7923 - fix: checkstyle remove separation in imports
  • PR-7917 - Bytes split function
  • PR-7904 - feat: allow expressions on left table columns in FK-joins
  • PR-7880 - Improves RQTT by using offsets to measure readiness
  • PR-7915 - fix: no named topologies yet
  • PR-7889 - feat: added missing tostring method
  • PR-7818 - build: update multiple modules for spotbug 4.3.0 and re-enable plugin
  • PR-7773 - feat: make SchemaRegistry permission validations on KSQL requests
  • PR-7582 - fix: silently ignores KEY qualifier if enclosed in parentheses (#5852)
  • PR-7871 - feat: allow Java clients to set HTTP/2 multiplex limit
  • PR-7876 - feat: enables BYTES for CONCAT and CONCAT_WS
  • PR-7813 - build: update execution module for spotbug 4.3.0 and re-enable plugin
  • PR-7872 - build: suppress spotbug warning to fix build
  • PR-7865 - feat: update len function to accept BYTES
  • PR-7817 - build: update rest-model module for spotbug 4.3.0 and re-enable plugin
  • PR-7816 - build: update rest-app module for spotbug 4.3.0 and re-enable plugin
  • PR-7815 - build: update parser module for spotbug 4.3.0 and re-enable plugin
  • PR-7812 - build: update engine module for spotbug 4.3.0 and re-enable plugin
  • PR-7810 - build: update common module for spotbug 4.3.0 and re-enable plugin
  • PR-7809 - build: update api-client module for spotbug 4.3.0 and re-enable plugin
  • PR-7804 - feat: implement bytes comparisons
  • PR-7861 - feat: Update SUBSTRING function to accept BYTES types
  • PR-7821 - feat: don’t start queries when corruption is detected during startup
  • PR-7823 - feat: support BYTES column types in Java client API
  • PR-7831 - feat: add TO_BYTES/FROM_BYTES functions for bytes/string conversions
  • PR-7729 - fix: Clean up a terminated query’s state stores
  • PR-7822 - build: fix deprecation warnings from KIP-744
  • PR-7791 - feat: implement serdes for bytes
  • PR-7788 - feat: add interface for metrics reporter
  • PR-7769 - feat: add observability metric skeleton
  • PR-7778 - feat: add BYTES type to ksqlDB
  • PR-7785 - fix: create more user friendly transient query ids
  • PR-7783 - refactor: wrap SchemaRegistry authorization errors into a KsqlSchemaAuthorizationException
  • PR-7761 - feat: add ARRAY_CONCAT UDF
  • PR-7750 - fix: return an error message on http2 /query-stream endpoint
  • PR-7742 - feat: add FROM_DAYS and update UNIX_DATE function
  • PR-7744 - feat: add DATEADD and DATESUB functions
  • PR-7739 - fix: enable KAFKA-10847 bugfix when GRACE PERIOD is used on Joins
  • PR-7733 - feat: add PARSE_DATE and FORMAT_DATE functions
  • PR-7740 - feat: enable date/time for the query interpreter
  • PR-7738 - feat: pull query bandwidth based throttling
  • PR-7727 - feat: add TIMEADD and TIMESUB functions
  • PR-7722 - feat: add PARSE_TIME and FORMAT_TIME functions
  • PR-7737 - fix: enable schema inference for timestamp/time/date
  • PR-7734 - feat: implement comparisons for TIME/DATE
  • PR-7719 - fix: fixed nondeterminism in UdfIndex
  • PR-7724 - fix: remove time/date component when casting timestamp to date/time
  • PR-7718 - feat: implement serde for DATE
  • PR-7709 - fix: enable time unit functions for interpreter
  • PR-7693 - fix: block out of order migrations in migrations tool
  • PR-7708 - feat: implement casting for DATE/TIME
  • PR-7664 - feat: serde for TIME type
  • PR-7678 - feat: call KStreams API grace() when GRACE PERIOD is used in joins
  • PR-7702 - fix: dont incorrectly log events for each pull query
  • PR-7700 - feat: define how DATE data is formatted
  • PR-7661 - Scalable push queries end to end
  • PR-7674 - feat: define how TIME data is formatted
  • PR-7667 - fix: make current java clients compatible with pre-0.15 servers
  • PR-7662 - feat: Support GRACE PERIOD syntax on WITHIN clause with simple window
  • PR-7672 - docs: Additional quickstarts
  • PR-7643 - docs: Removing a duplicated clause.
  • PR-7670 - fix: longer IT timeouts
  • PR-7627 - fix: Fixes race condition exposing uninitialized query
  • PR-7641 - feat: add the DATE and TIME sql types
  • PR-7639 - fix: Sync Vertx version
  • PR-7587 - feat: Adds Scalable Push Query Routing
  • PR-7625 - docs: document parallel build
  • PR-7620 - fix: Update Maven wrapper and document its use to fix version resolution
  • PR-7615 - feat: use Connect default precision for Avro decimals if unspecified (MINOR)
  • PR-7601 - fix: DROP stream for persistent query doesn’t always drop underlying query
  • PR-7577 - fix: allow for KSQL_GC_LOG_OPTS env variable to be passed through wit
  • PR-7591 - feat: ungate support for foreign key joins
  • PR-7585 - fix: qualified select * now works in n-way joins with repartitions and multiple layers of nesting
  • PR-7556 - feat: add idle timeout
  • PR-7554 - fix: better error message on illegal arithmetic with NULL values
  • PR-7544 - fix: reject mismatched decimals from avro topics
  • PR-7523 - chore: Run query anon logger
  • PR-7538 - docs: specify HTTP1.1 for curl
  • PR-7533 - fix: npe on transient query close (#7530)
  • PR-7517 - feat: build physical plan for FK table-table joins
  • PR-7430 - feat: Adds Scalable Push Query physical operators
  • PR-7483 - fix: change the isError to not use the state
  • PR-7424 - feat: Adds ScalablePushRegistry and peeking ability in a persistent query
  • PR-7452 - feat: Allow ksqlDB to detect FK-join table-table join condition
  • PR-6685 - feat: added NULLIF function (#6567)
  • PR-7453 - chore: query anonymizer interface
  • PR-7458 - fix: Fixes org.mock-server to be 5.11.2 to utilize newer netty
  • PR-7434 - feat: made class to compute essential meta-data
  • PR-7427 - fix: worker-poll tasks shouldn’t block main loop
  • PR-7420 - fix: long-running queries shouldn’t block the main event loop
  • PR-7418 - fix: revert schema registry version
  • PR-7357 - feat: implemented working query anonymizer
  • PR-7401 - feat(migrations): add –define flag to apply command
  • PR-7373 - fix: preserve the rest of a struct when one field has a processing error
  • PR-7366 - feat(migrations): support DEFINE and UNDEFINE statements
  • PR-7390 - feat: emit an error reason before closing websocket
  • PR-7246 - feat: Materialize Table-Table join results
  • PR-7365 - fix: add sessionVariables to /ksql endpoint validator
  • PR-7337 - fix: source key format compatibility with non primitive types
  • PR-7320 - refactor: reorganize query management and clean up listeners
  • PR-7335 - feat: enable variable substitution for java client
  • PR-7331 - feat: scatter gather task metadata for query descriptions
  • PR-7336 - fix: append additional fields to row values in KsqlMaterialization.java to fix pull query filtering
  • PR-7324 - fix: Add line breaks to error message
  • PR-7321 - fix: Make topic offset public
  • PR-7272 - feat: Detailed pull query metrics broken down by type and source
  • PR-7284 - feat: add ‘show connector plugins’ syntax
  • PR-7271 - feat: enable variable substitution for /query-stream and /ksql endpoints
  • PR-7309 - refactor: change how we store stats on SourceDescription
  • PR-7238 - fix: Bubble up errors from HARouting unless using StandbyFallbackException
  • PR-7304 - fix: only list known connect worker properties when listing server props
  • PR-7296 - feat(cli): print stats/errors breakdown by host
  • PR-7289 - Set should fail for config that cannot be changed in session
  • PR-7295 - docs: Normalize usage of ‘record’, ‘message’ and ‘event’ in Kafka doc
  • PR-7252 - feat(executor): Allow SourceDescription returning executors to return cluster wide stream stats
  • PR-7268 - fix: fix NPE when backing a record that has null key/values
  • PR-7222 - feat(java client): Add connector functions to java client
  • PR-7243 - fix(java client): allow java client to accept statements with more than one semicolon
  • PR-7151 - feat: expose TaskMetadata in QueryDescription
  • PR-7244 - refactor(executor): factor out a class allowing you to merge local results with results for running the same statement on other nodes
  • PR-7003 - fix: Append state.dir directive to ksql-server.properties
  • PR-7235 - feat: update sourcedescription and errorstats to have statistics as structured objects
  • PR-7212 - fix: Ensures BaseSubscriber.makeRequest is called on context in PollableSubscriber

REST Proxy

  • PR-904 - KREST-2408: Try longer consumer poll in getConsumerLag integration test
  • PR-901 - KREST-2303: Try harder to find ConsumerLag in integration test.
  • PR-897 - KREST-2260: Use super user when creating topic in AuthorizationErrorTest.
  • PR-894 - Fix ambiguous describeConfigs reference in tests
  • PR-893 - Remove ZooKeeperClient/KafkaZkClient usage from tests
  • PR-885 - KREST-1577 Replace a manually set Avro version with the Avro version variable.
  • PR-870 - Use a more lenient test for testBackoffMsControlsPollCalls.
  • PR-862 - KREST-1458: Always allow OPTIONS requests.
  • PR-859 - KREST-1334 Reduce time testing during ce-kafka-images packaging
  • PR-858 - KREST-728: Add OpenAPI spec for Produce V3.
  • PR-832 - KREST-46: Removed property ${confluent.maven.repo}
  • PR-847 - Fixing openapi spec and ConsumerAssignment lag relationship
  • PR-844 - MMA-10490: include count of partitions in topic(s) response
  • b0e7b11c - MMA-10463: list all broker configs
  • 72b4889c - fix: resolve ProduceController early/outside of lambda
  • PR-835 - KREST-865 Remove string usages of the class name of the deprecated SimpleAclAuthorizer
  • PR-834 - KREST-865 Remove usage of a deprecated Admin Client ConfigEntry constructor
  • PR-824 - KREST-771 Fix build breakage by also updating ProduceAction.java
  • PR-823 - KREST-771 Implement an allowlist for API endpoints.
  • PR-817 - Fix mtls test env run script.
  • PR-810 - MMA-9932 list all topic configs
  • 002df165 - Add test environment using mTLS.
  • c1a6387d - increasing poll time…
  • 8076cb46 - retrying only the http response
  • PR-808 - KREST-117: Add missing example for create acl
  • dd71233f - moving expectedOffsets assignment out of retry
  • 52fc9300 - adding additional retry to getConsumerLag_returnsConsumerLag
  • 1e7652a0 - adding retry to ConsumerLagsResourceIntegrationTest
  • PR-287 - SEC-2720: Add SecureRequestCustomizer for correct resolution of http vs https requests.
  • 6fa4e875 - MMA-11014: Wrap correct exception
  • PR-286 - MMA-11014: wrap SSLException
  • PR-280 - KREST-578: Add configurable DosFilter support. (#376
  • PR-270 - Upgrade Jetty to 9.4.43.v20210629
  • PR-263 - KREST-1403: Set RFC7230 mode for connections
  • PR-262 - KREST-1346: Add option to apply rate-limit globally across all connections.
  • 3caee6f9 - Remove flakiness source from some tests.
  • PR-258 - KREST-661: Add support for HTTP/2 on Java 11 and later
  • PR-234 - Update Jetty: 9.4.36.v20210114 → 9.4.38.v20210224
  • PR-238 - KREST-578: Add configurable DosFilter support.
  • PR-240 - KREST-860 Fix deprecated usages of org.apache.kafka.common.Metric.value() in master
  • PR-230 - MMA-10032: Make configure handler methods public

Schema Registry

Note: ZooKeeper is no longer supported for leader election when using Schema Registry.

  • PR-2052 - DGS-2315: Only tombstone previous schemas during hard deletion
  • PR-2043 - DGS-2259 Omit defaults for fields with type Message
  • PR-2026 - DGS-2078 Fix optional Avro schema generation
  • PR-2024 - DGS-2076 Fix NPE when cross referencing schema w/o name
  • PR-2023 - DGS-2073 Fix Protobuf converter for multiple map refs
  • PR-2022 - DGS-2036 Retain deprecated option on Protobuf messages, enums, etc.
  • PR-2018 - DGS-2036 Retain deprecated option on Protobuf fields
  • PR-2017 - DGS-2034 Add force query param when setting mode
  • PR-2016 - DGS-2026 Allow generating JSON schemas with additionalProperties=true
  • PR-2014 - DGS-2020 Allow optional to be overridden in JSON SchemaWrapper
  • PR-2013 - DGS-2018 Ensure fixed names in Avro union are scrubbed
  • PR-2012 - DGS-2018 Ensure names in Avro union are scrubbed
  • PR-2011 - DGS-2016 Add Protobuf compat rule for label
  • PR-2010 - Rename list in ContextResources to listContexts
  • PR-2008 - DGS-1987 Check for invalid contexts
  • PR-2003 - DGS-1976 Add support for context wildcard
  • PR-2000 - DGS-1965 Handle default ctx in context filter
  • PR-1990 - DGS-1879 changed timeout parameter
  • PR-1989 - Upgrade commons-compress to 1.21 (5.4.x and later)
  • PR-1985 - Add confluent-log4j as dependency
  • PR-1977 - DGS-1909 Fix AVRO_FIELD_DEFAULT_FLAG_PROP in AvroData
  • PR-1974 - Add batch methods to update handler
  • PR-1972 - DGS-1902 Use fully qualified names for Protobuf Map types
  • PR-1969 - SEC-2492 exclude guava version
  • PR-1968 - DGS-1880 Allow additionalProperties to be set in converter
  • PR-1967 - Logging schema size when register
  • PR-1966 - DGS-1869 Clean up mock client to be similar to cached client
  • PR-1965 - Remove use of WrappedParsedSchema in mock client
  • PR-1963 - DGS-1878 Remove use of SynchronizedCache
  • PR-1962 - DGS-1872 Remove use of VerifiableProperties from serdes
  • PR-1961 - DGS-1869 Reduce synchronization in CachedSchemaRegistryClient
  • PR-1959 - DGS-1563 Minor API cleanup
  • PR-1956 - DGS-1853 Fix handling of slashes with refs in maven plugin
  • PR-1955 - DGS-1563 Add getGroupId method
  • PR-1954 - DGS-1850 Prevent duplicate imports in Protobuf converter
  • PR-1950 - DGS-1563 Add contexts resource
  • PR-1925 - DGS-1563 Add support for contexts
  • PR-1949 - DGS-1515 Remove ZK utils used by ZK leader election
  • PR-1947 - DGS-1829 Handle duplicate refs during Protobuf Schema compat check
  • PR-1944 - Build fix for SR related to kafka prepareResponse function change
  • PR-1943 - DGS-1826 Fix NPE when struct default has null
  • PR-1945 - DGS-1825 Fix DataException when scrubbing invalid names (Avro)
  • PR-1942 - DGS-1825 Fix DataException when scrubbing invalid names (Protobuf)
  • PR-1939 - DGS-1820 Pass ObjectMapper to JSON Schema generator
  • PR-1937 - DGS-1820 JSON Schema validation should take a JsonNode
  • PR-1936 - DGS-1515 Remove deprecated ZK for leader election
  • 910c9d9f - DGS-1794 Fix read only buffer in Protobuf converter
  • PR-1931 - DGS-1794 Don’t return read-only byte buffers from Protobuf converter
  • PR-1928 - DGS-1758: Allow AvroData take JSON null as default
  • PR-1923 - DGS-1737: MockSchemaRegistryClient doesn’t generate new schema id with protobuf
  • PR-1865 - DGS-1654 Upgrade Protobuf to 3.17.0; add optional support
  • PR-1922 - DGS-1752 Add rest of google common types for Protobuf
  • PR-1920 - DGS-404: Improving error messages - adding 40408 and 40409 for subject level compatibility and mode error
  • PR-1918 - small fix in deleteSchemaVersion for MockSchemaRegistryClient
  • PR-1917 - DGS-1734: MockSchemaRegistryClient need to perform a correct deleteSchemaVersion
  • PR-1907 - DGS-1658: Preserve field names when rendering Protobuf objects as JSON
  • PR-1901 - DGS-1726 Only convert wrappers to primitive if flag is set
  • PR-1902 - DGS-1727 Don’t throw exception when comparing array item
  • PR-1897 - DGS-1716 Remove logging of configs
  • PR-1893 - DGS-1607 Skip well-known Protobuf types on client side
  • PR-1896 - DGS-1704: override commons-compress
  • PR-1895 - DGS-1704: Upgrade jackson-mapper-asl
  • PR-1888 - DGS-1696 Pass previous value to LookupCache hooks
  • PR-1886 - DGS-1653: Add configs properties for SR Maven Plugin
  • PR-1858 - DGS-1622 Expose checkpoints to cacheInitialized method
  • PR-1856 - DGS-1607 Support rest of well known Protobuf types
  • PR-1848 - DGS-585 Add struct/empty Protobuf types
  • PR-1840 - DGS-585 Add Any/Duration logical types
  • PR-1838 - Fix documentation on the built package path
  • PR-1820 - DGS-1499 Add subject query param to schema REST APIs
  • PR-1827 - DGS-1522 Preserve sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, uint32, uint64 type in ProtobufConverter
  • PR-1830 - DGS-1528 refactor out SubjectValue class
  • PR-1828 - init was deprecated in MessageFormatter
  • PR-1823 - DGS-1513 Store offset and timestamp with cached values
  • PR-1818 - DGS-1484: throw InvalidConfigurationException to avoid NPE when schem
  • PR-1814 - Serializer now throws a user friendly message if not configured
  • PR-1813 - Fix NPE when checking compatibility
  • PR-1794 - Fix String.format in compatibility errorMessages
  • PR-1790 - DGS-1361 Add API to Test Compatibility against specified subject
  • PR-1789 - Change subject-level mode DELETE response type from String to ModeGet
  • PR-1780 - DGS-1328 Add DELETE API for subject config and subject mode
  • PR-1783 - Update to avro-data to support fixed type fields inside of a union
  • PR-1782 - DGS-1346 Add defaultToGlobal queryParam to GET /mode/{subject}
  • PR-1763 - Add logical type converter to Kafka avro serializer/deserializer (#1762)
  • PR-1323 - API update for Jetty 9.4.26.v20200117

Commercial Features

Confluent Server

All changes listed in the Kafka section also apply to Confluent Server

  • KGLOBAL-852: Send the correct consumer groups to the coordinator (#4718)
  • KGLOBAL-834: Prevent early cancellation of DelayedFetches which fetch from tiered storage
  • KGLOBAL-840: Fix bug with backward compatibility to old source clusters w/ consumer offset sync
  • KENGINE-33/KENGINE-51; lastCaughtUpTime is always reset by leader / Expansion should only consider caught-up followers
  • KSTORAGE-1834: followerRestorePoint is the Last Tiered Segment with offsets before localLogStartOffset
  • KGLOBAL-832: Fix local listener override for source initiated links, add multi-listener tests
  • KC-1906 Call maybeExpandIsr in all cases when handling AISR response
  • KGLOBAL-294: Reverse connection calls are stuck when source controlle
  • KGLOBAL-822: Fix bug for altering topic filters for cluster links
  • KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown (#11351)
  • KAFKALESS-655: Clear Executor state on exceptions encountered before proposals start executing
  • KC-1902: Disallow MDS with KRaft
  • KGLOBAL-742: LINK_FAILED replica status sholdn’t stop failover. (#4531)
  • CPKAFKA-7455: upgrade system test archive failures (#4528)
  • CC-13747 CIAM-1263 FF-4419: Limit max retries in MDS URL Refresh loop
  • CPKAFKA-7608: reassign license topic in unclean leader election syste
  • KSTORAGE-1802: remove linux-diskstats metric from CP 7.0
  • KGLOBAL-779: Avoid truncation in circular mirror test when recreating source topic
  • KGLOBAL-751: Improve error handling for source initiated link creation without dest link
  • KGLOBAL-750: Allow failed links to be reconfigured after password encoder secret update
  • KGLOBAL-741: Improve error message for invalid cluster link configuration
  • CIAM-1253 Fix Logging of various timeouts in ce-rest-auth RestClient
  • KGLOBAL-739: Notify cluster link fetcher thread on high watermark update to resume fetching
  • CONFLUENT: Remove force_use_zk_connection from topic command
  • SEC-1537 RestClient Retry Logic Fixed (#4395)
  • KC-1878 Debug Logging for LDAP Context creation
  • KAFKALESS-592: Revert rebalance storm prevention code to require full fixes.
  • KGLOBAL-686: Detect circular mirrors for older IBPs by tracking epoch update state
  • KGLOBAL-713: Use topic ids to detect source topic deletion and recreation to avoid circular mirror
  • KGLOBAL-720: Don’t fail LeaderAndIsr request for mirror partitions if link metadata not yet processed
  • KC-1800: Upgrade Guava to 30.0-android
  • KGLOBAL-719: Report link count metric only on controller
  • KSTORAGE-1713: Document the code asking user to ignore KIP-405 tiered storage configuration in CE-Kafka
  • KC-1850: Ensure topic ID upgrade does not overwrite pending isr state when upgrading to topic IDs (7.0.x)
  • KAFKALESS-528: Add shutdown support to multi-broker removal.
  • KGLOBAL-700: Add more TRACE level logging for periodic tasks
  • KAFKALESS-354: Update the Broker Removal State Machine to include all necessary phases
  • KAFKALESS-586: Do all removal ProgressListener callbacks in the PhaseExecutor
  • KC-1865: KRaft BrokerServer missing certain features/fixes
  • KC-1875: Do not enable both KRaft and CL in ducktape
  • KGLOBAL-692: Don’t truncate mirror topic for source epoch 0 when source topic is recreated
  • KC-1869; IllegalStateException in ControlMetadataBatch due to topicId mismatch
  • KC-1852: Upgrade commons-compress
  • KGLOBAL-478: Replace references to source connection origination with source initiated link
  • KAFKA-13209: Upgrade jetty-server
  • KAFKALESS-437: Return BrokerShutdownStatus based on request version in order to not break older clients
  • KAFKALESS-398: Add UNKNOWN status value to balancer operation statuses
  • KGLOBAL-696: Remove unnecessary purgatory when cluster linking is disabled
  • MINOR: Controlled shutdown is triggered twice when a Broker shuts down
  • KGLOBAL-695: Change purgatory to val from method to mitigate RCCA-3631
  • KGLOBAL-548: Rename topic mirror to mirror topic in codebase
  • KGLOBAL-689: Adding lag to kafka-mirrors –describe
  • KGLOBAL-682: Propagate SSL handshake failures during cluster link validation to clients
  • KGLOBAL-672: Process cluster link pause state during link creation and broker startup
  • KAFKALESS-588: Update system tests to remove multiple brokers at once
  • MMA-9932 Max timestamp changes from KIP-734
  • KAFKALESS-594: Extend BrokerShutdownManager to shut down multiple brokers
  • KGLOBAL-665: Fixed bug with auto mirror topic create and cluster link config updates
  • KAFKA-12155: Metadata log and snapshot cleaning #10864
  • KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-m
  • KGLOBAL-645: Don’t count non-persistent failures towards persistent failure timeout
  • KGLOBAL-663: Added RetriableException to return true for isRetriable to prevent excessive logging
  • KGLOBAL-657: describeMirrors() state should return STOPPED for promoted/failover mirrors irrespective of cluster link state.
  • AUDIT-494: Add configuration setting to include client addresses in audit logs (default false)
  • KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-mirrors cli.
  • KGLOBAL-658: Changed error message for auto-mirrored topic deletion
  • KSTORAGE-1644: Allow empty last segment to have missing offset index during recovery
  • KAFKALESS-402: Add replica exclusion system test
  • KGLOBAL-633: Update buffered bytes for reverse channels, fix memory leak
  • KAFKALESS-568: Rephrase broker failure detection message
  • KSTORAGE-1655: Broker-side envelope encryption for tiered data
  • KGLOBAL-491: Implementing Confluent specific listConsumerGroupOffsets API for CL
  • KGLOBAL-571: Add system test for auto mirror topic creation
  • KGLOBAL-620: Introduce new mirror topic state LINK_PAUSED to represent cluster link level pause.
  • KC-1843: Validate KIP-500 compatibility with confluent configurations
  • DGS-1903: Add config option for missing schemas TTL
  • KGLOBAL-606: Disallow addition of conflicting topic filters
  • KSTORAGE-1622: Update the documentation for Tier Object Store Compatibility Checker
  • KAFKALESS-538: Handle offline brokers for RemoveBroker request idempotency guarantees
  • SBC: Wait for Executor task to finish in broker removal
  • KGLOBAL-426: enable cluster linking in default server.properties file
  • KSTORAGE-1700: Increase the Future await timeout allotted to each ArchiveTaskTest.
  • KAFKALESS-478: Modify validation to handle multiple brokers
  • KGLOBAL-634: Allow mirror topic creation in kafka-mirrors without source-topic argument
  • Add a new thread pool to execute state machines
  • KGLOBAL-605: Prevent deletion of auto-mirrored topic
  • KAFKALESS-538: Make the broker removal API idempotent
  • PRSP-836 : Add broker configs to include list
  • KC-1817; Serialize principal details as JSON in request logs and add extra fields
  • KAFKA-13104: Controller should notify raft client when it resigns #11082
  • KC-1824: KC-1825: Infinite recursion in Partition with ZkIsrManager & Deadlock
  • KC-1810: Upgrade commons.io in ce-kafka
  • KGLOBAL-552: Populate describe mirrors response with last source fetch high water mark for stopped mirrors.
  • KGLOBAL-564: Enable cluster link fetcher destination lag flow control
  • KGLOBAL-569: Automatic mirroring
  • CONFLUENT: Make cluster link extensions in KafkaAdminClient package-private
  • KAFKA-13090: Improve kraft snapshot integration test
  • KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • KAFKA-12964: Collect and rename snapshot files prior to async deletion
  • KSTORAGE-1580: fix NPE when record==null in append
  • Safely log configs for IncrementalAlterConfigs error
  • KC-1789: asynchronously flush partition metadata file
  • KGLOBAL-601: Add more info for debugging cluster link lag
  • KAFKALESS-479: Modify BrokerRemovalStateTracker to track state for multiple brokers at once
  • KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • KAFKALESS-478: Add minimal multi broker removal validation
  • KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • KGLOBAL-603: Convert the Kafka Cluster Link command line option ‘–link-name’ to ‘–link’
  • KSTORAGE-1206 Enable system tests to be run with durability audit enabled
  • KAFKALESS-537: Update system tests to work wtih new broker removal.
  • KCFUN-84: Make sure Broker pick up the default config.
  • KSTORAGE-1653: add spurious offset out of range assertion to the tiered storage system tests
  • EVENTS-278: Update TelemetryReporter on how to load exporter
  • KGLOBAL-564: Throttle cluster linking fetch
  • KAFKALESS-540: Create topics for executor test before creating exclus
  • KAFKALESS-537: System test changes for exclusion-enabled broker remove.
  • KGLOBAL-440: Change mirror-topic-byte-rate to mirror-topic-byte-total
  • KAFKALESS-477: Shutdown-free single-broker removal with exclusion.
  • KAFKA-13007; KafkaAdminClient getListOffsetsCalls reuse cluster snaps
  • KGLOBAL-354: Make socket.send.buffer.bytes dynamically updatable on brokers
  • KAFKA-12520: Ensure log loading does not truncate producer state unless required
  • KSTORAGE-1674: fetch below log start offset may incorrectly scan forward
  • KAFKALESS-401: Prohibit replica placement on brokers that are excluded
  • KAFKA-12155: Metadata log and snapshot cleaning #10864
  • Upgrade protobuf to latest 3.17.3
  • KAFKALESS-523: Topic placement constraints not used in CreatePartitions
  • CNK-1420; Add broker tags to the DescribeCluster API
  • KPERF-3; Augment Fetch Session w/ Partitions’ metadata and process only active partitions
  • KAFKA-10587; Rename kafka-mirror-maker CLI command line arguments for KIP-629
  • KDATA-33: Add checksum validation of tier state file
  • KC-1567; Add support to disable delete retention for __consumer_offsets
  • KSTORAGE-1664: Force defaults for TierObjectMetadata entries except OpaqueData
  • KAFKALESS-447: Fix Property based test build configuration
  • KC-1778: partition metadata file flushed twice on partition creation
  • KGLOBAL-579: Fix mirror topic creation
  • KAFKALESS-436: Additional test for balancer startup failure
  • adding support for exporting messageSizeHistogram in Log.scala
  • KAFKALESS-466 Generate topic level properties for balancing
  • KC-1784: Use URI instead of URL for RBAC writer comparison to avoid hostname resolution issues
  • CNK-1600: Fix FetchRequestBenchmark.testFetchRequestForConsumer
  • KSTORAGE-1503: add metric for spurious OffsetOutOfRangeException
  • KPERF-103; Check shrink/expand status when AlterIsr returns
  • KAFKALESS-421: Update broker removal persistence records for Multi-Broker Removal
  • KCFUN-77: Add interface to retrieve TLV (type-length-value)
  • KSTORAGE-1450 Add metrics for unclean leader recovery
  • KAFKALESS-502: Interrupt in-progress SBC operations when new replica exclusions are detected
  • KDATA-26: Integrate TierMetadata with CheckedFileIO implementation
  • KENGINE-4; KAFKA-12890; Consumer group stuck in CompletingRebalance
  • KGLOBAL-533: Migrate mirror topic creation in CLI from kafka-topics to kafka-mirrors
  • KSTORAGE-1572 - Better Error Reporting for S3 Region Setting
  • KAFKALESS-441: Add Property based test for Replica Placement goal
  • KSTORAGE-1641: Fix LogLoader to pass materialized view of segments for deletion
  • KCFUN-37: Add interfaces and plumbing for PROXY protocol support
  • KAFKALESS-514: Update SBC to use Junit5
  • KAFKALESS-395: PBT same goal and cluster means same proposal
  • KSTORAGE-1624: Logs with holes in them may trigger spurious offset out of ranges
  • KAFKA-12701: NPE in MetadataRequest when using topic IDs
  • CIAM-971: Support user resourceId for RBAC authorization
  • KGLOBAL-529: Better handling of auth failures when stopping mirrors
  • CCLOG-403: Connector Data Preview Rest Extension
  • CCLOG-403: Connector Tracing Update delivery timeout.
  • KGLOBAL-527: Add ttl for old encryption secret, re-encrypt cluster link credentials during restart
  • KSTORAGE-1606: FileTierPartitionIterator incorrectly handles mixed size prefixed entries
  • Fix failing CL sytem tests from from KGLOBAL-532
  • KCFUN-74-2: Dynamically access averageServiceRequestTime config
  • KGLOBAL-532 Removed redundant mirror-action stop command
  • KC-1761: Controller Event Loop Perf Regression
  • KGLOBAL-440: Remove lkc from topic and link-name tags.
  • [KSTORAGE-1457] separate fetcher for in sync replicas
  • KGLOBAL-475: Add new error code for cluster link operations while CL is disabled.
  • KGLOBAL-445: Rename the mirror/replica state from LINK_NOT_AVAILABLE to SOURCE_UNAVAILABLE to make the error string more customer-friendly.
  • remove relocation for dependencies that no longer exist
  • KGLOBAL-440: Mirror topic lag
  • CPKAFKA-7255: Change SIMPLE_AUTH to ACL_AUTH
  • KC-1656: Metrics and improved tests for ConnectionMaxAge work
  • CONFLUENT: server_prop_overides was renamed to server_prop_overrides
  • CPKAFKA-6198: Fix broker removal system test parsing error
  • CONFLUENT: Upgrade protobuf and flatbuffers plugins
  • KSTORAGE-1609: Fix ConsumeBenchWorker exit behavior for maxMessage
  • KSTORAGE-1611: Allow retention settings in trogdor script and fix few other things
  • [WIP][CPKAFKA-3168] Augment ProduceResponse error messaging for specific culprit records as part of KIP-467
  • KSTORAGE-1609: Fix ConsumeBenchWorker exit behavior for maxMessage
  • KDATA-63: Postpone startoffset lapse under heavy user delete event, for a window of time for self recovery
  • KSTORAGE-1569: Incorporate LogLoader and recovery logic refactor from AK
  • CONFLUENT: rename all _confluent-license usages to _confluent-command
  • KC-1647: Include confluent. prefix for close.connections.on.credential.delete config
  • CNK-941: Connection rate auto-tuning
  • KAFKALESS-444, -445 Added NetworkCapacityGoals PBT Tests
  • KC-1764: Ensure dynamic logins are initialized before broker plugins are started
  • CONFLUENT: Handle TopicAuthorization while checking “_confluent-license” topic existence
  • CCLOG-403: Add support for data tracing in connect [Connector Data Previews]
  • KAFKALESS-412: Do not run SBC operations when there are broker replica exclusions present (part 1/2)
  • CONFLUENT: Fix downstream build failures after recent LicenseStore changes
  • KDATA-14: Add healthcheck for Durability Audit
  • KDATA-15 : DA to be more tolerant for missing registration and retention change event.
  • KAFKA-12791: ConcurrentModificationException in AbstractConfig use by KafkaProducer (#10704)
  • KAFKA-12856: Upgrade Jackson to 2.12.3 (#10778)
  • KC-1266: Update LicenseManager to support both _confluent-command and _confluent-license topics
  • KPERF-7; Add listener mechanism in the Log and Partition layers
  • Connection metrics for Broker Load measurement
  • KAFKALESS-442: Add ReplicaCapacityGoal PBT
  • KAFKALESS-443: Add DiskCapcityGoal PBT
  • METRICS-3264 Make TELEMETRY_PROXY_PASSWORD from STRING type to PASSWORD
  • KAFKALESS-415: Keep SBC AdminClient open through full shutdown.
  • KCFUN-7: Make sure handshake time is measured upon failure.
  • KSTORAGE-1478: topic recreated after deletion due to LISR request
  • CPKAFKA-3264: TierTopicManager thread dies materializing CATCHUP TierPartitionStates
  • KC-1656: Add a unit test for connection max age updates and handling.
  • KSTORAGE-1596: Fix topic configuration and params in trogdor scripts
  • CONFLUENT: Add comment to Partition.topicId() that was lost during the merge
  • KC-1332: reduce queue memory usage in producer id maintainence
  • CONFLUENT: Improve ConfluentMetricsFormatter.getString when default is null
  • KDATA-60: Materialize delete partition
  • KSTORAGE-1594: Use segment.bytes topic config instead of confluent.tier.segment.hotset.roll.min.bytes
  • KGLOBAL-487: Make new fields in ListClusterLinksResponse ignorable for compatibility
  • KSTORAGE-1593: Fix exception stacktrace printing in JmxMetricsScraper
  • KGLOBAL-510: Fix testDestLeaderChanges intermittent failure
  • KDATA-21 : Integration test gaps DA.
  • KAFKALESS-384: Add property based test for RackAwareGoal
  • KAFKALESS-378, KAFKALESS-393: Add invariants for PBT
  • KAFKALESS-372: Implement replica exclusion handling and persistence
  • SEC-298 - Log errors when MDS can not start
  • KC-1646: Dynamic SASL/PLAIN login for Kafka clients in broker plugins
  • KDATA-24: CheckedFileIO package for providing checksum backed File IO
  • KSTORAGE-1397: Fix test for Duplicate Registration with TierTopicConsumer
  • METRICS-2878: add event emitter api
  • KGLOBAL-487: Add version check for available field for version < 2
  • KSTORAGE-1528: Remove flakey tests for TierObjectStoreFactory
  • KSTORAGE-1268: Filtering ability for durability db dumping tool
  • KAFKALESS-369: Introduce kafka-replica-exclusions.sh CLI
  • CNK-1373: Delay external connectors until both metadata and acceptors load
  • KSTORAGE-1569: Incorporate changes from AK leading up to recovery logic refactor
  • CONFLUENT: Re-enable testClientSideTimeoutAfterFailureToSend
  • KSTORAGE-1569: Introduce LogSegments class abstracting the segments map
  • KSTORAGE-1544: Convert confluent.tier.max.partition.fetch.bytes.override into a dynamic broker config
  • KSTORAGE-1487: support Tiered Storage TLS for S3/PureStorage via configuration properties
  • KSTORAGE-292: fsync the parent of FileTierPartitionState files
  • KSTORAGE-1397: Prevent race on maybeBeginTierMaterialization
  • KGLOBAL-114: Removing controller metrics to consolidate mirror topic metrics
  • KC-1610: Upgrade dataformat-cbor version (#3589)
  • CONFLUENT: Fix RbacRolesTest
  • KDATA-6: Prevent logging of AuditJob’s skip message for compacted partitions
  • MINOR: Adds a check before reading empty partitionMetadataFile to avoid NPE
  • CONFLUENT: Don’t publish tier-object-store-compatibility-checker
  • KAFKALESS-369: Admin API changes for {Alter,Describe}BrokerReplicaExclusions
  • CONFLUENT: Adjusting get_offset_shell parameters in reassign_partitions_test.py
  • CONFLUENT: Pass correct requestHandlerAvgIdleMetricName
  • CNK-1567: Expire old connections unconditionally (at SocketServer level)
  • KAFKALESS-358: Update remove broker cli to support multi broker removal with optional –no-shutdown flag
  • CONFLUENT: Remove stray artifacts block
  • KAFKALESS-356: Update removeBrokers AdminApi with shutdownCompleted flag and BrokerReplicaExclusion status
  • Disable BrokerLoad for KRaft servers
  • Guard against null client ID in request size calculation
  • KC-580: remove epoch cache flush when recovery point is not advanced without flushing
  • QEC-5889: Allow SBC commands to run with custom security config (#3531)
  • KC-1624: fix perf regression due to removal of BufferSupplier (#3555)
  • KC-1610: Upgrade dataformat-cbor version
  • KC-1618: Adapt the KRaft README and example configs for CP
  • Replaced CL source soak with new one
  • KSTORAGE-1567 Fix errorStateReachedViaFenceEvent flag for log start offset recovery
  • KAFKALESS-369: Add DescribeBrokerReplicaExclusions RPC (part 2/3)
  • KAFKALESS-369: Add AlerBrokerReplicaExclusions RPC (part 1/3)
  • CNK-1685: Add configs for broker-wide limit on produce and consume bandwidth
  • KSTORAGE-1568: object durability audits must use key baseOffset not virtualBaseOffset
  • KC-1624: fix perf regression due to removal of BufferSupplier
  • KC-1614: Update BrokerFailureDetector to use executeRemoval
  • QEC-5889: Allow Balancer commands to run with custom security config
  • KDATA-10: Send audit request for delete record request before start offset request
  • KGLOBAL-411: Don’t update destination high watermark when processing fetch response
  • KPERF-2; Leader resets LastCaughtUpTimeMs of caught-up replicas when it appends to its log
  • CNK-1695: Log cause of unfixable hard goals in goal violation self-healing
  • ST-5293: Upgrading version of netty
  • CONFLUENT: Fix javadoc on KafkaBasedLog constructor
  • KGLOBAL-414: Reset AlterIsr in-flight state for duplicate update requests
  • KSTORAGE-1455: Bump object store dependencies to latest
  • KSTORAGE-1525: Track inbound throughput of compacted partitions
  • CONFLUENT: Change visibility of LicenseConfig constants to public
  • CIAM-635 Add reason argument to AuthWriter
  • CONFLUENT: stopPartitions should call clusterLinkingManager.removePartitionsAndMetadata
  • CONFLUENT: TierMessageFormatter should inherit from o.a.k.c.MessageFormatter
  • CONFLUENT: Pass no channelManager in AutoTopicCreationManagerTest.testErrorWithCreationInZk
  • CONFLUENT: Fix admin test failures due to bug in ConfigHelper.filterTopicConfigs
  • CONFLUENT: Fix failing testGenerateAssignmentWithInvalidPartitionsFails
  • CONFLUENT: Close backend in TierMetadataValidatorTest
  • CONFLUENT: Remove PartitionTest.testDelayedFetchAfterAppendRecords
  • Handle node.id in metrics context and null auditLogProvider for KRaft
  • CONFLUENT: Compare filtered configs instead of config counts in PlaintextAdminIntegrationTest
  • CONFLUENT: Convert UUID usage into Uuid in Log and MergedLog
  • CONFLUENT: Remove some of the default arguments from the Log/MergedLog constructors and apply methods
  • CONFLUENT: Add listeners to all of the CE requests
  • KSTORAGE-1511: Remove the delay of 1ms when scheduling flush in Log.roll()
  • KSTORAGE-1545: Add test verifying that the TierReplicaManager deletion hook is called
  • CIAM-827: Create Roles.json entry for Data Plane DG Role(s)
  • [KSTORAGE-1474] set unclean.leader.election.enable to false for tier state topic (#3429)
  • KSTORAGE-1545: Partition deletion does not stop leadership of tier tasks
  • KSTORAGE-1379: Durability Audit Update commit offset after alert
  • KAFKALESS-400: Handle non-PLAINTEXT setups for SBC commands (#3488)
  • KAFKALESS-409: Handle License topic MinISR of 2 in SBC tests
  • KAFKALESS-400: Handle non-PLAINTEXT setups for SBC commands
  • KAFKALESS-409: Handle License topic MinISR of 2 in SBC tests
  • CIAM-809 : Create new role(s) for SA issue
  • KAFKALESS-365: Validate cluster configuration before creating cluster object
  • KGLOBAL-347: Make fetcher thread count dynamically configurable
  • Start licesne validator after unfencing KRaft broker
  • CONFLUENT: Fix all of the tests in LogManagerTest
  • Handle node.id in metrics context and null auditLogProvider for KRaft
  • KAFKALESS-387: Respect broker concurrency limit even for “dataless” m
  • Fixed BrokerEpochIntegrationTest, ensure compatability for error, and minor style fix
  • KAFKA-12381: remove live broker checks for forwarding topic creation (#10240)
  • KC-1580 Remove clearPending method from AlterIsrManager
  • KAFKALESS-387: Respect broker concurrency limit even for “dataless” moves
  • KAFKALESS-367: Add first PBT test for cluster model
  • CONFLUENT: Manually apply upstream revert
  • KAFKA-12600: Remove deprecated config value default for client config client.dns.lookup
  • KAFKALESS-366: Add support for property based testing in SBC
  • CCLOG-503 Enabling partitioning key while produce events from KafkaExporter
  • CONFLUENT: Add listeners to all of the CE requests
  • [KSTORAGE-1474] set unclean.leader.election.enable to false for tier state topic
  • KCFUN-5: Fix NPE thrown by clusterId
  • KSTORAGE-1494: Prevent deletion of empty compacted segments
  • KSTORAGE-1497 Tool to inject TierPartitionUnfreezeLogStartOffset event
  • KGLOBAL-282: Ensure mirrors progress after non-fatal errors when issues are fixed
  • CCLOG-353: Emitting of ConnectLogEvents when connector/task fails
  • KSTORAGE-1304 - Restore Log Start Offset Feature Implementation
  • KAFKALESS-342: Retry topic description failures in the Executor
  • DGS-1350: Reduce schema validation logging output
  • KAFKALESS-348: Fix previous status null case returned from EvenClusterLoadStateRecord
  • KSTORAGE-1476: perf regression in log load caused by incorrect use of containsKey
  • KSTORAGE-1471 Fix fallback bootstrap server (#3405)
  • CL-14: Failed cluster link doesn’t doesn’t reflect in mirror topic state
  • KSTORAGE-1471 Fix fallback bootstrap server
  • KSTORAGE-1442: refactor TierPartitionState data structures to enable future compaction support
  • SBC-335: Handle KafkaSampleStore loading that fails.
  • SBC-309: Update EvenClusterLoad status in persistenceStore and return from it
  • SBC-306: Retry on InvalidReplicaAssignmentException
  • KGLOBAL-325: Fixed bug where sync acls task could get stuck
  • KAFKA-12583: Upgrade netty to 4.1.62.Final
  • KAFKA-12400: Upgrade jetty
  • KAFKA-12334: Add the KIP-500 metadata shell
  • KAFKA-12297: Make MockProducer return RecordMetadata with values as per contract

Security

  • Fix various test failures
  • APPSEC-287: Override commons-codec
  • DGS-2010 Add support for contexts in authorizers
  • Update KsqlRestAuthorizer with new checkPrivileges() method
  • Fix signature of ConnectorsResource.restartConnector
  • DGS-1903: Add missing schema cache to LruSchemaRegistryClient
  • SEC-2533 upgrade commons-io to 2.11.0
  • SEC-2493 Remove bouncy-castle explicit dependency
  • DGS-1563 Add contexts resource
  • DGS-1563 Add support for contexts
  • Fix SR ACL tests due to ZK removal
  • FF-4684 mTLS support, second attempt
  • DGS-1691: Perform license checks in background thread
  • FF-4684 Enable Schema Registry RBAC authorization with mTLS authentication (reopen PR346)
  • KQE-229: Update vertx-core to 3.9.7
  • Replace usage of deprecated SimpleAclAuthorizer with AclAuthorizer
  • Pins Ksql to a jetty-jaas 9.4.28.v20200408
  • DGS-1350: Reduce schema validation logging output
  • DGS-1499 Add subject param to APIs
  • SEC-2023: TestUtils method was renamed upstream
  • DGS-908: Schema validation metrics api integration
  • DGS-1351: Add more schema validation metrics
  • DGS-1328 Add authorizations for DELETE of Config and Mode endpoints

Secret Registry

  • MINOR: Remove usage of WorkerConfig.REST_PORT_CONFIG config
  • CC-15204: Upgrade bcprov-jdk15on to 1.68
  • Remove deprecated sasl enabled mechanisms usage
  • Remove usages of ClientDnsLookup.DEFAULT

Control Center

  • MMA-11113: Feature flag changes for new C3 broker settings
  • PG-389: short-lived http connection for non-idempotent reqs and configurable mds client
  • MMA-10607: fix testBadClientDoesNotStallRegistration by awaiting method invocation before verify
  • KGLOBAL-684: Change naming convention to match upstream
  • fix build failure due to StreamsConfig change
  • MMA-7623: hide stack trace in error response (#2178)
  • Use confluent repackaged log4j
  • MMA-10895: add c3 property prefix to auth.bearer.roles.claim
  • Upgrade protobuf from 3.17.0 to 3.17.3
  • PG-205: Make bin scripts compatible with all c3 modes
  • fix protobuf version in accordance to DGS-1654
  • replace default jar with fat hjar
  • refactor-injector: centralize creation of Guice injectors
  • add new feature flags to enable/disable embedded rest proxy
  • More deprecated APIs to remove
  • Remove TestDriver Deprecated API
  • MMA-10673: Introduce a new SSL-specific SMTP port configuration for email alerts
  • Upgrade Armeria to match the netty version
  • PG-182: Create MMO version of ControlCenter
  • PG-178: Create MMO version of HealthCheck
  • Replace usages of methods/classes deprecated for removal in Apache Kafka 3.0 (part 2)
  • Replace usage of kafka.common.MessageFormatter with o.a.k.c.MessageFormatter
  • Use KafkaStreams.store(StoreQueryParameters) instead of the overload removed in Apache Kafka 3.0
  • PG-190: Create MMO version of KafkaHelper and AlertsResource
  • PG-198: Filter available resources based on running mode
  • PG-181: Create Mode annotation and update C3 application
  • Add websocket parameter consumerPollDelay
  • PG-138: Add new endpoint in AlertsResource to return available triggers
  • PG-124: Refactor ControlCenterApplication to use reflection
  • Add control-center mode to feature flags
  • PG-109: add confluent.controlcenter.mode.enable to change c3 mode
  • give unique path to v3 web-socket, implement ControlCenterKafkaRestContext.getProducer
  • Replace usage of deprecated ConsumerRecord/RecordMetadata.checksum()
  • Fix KafkaResourceTest due to breaking change in ce-kafka
  • Disable audit logging to fix build
  • add C3Stream.split to fix build
  • MMA-10034: add cluster access check and add refresh logic
  • CL-130: Fixed imports after changing packages in ce-kafka
  • MMA-9949: Integrate KafkaRestApplication in C3
  • MMA-9967: loosen restrictions for customized topics’ data export
  • MMA-9971: remove “config” flag as part of usage prompt

MQTT Proxy

  • Added telemetry-events-api as compile dependency.
  • Fix SocketChannel mock in PipelineFactoryTest
  • CC-15588: Added confluent-serializers-new as explicit dependency.
  • DGS-2161: Inherit netty.version from confluentinc/common
  • CCMSG-1251: log4j migration to confluent repackaged version
  • CCMSG-1101: Upgrade Netty to 4.1.63.Final
  • CC-14445: Remove compile dependency on Kafka broker
  • Replace usage of deprecated SaslConfigs.SASL_ENABLED_MECHANISMS

Metadata Service

  • SEC-2666: Fix test breakage caused by Jackson 2.12 CoercionConfig
  • Override commons-codec to 1.13 in 5.3 Only
  • SEC-2440 - MDS - Fix MdsScope Validation and messages and add tests
  • add negative tests for testing changes made in lookupPrincipalsWithRoleOnResource
  • add positive tests for testing changes made in lookupPrincipalsWithRoleOnResource
  • correct spelling
  • Add comment to explain try-catch block
  • Add try catch to first authorize on SECURITY_METADATA
  • Fix lookupPrincipalsWithRoleOnResource to authorize on valid ResourcePattern using DESCRIBE_ACCESS operation
  • CIAM-989 Move Data Governance roles to own namespace (#605)
  • SEC-1034: log4j migration to confluent repackaged version
  • CIAM-1060: Fixed CPD crashed issue due to wrong credentials to DB and Events Kafka
  • CIAM-1048 - DB Migrate Extractor State table to stag, prod, cpd
  • CIAM-1050: Improve checking in HttpRefreshRoleBindingTest
  • CIAM-725: Extractor Full Org Refresh via Http call
  • CIAM-347 Prevent OrgAdmin from deleting their own OrgAdmin role
  • Update test to check for Kafka HTTP Server.
  • CIAM-1017 Allow multiple policies at a binding scope
  • CIAM-999 Use LaunchDarkly to control whether to publish updated role
  • Fixed an accidental change in halyard config
  • rbac-extraxtor deployment fix
  • CIAM-726 Integration test of Dataplane enforcement
  • CIAM-990 Escape input in healthcheck query
  • CIAM-710 Sync rolebindings periodically
  • Fix NPE in Scope Comparator
  • CIAM-709: Build Extractor Rest Foundation
  • CIAM-709: Fixed halyard chart bug
  • Adding stage/prod semaphore deployment reference
  • CIAM-709: Added more configurations for rbac-extractor
  • CIAM-696:Register the new version for cpd
  • rbac-api-server should have explicit dependency on confluent-licensing-new
  • Add log4j package in rbac-extractor pom file
  • CIAM-946 Migrate Stag to include reason field in role_binding
  • Add jakarta rs api explicit dependency
  • Add explicit guava dependency to rbac-common
  • CIAM-946 Migrate Stag to include reason field in role_binding
  • CIAM-693: Adding Sample Config to rbac-extractor
  • CIAM-706 Implement RBAC Writes Infrastructure.
  • SEC-2393 Remove reliance on KafkaPrincipal.fromString()
  • CIAM-945 Remove cc_rbac_api user via dbmigrate
  • CIAM-707 Recently Updated Role Bindings query
  • CIAM-703 JDBC Query Infrastructure
  • CIAM-680: Updated Image.repository in halyard.mk
  • CIAM-680: Fix in helm/halyard to handle mds specific version
  • CIAM-680:Dummy check-in to re-initiate build
  • CIAM-680:Updated for halyard auto-deploy
  • CIAM-941–Fixed master build error.
  • CIAM-941 – Fixed double build docker image which run mvn install twice.
  • Increase DB Test query timeouts (3 -> 10 seconds)
  • CIAM-914 Migrate prod to have two rbac_api users
  • CIAM-894 Authorizer test for new DG RBAC dataplane roles
  • CIAM-680: Adding cc-semaphore.mk to makefile
  • CIAM-914 Migrate stag to have two rbac_api users
  • CIAM-680: Added cc-cpd.mk for helm version update
  • Use postgres 12.5
  • CIAM-680 Support for halyard/helm version update and deployment
  • CIAM-702: Add schema for ResourcesRequest to Swagger Spec
  • CIAM-920 — Fixed version and labels docker image
  • CIAM-324 DB Migration to Support multiple db users for password rotation
  • CIAM-863: Updating directory for halyard set-version
  • removed chart.target from the Makefile
  • CIAM-863: Added helm charts and halyard spec files
  • CIAM-679 Build rbac-extractor docker image
  • CIAM-892 Allow multiple namespaces to be specified in /roles and /rol
  • CIAM-635 : Add endpoints to allow for Organization Merges
  • CIAM-843 Address Integration Test Gaps for V2 Resources
  • Increase DB Test query timeouts
  • CIAM-809: SA Issue : Create new role(s) for SA issue
  • CIAM-675 Create Pseudo-Extractor shaded jar
  • CIAM-240 - Create v2 tests for Authorize to parity Topic/DB Framework
  • CIAM-765 Refactor/Rename Validation Classes common for V1 and V2 API
  • CIAM-611: Add smoke test for use in ce-kafka-images build
  • CIAM-354: CLI : Misspelled Role Name does not throw error for rolebinding list
  • CIAM-457: Move healthcheck endpoint to v2alpha version
  • CIAM-659 - Add implementations for new interface methods added upstream

Replicator

  • REPL-1810: Removing ZK dependencies from Replicator Verifier
  • REPL-1774: Increase default value of offset.translator.batch.size
  • REPL-1655: Removed logging of properties that could expose secrets

Confluent Platform Docker Images

Confluent Platform Docker Images will no longer ship with Yum/Apt configuration that allowed users to update running Confluent Platform containers to the next Confluent Platform release. Confluent recommends upgrading Confluent Platform service components running in containers by replacing the running container with the next Confluent Platform released docker image and using docker volumes to persist data between upgrades.

Confluent Platform Documentation

  • DOCS-10709: Update ksqlDB links to use versioned role in Overview topic
  • DOCS-10671 - Add missing release notes
  • DOCS-10677: Update metadata for the Health+ overview topic
  • DOCS-10117 - Release notes for 7.0.0
  • DOCS- 10612 - Update System Requirements page for Control Center
  • DOCS-10176: Update steps in CP quick start for downloading tar file
  • Docs 9478 - add anchor to disable metrics reporter
  • DOCS-9329: Update monitoring topics with autogenerated metrics for CP 7.0 (and DOCS-10602)
  • Docs 9149 - Management-only mode changes and docs refresh
  • DOCS-9333: Update the CP ksqlDB Upgrade Guide for 7.0
  • Generate Changelog for 7.0.0
  • DOCS-9326: Update the Streams Upgrade Guide for CP 7.0
  • DOCS-10116: Update the CP upgrade guide for 7.0.x (and DOCS-8752)
  • DOCS-9344: Fix metadata description in ksqlDB tutorials index
  • DOCS-9344: Update CP ksqlDB resources for CP 7.0
  • DOCS-8936: Add RBAC role bindings to Kafka Streams Security topic
  • DOCS-10428: Move CP Quick Start topic up one level in the TOC
  • ST-6458: Add CP8.0 Deprecation Notice
  • DOCS-9302: Update Health+ content for CP 7.0
  • DOCS-10232: Add new Telemetry Reporter metrics (and DOCS-9330)
  • DOCS-9500 7.0.x compatibility docs
  • DOCS-10221: Retire legacy ksqlDB tutorials
  • DOCS-9334: Update CP ksqlDB compatibility table for CP 7.0
  • DOCS-9334: Update CP ksqlDB Supported Versions table for CP 7.0
  • DOCS-9356: Bump ksqlDB version to 0.21.0
  • ST-5633: Add instructions for Confluent Clients repo
  • DOCS-9993: Remove legacy quickstart/includes directory
  • DOCS-7844- Refactor the CP quick start
  • KGLOBAL-604 Update CP 7.0 cluster linking docs commands: change –link-name to –link
  • DOCS-9139 Streamline DOCS README, move Sphinx/RST info to wiki
  • DOCS-9065: Copy new PAPI content from AK docs
  • DOCS-7416 Add compatibility API endpoint for Schema Registry new in CP 7.0