Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 7.4.0¶

Released May 2023

Community Features¶

Common¶

22a9da57 - APPSEC-2551: Update Prometheus JMX Exporter
5ad03d12 - APPSEC-2551: Update Jackson
PR-514 - Update snakeyaml
PR-512 - Add snakeyaml to dependency management
7bcaab9e - Fix typo in with method calls
3868e27b - Fix typo
783c5721 - Return the original “version_range” if it isn’t an actual Maven version range
e71293e4 - Update logredactor depdendency to 1.0.11
PR-502 - Fix: override Maven version plugin’s default versionrange
9350d231 - Update resolver-maven-plugin to 0.6.0
PR-497 - Upgrade Netty to 4.1.86.Final
PR-496 - Bump to 7.4.4
PR-492 - APIF-3112: Upgrade jmx_prometheus_javaagent to 0.17.2.
PR-489 - APIF-3117: Upgrade Scala to 2.13.10.
PR-485 - APIF-2957: Upgrade Jackson and Protobuf versions.

Kafka¶

PR-13592 - KAFKA-14862: Outer stream-stream join does not output all results with multiple input partitions (#13592)
PR-13575 - KAFKA-14905: Reduce flakiness in MM2 ForwardingAdmin test due to admin timeouts (#13575)
PR-13604 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13604)
03b41b54 - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
PR-13511 - KAFKA-14869: Ignore unknown record types for coordinators (KIP-915, Part-1) (#13511)
PR-13534 - KAFKA-14054: Handle TimeoutException gracefully (#13534)
PR-13541 - KAFKA-14894: MetadataLoader must call finishSnapshot after loading a snapshot (#13541)
PR-13462 - KAFKA-14857: Fix some MetadataLoader bugs (#13462)
PR-13499 - KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable (#13499)
PR-13369 - KAFKA-14172: Should clear cache when active recycled from standby (#13369)
PR-13470 - KAFKA-14864: Close iterator in KStream windowed aggregation emit on window close (#13470)
PR-13472 - KAFKA-14774 the removed listeners should not be reconfigurable (#13472)
PR-13445 - KAFKA-14843: Include Connect framework properties when retrieving connector config definitions (#13445)
PR-13148 - KAFKA-14645: Use plugin classloader when retrieving connector plugin config definitions (#13148)
PR-13367 - KAFKA-14797: Emit offset sync when offset translation lag would exceed max.offset.lag (#13367)
PR-13415 - KAFKA-14816: Only load SSL properties when issuing cross-worker requests to HTTPS URLs (#13415)
PR-13386 - KAFKA-14809 Fix logging conditional on WorkerSourceTask (#13386)
PR-13379 - KAFKA-14799: Ignore source task requests to abort empty transactions (#13379)
PR-13351 - KAFKA-14781: Downgrade MM2 log message severity when no ACL authorizer is configured on source broker (#13351)
PR-13182 - KAFKA-14649: Isolate failures during plugin path scanning to single plugin classes (#13182)
PR-13193 - KAFKA-14659 source-record-write-[rate|total] metrics should exclude filtered records (#13193)
PR-13279 - KAFKA-14295 FetchMessageConversionsPerSec meter not recorded (#13279)
PR-13297 - Kafka-14743: update request metrics after callback (#13297)
PR-13052 - KAFKA-14545: Make MirrorCheckpointTask.checkpoint handle null OffsetAndMetadata gracefully (#13052)
PR-13181 - KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method (#13181)
PR-11818 - KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (#11818)
PR-13161 - KAFKA-14128: Kafka Streams does not handle TimeoutException (#13161)
PR-13282 - KAKFA-14733: Added a few missing checks for Kraft Authorizer and updated AclAuthorizerTest to run tests for both zk and kraft (#13282)
PR-13273 - KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
PR-13262 - KAFKA-14727: Enable periodic offset commits for EOS source tasks (#13262)
PR-13168 - Kafka 14565: On failure, close AutoCloseable objects instantiated and configured by AbstractConfig (#13168)
PR-13208 - KAFKA-5756: Wait for concurrent source task offset flush to complete before starting next flush (#13208)
PR-13227 - KAFKA-14693; Kafka node should halt instead of exit (#13227)
PR-13230 - KAFKA-14704; Follower should truncate before incrementing high watermark (#13230)
PR-13241 - KAFKA-14711: kafaka-metadata-quorum.sh does not honor –command-confi… (#13241)
PR-13211 - KAFKA-14676: Include all SASL configs in login cache key to ensure clients in a JVM can use different OAuth configs (#13211)
PR-12984 - KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic (#12984)
PR-13175 - KAFAK-14660: Fix divide-by-zero vulnerability (#13175)
PR-13107 - KAFKA-13972; Ensure replica state deleted after reassignment cancellation (#13107)
2e1947d2 - Bump version to 3.4.0
PR-880 - KC-2332: Upgrade Netty to 4.1.86
PR-13159 - KAFKA-14656: Send UMR first during ZooKeeper migration (#13159)
PR-13166 - KAFKA-14623: OAuth’s HttpAccessTokenRetriever potentially leaks secrets in logging (#13119) (#13166)
PR-13156 - KAFKA-14533: re-enable ‘false’ and disable the ‘true’ parameter of SmokeTestDriverIntegrationTest (#13156)
PR-13147 - KAFKA-14533: temporarily disable the ‘false’ parameter of SmokeTestDriverIntegrationTest (#13147)
PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
PR-13130 - KAFKA-14637: Fix upgrade compatibility issue from older versions to 3.4 (#13130)
PR-13117 - KAFKA-14621: Disallow authorizers during ZooKeeper migration (#13117)
PR-13108 - KAFKA-14618: Fix off by one error in snapshot id (#13108)
PR-13106 - KAFKA-13709: (follow-up): Avoid mention of ‘exactly-once delivery’ or ‘delivery guarantees’ in Connect (#13106)
PR-13104 - KAFKA-14612: Make sure to write a new topics ConfigRecords to metadata log if the topic is created (#13104)
PR-13103 - KAFKA-14304: Use boolean for ZooKeeper migrating brokers in RPC/record (#13103)
PR-13058 - KAFKA-14557: Lock metadata log dir (#13058)
PR-13077 - KAFKA-14279: Add 3.3.x streams system tests (#13077)
PR-12998 - KAFKA-14493: Introduce Zookeeper to KRaft migration state machine STUBs in KRaft controller. (#12998)
PR-13083 - 2023 (#13083)
347042cc - KAFKA-14458: Introduce RPC support during ZK migration #13028
PR-13001 - KAFKA-14446: code style improvements for broker-to-controller forwarding (#13001)
PR-13020 - KAFKA-14529: Use the MetadataVersion from ClusterConfig in ZooKeeper tests (#13020)
PR-13076 - KAFKA-14279: Add 3.3.x to core compatibility tests (#13076)
PR-13073 - KAFKA-14571: Include rack info in ZkMetadataCache.getClusterMetadata (#13073)
PR-13061 - [MINOR] Update upgrade documentation for 3.4 (#13061)
PR-13016 - KAFKA-14498: reduce the startup nodes to avoid timeout error (#13016)
PR-12008 - KAFKA-13439: move upgrade note to stream upgrade doc (#12008)
PR-12937 - KAFKA-13881: Add Connect package infos (#12937)
PR-12613 - MINOR: Fix punctuation marks (#12613)
PR-12936 - KAFKA-13881: Add Streams package infos (#12936)
PR-13023 - KAFKA-14532: Correctly handle failed fetch when partitions unassigned (#13023)
PR-13019 - KAFKA-14531: Fix controller snapshot interval (#13019)
PR-12994 - KAFKA-14457: Controller metrics should only expose committed data (#12994)
PR-12968 - KAFKA-14417: Address incompatible error code returned by broker from InitProducerId (#12968)
PR-12961 - KAFKA-14446: API forwarding support from zkBrokers to the Controller (#12961)
PR-12965 - KAFKA-14448: Let ZooKeeper brokers register with KRaft controller (#12965)
PR-12856 - KAFKA-14392: Fix overly long request timeouts in BrokerToControllerChannelManager (#12856)
PR-12946 - KAFKA-14427: ZooKeeper client support for migrations (#12946)
PR-13000 - KAFKA-14496: Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler (#13000)
PR-12977 - Removing Multicasting partitioner for IQ (#12977)
PR-12985 - KAFKA-13602: Remove unwanted logging in RecordCollectorImpl.java (#12985)
PR-12956 - KAFKA-14379: Consumer should refresh preferred read replica on update metadata (#12956)
PR-12971 - KAFKA-14454: Making unique StreamsConfig for tests (#12971)
PR-845 - CONFLUENT: Skip publishing for projects with no Scala suffix when the Scala version is not the default
PR-842 - DP-9030: Use the new withGradleFile closure
56f7a277 - KAFKA-14435: Fix allow.everyone.if.no.acl.found config behavior for StandardAuthorizer
PR-12935 - KAFKA-14432: RocksDBStore relies on finalizers to not leak memory (#12935)
PR-12955 - KAFKA-14443: Close topic creation Admin clients in MM2 connectors (#12955)
PR-12954 - KAFKA-14352: Rack-aware consumer partition assignment protocol changes (KIP-881) (#12954)
PR-12893 - KAFKA-14260: Add synchronized to prefixScan method (#12893)
PR-12903 - KAFKA-14415: Faster ThreadCache (#12903)
PR-12892 - KAFKA-14386: Return TopicAssignment from the ReplicaPlacer (#12892)
PR-12915 - KAFKA-14417: Producer doesn’t handle REQUEST_TIMED_OUT for InitProducerIdRequest, treats as fatal error (#12915)
PR-12800 - KAFKA-14342: Clear offsets for connector source partitions on tombstone messages (#12800)
PR-12803 - KAFKA-13602: Adding ability to multicast records (#12803)
PR-12846 - KAFKA-14293: Basic Auth filter should set the SecurityContext after a successful login (#12846)
PR-12850 - KAFKA-14367: Add LeaveGroup to the new GroupCoordinator interface (#12850)
PR-12675 - KAFKA-14256: Upgrade from Scala 2.13.8 to 2.13.10 (#12675)
PR-12928 - KAFKA-14304: Add RPC changes, records, and config from KIP-866 (#12928)
PR-12847 - KAFKA-14367: Add SyncGroup to the new GroupCoordinator interface (#12847)
PR-12175 - KAFKA-14146: Config file option for MessageReader/MessageFormatter in ConsoleProducer/ConsoleConsumer (KIP-840) (#12175)
PR-12896 - KAFKA-14398: Update EndToEndAuthorizationTest to test both ZK and KRAFT quorum servers (#12896)
PR-12942 - KAFKA-14433: Clear Yammer metrics in QuorumTestHarness#tearDown (#12942)
PR-12898 - KAFKA-14430: Specify JMX RMI port system property when not already set (#12898)
PR-12885 - KAFKA-14358: Disallow creation of cluster metadata topic (#12885)
PR-12848 - KAFKA-14367: Add Heartbeat to the new GroupCoordinator interface (#12848)
PR-12355 - KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector (#12355)
PR-12748 - KAFKA-13715: Add generationId field in subscription (#12748)
PR-12778 - KAFKA-13152: Add cache size metrics (#12778)
PR-12899 - KAFKA-14413: Separate MirrorMaker configurations for each connector (#12899)
PR-11890 - KAFKA-13731: Allow standalone workers to be started without providing any connector configurations (#11890)
PR-12917 - KAFKA-14414: Fix request/response header size calculation (#12917)
PR-12920 - KAFKA-14339: Do not perform producerCommit on serializationError when trying offsetWriter flush (#12920)
PR-12845 - KAFKA-14367: Add JoinGroup to the new GroupCoordinator interface (#12845)
PR-12876 - KAFKA-12476: Prevent herder tick thread from sleeping excessively after slow operations (#12876)
PR-12909 - KAFKA-14422: Consumer rebalance stuck after new static member joins a group with members not supporting static members (#12909)
PR-12884 - KAFKA-14393: Default metadata retition by bytes (#12884)
PR-12875 - KAFKA-12679: Handle lock exceptions in state updater (#12875)
PR-12890 - KAFKA-14414: Remove unnecessary usage of ObjectSerializationCache (#12890)
PR-12904 - KAFKA-14299: Handle double rebalances better (#12904)
PR-12639 - KAFKA-14242: use mock managers to avoid duplicated resource allocation (#12639)
d139379e - KAFKA-14009: Rebalance timeout should be updated when static member rejoins
PR-12877 - KAFKA-14372: Choose replicas only from ISR for preferred read replica (#12877)
PR-12805 - KAFKA-12610: Implement PluginClassLoader::getResource and getResources (#12805)
PR-12761 - KAFKA-14307: Controller time-based snapshots (#12761)
PR-12505 - KAFKA-14133: Replace EasyMock with Mockito in streams tests (#12505)
PR-12859 - KAFKA-14325: Fix NPE on Processor Parameters toString (#12859)
PR-12869 - KAFKA-14382: wait for current rebalance to complete before triggering followup (#12869)
PR-12842 - KAFKA-14406: Fix double iteration of restoring records (#12842)
PR-12840 - KAFKA-14320: Updated Jackson to version 2.13.4 (#12840)
PR-12295 - KAFKA-13586: Prevent exception thrown during connector update from crashing distributed herder (#12295)
PR-12866 - KAFKA-14346: Remove hard-to-mock javax.crypto calls (#12866)
PR-12828 - KAFKA-14346: Remove hard-to-mock RestClient calls (#12828)
PR-12752 - KAFKA-14303: Producer.send without record key and batch.size=0 goes into infinite loop (#12752)
PR-12843 - KAFKA-14375: Remove use of “authorizer-properties” from EndToEndAuthorizerTest (#12843)
PR-12839 - KAFKA-14346: Replace static mocking of WorkerConfig::lookupKafkaClusterId (#12839)
PR-12861 - KAFKA-14388: - Fixes the NPE when using the new Processor API with the DSL (#12861)
PR-12783 - KAFKA-14334: Complete delayed purgatory after replication (#12783)
PR-12857 - KAFKA-14360: Fix links in documentation (#12857)
PR-12418 - KAFKA-13414: Replace PowerMock/EasyMock with Mockito in connect.storage.KafkaOffsetBackingStoreTest (#12418)
PR-12561 - KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance (#12561)
PR-826 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
PR-12577 - KAFKA-13401: KIP-787 - MM2 manage Kafka resources with custom Admin implementation. (#12577)
PR-12795 - KAFKA-14299: Initialize tasks in state updater (#12795)
PR-12835 - KAFKA-14294: check whether a transaction is in flight before skipping a commit (#12835)
PR-12836 - KAFKA-14282: stop tracking Produced sensors by processor node id (#12836)
PR-12809 - KAFKA-14324: Upgrade RocksDB to 7.1.2 (#12809)
PR-12827 - KAFKA-14363: Add new group-coordinator module (KIP-848) (#12827)
PR-12808 - KAFKA-14299: Avoid allocation & synchronization overhead in StreamThread loop (#12808)
PR-12544 - KAFKA-14098: Add meaningful client IDs for Connect workers (#12544)
PR-12760 - KAFKA-14309: FK join upgrades not tested with DEV_VERSION (#12760)
PR-12823 - KAFKA-14132: Replace PowerMock and EasyMock with Mockito in connect tests (#12823)
PR-12772 - KAFKA-14299: Avoid busy polling in state updater (#12772)
PR-12817 - KAFKA-14346: Remove difficult to mock Plugins.compareAndSwapLoader usages (#12817)
PR-12684 - KAFKA-14254: Format timestamps as dates in logs (#12684)
PR-12773 - KAFKA-14299: Return emptied ChangelogReader to ACTIVE_RESTORING (#12773)
PR-12804 - KAFKA-14344: Build EmbeddedKafkaCluster with common configs used for all clients (#12804)
PR-12756 - Kafka 12960: Follow up Commit to filter expired records from Windowed/Session Stores (#12756)
PR-815 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
PR-12822 - KAFKA-14355: Fix integer overflow in ProducerPerformance (#12822)
PR-12409 - KAFKA-14058: Migrate ExactlyOnceWorkerSourceTaskTest from EasyMock and Powermock to Mockito (#12409)
PR-12771 - KAFKA-14299: Handle TaskCorruptedException during initialization (#12771)
PR-12725 - KAFKA-14132: Replace EasyMock with Mockito ConnectorsResourceTest (#12725)
PR-12789 - KAFKA-13989: Errors while evaluating connector type should return UNKNOWN (#12789)
PR-12775 - KAFKA-14247: add handler impl to the prototype (#12775)
PR-12791 - KAFKA-14338: Use MockTime in RetryUtilTest to eliminate flakiness (#12791)
PR-12790 - KAFKA-14337: Correctly remove topicsWithCollisionChars after topic deletion (#12790)
PR-12769 - KAFKA-14314: Add check for null upstreamTopic (#12769)
PR-12465 - Move to mockito (#12465)
PR-12543 - KAFKA-10149: Allow auto preferred leader election when there are ongoing partition reassignments (#12543)
PR-12758 - KAFKA-13152: KIP-770, cache size config deprecation (#12758)
PR-12672 - KAFKA-14247: Consumer background thread base implementation (#12672)
PR-12762 - KAFKA-14299: Never transition to UpdateStandby twice (#12762)
PR-12765 - KAFKA-14316: Fix feature control iterator metadata version handling (#12765)
PR-12747 - KAFKA-14300: Generate snapshot after repeated controller resign (#12747)
PR-12749 - KAFKA-14299: Fix busy polling with separate state restoration (#12749)
PR-12743 - KAFKA-14299: Fix incorrect pauses in separate state restoration (#12743)
PR-12754 - Kafka Streams Threading P3: TaskManager Impl (#12754)
PR-12744 - Kafka Streams Threading P2: Skeleton TaskExecutor Impl (#12744)
PR-12741 - KAFKA-14296: Partition leaders are not demoted during kraft controlled shutdown (#12741)
PR-12736 - KAFKA-14292: Fix KRaft controlled shutdown delay (#12736)
PR-10528 - KAFKA-12497: Skip periodic offset commits for failed source tasks (#10528)
PR-11211 - KAFKA-12960: Enforcing strict retention time for WindowStore and Sess… (#11211)
PR-12737 - Kafka Streams Threading P1: Add Interface for new TaskManager and TaskExecutor (#12737)
PR-12434 - KAFKA-14099: - Fix request logging in connect (#12434)
PR-10910 - KAFKA-12965: - Graceful clean up of task error metrics (#10910)
PR-810 - Fix format of CODEOWNERS
PR-809 - Add ksql to CODEOWNERS
PR-12634 - KAFKA-14225: Fix deadlock caused by lazy val exemptSensor (#12634)
PR-12709 - KAFKA-14275: KRaft Controllers should crash after failing to apply any metadata record (#12709)
PR-12721 - KAFKA-14283: Fix connector creation Auth tests (#12721)
PR-12723 - MINIOR: some typos in javadoc (#12723)
PR-12676 - KAFKA-14209: Integration tests 3/3 (#12676)
PR-12644 - KAFKA-14209: Rewrite self joins to use single state store 2/3 (#12644)
PR-12700 - KAFKA-14270: Fix generated Kafka Streams version file name (#12700)
PR-12663 - KAFKA-14247: Define event handler interface and events (#12663)
PR-12704 - Fix ByteBufferSerializer#serialize(String, ByteBuffer) not roundtrip input with ByteBufferDeserializer#deserialize(String, byte[]) (#12704)
PR-12527 - KAFKA-14133: Replace EasyMock with Mockito in streams tests (#12527)
PR-12683 - KAFKA-4852: Fix ByteBufferSerializer#serialize(String, ByteBuffer) not compatible with offsets (#12683)
eefe8671 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
b9da249b - Fix test
fc786c33 - add unit and integration tests
ba89eaf0 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
PR-12659 - KAFKA-10199: Integrate Topology Pause/Resume with StateUpdater (#12659)
PR-12687 - KAFKA-10199: Fix switching to updating standbys if standby is removed (#12687)
PR-12642 - KAFKA-14207: KRaft Operations documentation (#12642)
PR-12670 - KAFKA-14239: Merge StateRestorationIntegrationTest into RestoreIntegrationTest (#12670)
PR-12681 - KAFKA-14259: BrokerRegistration#toString throws an exception, terminating metadata replay (#12681)
PR-12677 - KAFKA-14132: Replace PowerMock/Easymock with Mockito for WorkerMetricsGroupTest (#12677)
PR-12039 - KAFKA-13725: KIP-768 OAuth code mixes public and internal classes in same package (#12039)
PR-12643 - KAFKA-14097: Make producer ID expiration a dynamic config (#12643)
PR-12641 - KAFKA-14209: Change Topology optimization to accept list of rules 1/3 (#12641)
PR-12652 - KAFKA-14236: ListGroups request produces too much Denied logs in authorizer (#12652)
9d1f9f77 - Bump version to 3.3.0
PR-12628 - KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
PR-12664 - KAFKA-14243: Temporarily disable unsafe downgrade (#12664)
PR-12566 - KAFKA-13927: Fix sink task offset tracking during exception retries (#12566)
PR-12651 - KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error message (#12651)
PR-12653 - KAFKA-14240: Validate KRaft snapshot state on startup (#12653)
PR-12650 - KAFKA-10199: Adapt restoration integration tests to state updater (#12650)
PR-12632 - KAFKA-12878: Support –bootstrap-server in kafka-streams-application-reset tool (#12632)
PR-12658 - KAFKA-14233: disable testReloadUpdatedFilesWithoutConfigChange first to fix the build (#12658)
PR-12655 - KAFKA-14238: KRaft metadata log should not delete segment past the latest snapshot (#12655)
PR-12638 - KAFKA-10199: Register and unregister changelog topics in state updater (#12638)
PR-12602 - KAFKA-13985: Skip committing MirrorSourceTask records without metadata (#12602)
PR-12570 - KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
PR-12615 - KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito (#12615)
PR-12600 - KAFKA-10199: Suspend tasks in the state updater on revocation (#12600)
PR-12414 - KAFKA-14073: Log the reason for snapshot (#12414)
PR-12625 - KAFKA-14222: KRaft’s memory pool should always allocate a buffer (#12625)
PR-12626 - KAFKA-14208: Do not raise wakeup in consumer during asynchronous offset commits (#12626)
PR-12603 - KAFKA-14196: Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603)
PR-12624 - KAFKA-14215: Ensure forwarded requests are applied to broker request quota (#12624)
b2639c8d - Remove the html end tag from upgrade.html
PR-12597 - KAFKA-14205: Document how to replace the disk for the KRaft Controller (#12597)
PR-12596 - KAFKA-14203: Disable snapshot generation on broker after metadata errors (#12596)
PR-12616 - KAFKA-14198: Define separate configuration for Swagger (#12616)
PR-12617 - KAFKA-14216: Remove ZooKeeper reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc (#12617)
PR-12618 - KAFKA-14217: app-reset-tool.html should not show –zookeeper flag that no longer exists (#12618)
PR-12609 - KAFKA-14198: swagger-jaxrs2 dependency should be compileOnly (#12609)
PR-12584 - KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584)
PR-12604 - KAFKA-14188: Getting started for Kafka with KRaft (#12604)
PR-12599 - KAFKA-14201: Consumer should not send group instance ID if committing with empty member ID (#12599)
PR-12598 - KAFKA-14201: Consumer should not send group instance ID if committing with empty member ID (server side) (#12598)
PR-12595 - KAFKA-14204: QuorumController must correctly handle overly large batches (#12595)
PR-11783 - KAFKA-14143: Exactly-once source connector system tests (#11783)
PR-12478 - KAFKA-13952: fix RetryWithToleranceOperator to respect infinite retries configuration (#12478)
6397d503 - Remove the html end tag from upgrade.html
PR-12586 - KAFKA-14200: kafka-features.sh must exit with non-zero error code on error (#12586)
PR-12583 - KAFKA-10199: Separate state updater from old restore (#12583)
PR-12492 - KAFKA-14133: Replace EasyMock with Mockito in streams tests (#12492)
c3595588 - KAFKA-14015: Reconfigure tasks if configs have been changed for restarted connectors in standalone mode(#12568)
PR-12519 - KAFKA-10199: Handle exceptions from state updater (#12519)
PR-12578 - KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case (#12578)
PR-12573 - KAFKA-10199: Remove changelog unregister from state updater (#12573)
PR-12533 - KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Controller (#12533)
PR-12576 - Update expected task configs for FileStream source and sink connectors in ConnectRestApiTest (#12576)
PR-12569 - KAFKA-10199: Shutdown state updater on task manager shutdown (#12569)
PR-12294 - KAFKA-13990: KRaft controller should return right features in ApiVersionResponse (#12294)
PR-12571 - KAFKA-14187: kafka-features.sh: add support for –metadata (#12571)
PR-12459 - KAFKA-13036: Replace EasyMock and PowerMock with Mockito for RocksDBMetricsRecorderTest (#12459)
PR-795 - CONFLUENT: Retry compilation after zinc compile cache error
PR-12562 - KAFKA-10199: Remove tasks from state updater on shutdown (#12562)
PR-12554 - KAFKA-10199: Handle restored tasks output by state updater (#12554)
PR-12565 - KAFKA-14183: Cluster metadata bootstrap file should use header/footer (#12565)
PR-12513 - KAFKA-14177: Correctly support older kraft versions without FeatureLevelRecord (#12513)
PR-12103 - KAFKA-13850: Show missing record type in MetadataShell (#12103)
PR-12046 - KAFKA-10360: Allow disabling JMX Reporter (KIP-830) (#12046)
PR-12551 - KAFKA-14178: Don’t record queue time for deferred events (#12551)
PR-12549 - KAFKA-10199: Introduce task registry (#12549)
PR-12473 - KAFKA-13133: Replace EasyMock and PowerMock with Mockito for AbstractHerderTest (#12473)
PR-12536 - KAFKA-14160: Streamline clusterId retrieval in Connect (#12536)
PR-12502 - KAFKA-14162: Stop adding immutable maps/lists to record keys/values in HoistField and MaskField SMTs (#12502)
c6c1895f - CONFLUENT: Add original files from confluent/master
PR-12547 - KAFKA-10199: Remove tasks from state updater on revoked and lost partitions (#12547)
a1690ecd - CONFLUENT: Add original files from confluent/master
PR-12509 - KAFKA-14133: Replace EasyMock with Mockito in WorkerCoordinatorTest and RootResourceTest (#12509)
4fe80532 - CONFLUENT: Add original files from confluent/master
44a88aee - CONFLUENT: Add original files from confluent/master
PR-12490 - KAFKA-14147: Prevent deferredTaskUpdates map from growing monotonically in KafkaConfigBackingStore (#12490)
2c8a781c - CONFLUENT: Add original files from confluent/master
PR-12501 - KAFKA-14097: Separate configuration for producer ID expiry (KIP-854) (#12501)
88448f61 - CONFLUENT: Add original files from confluent/master
PR-12469 - KAFKA-13914: Add command line tool kafka-metadata-quorum.sh (#12469)
b212677b - CONFLUENT: Add original files from confluent/master
PR-12508 - KAFKA-13888: Implement LastFetchTimestamp and in LastCaughtUpTimestamp for DescribeQuorumResponse [KIP-836] (#12508)
97d9c73a - CONFLUENT: Add original files from confluent/master
PR-12525 - remove sleep in test (#12525)
PR-12535 - KAFKA-13769: Fix version check in SubscriptionStoreReceiveProcessorSupplier (#12535)
PR-12518 - KAFKA-14167: Completion exceptions should not be translated directly to error codes (#12518)
PR-12517 - KAFKA-13940: Return NOT_LEADER_OR_FOLLOWER if DescribeQuorum sent to non-leader (#12517)
PR-12520 - KAFKA-10199: Remove tasks from state updater on revocation (#12520)
PR-12521 - KAFKA-10199: Remove tasks from state updater on partition lost (#12521)
PR-12277 - KAFKA-13971: Atomicity violations caused by improper usage of ConcurrentHashMap (#12277)
PR-12491 - KAFKA-14148: Update ResetOffsetsDoc (#12491)
PR-12466 - KAFKA-10199: Handle task closure and recycling from state updater (#12466)
PR-12450 - KAFKA-13809: Propagate full connector configuration to tasks in FileStream connectors (#12450)
PR-12514 - KAFKA-14154: Kraft controller should return NOT_CONTROLLER if request epoch is ahead (#12514)
PR-12416 - KAFKA-13559: Fix issue where responses intermittently takes 300+ ms to respond, even when the server is idle. (#12416)
PR-12163 - KAFKA-13900: Support Java 9 direct ByteBuffer Checksum methods (#12163)
PR-12274 - KAFKA-13959: Controller should unfence Broker with busy metadata log (#12274)
PR-12497 - KAFKA-10199: Expose read only task from state updater (#12497)
PR-12506 - KAFKA-14154: Return NOT_CONTROLLER from AlterPartition if leader is ahead of controller (#12506)
PR-12498 - KAFKA-13986: Brokers should include node.id in fetches to metadata quorum (#12498)
PR-12507 - KAFKA-14163: Retry compilation after zinc compile cache error (#12507)
a6a8ff9e - Add muckrake mapping for 7.3
PR-12407 - Remove duplicate common.message.* from clients:test jar file (#12407)
PR-12484 - KAFKA-13060: Replace EasyMock and PowerMock with Mockito in WorkerGroupMemberTest.java (#12484)
PR-12184 - Fix the rate window size calculation for edge cases (#12184)
PR-12487 - KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode (#12487)
f3cf6db3 - KAFKA-14114: Add Metadata Error Related Metrics
PR-12396 - KAFKA-14051: Create metrics reporters in KRaft remote controllers (#12396)
PR-12403 - KAFKA-13166: Fix missing ControllerApis error handling (#12403)
PR-12467 - KAFKA-14129: KRaft must check manual assignments for createTopics are contiguous (#12467)
PR-12447 - KAFKA-14124: improve quorum controller fault handling (#12447)
c2422f63 - KSECURITY-478: migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
PR-12489 - KAFKA-14144: Compare AlterPartition LeaderAndIsr before fencing partition epoch (#12489)
PR-12472 - KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest (#12472)
PR-12457 - KAFKA-14104: Add CRC validation when iterating over Metadata Log Records (#12457)
PR-12440 - KAFKA-14107: Upgrade Jetty version (#12440)
PR-12483 - KAFKA-14136: Generate ConfigRecord for brokers even if the value is unchanged (#12483)
PR-12455 - KAFKA-14111: Fix sensitive dynamic broker configs in KRaft (#12455)
PR-12468 - KAFKA-13877: Fix flakiness in RackAwarenessIntegrationTest (#12468)
PR-11615 - KAFKA-13546: Do not fail connector validation if default topic creation group is explicitly specified (#11615)
PR-12432 - KAFKA-14095: Improve handling of sync offset failures in MirrorMaker (#12432)
PR-12441 - KAFKA-14108: Ensure both JUnit 4 and JUnit 5 tests run (#12441)
PR-12321 - KAFKA-14012: Add warning to closeQuietly documentation about method references of null objects (#12321)
PR-12442 - KAFKA-10199: Bookkeep tasks during assignment for use with state updater (#12442)
PR-12429 - KAFKA-14089: Only check for committed seqnos after disabling exactly-once support in Connect integration test (#12429)
PR-12309 - KAFKA-14007: Close header converters during Connect task shutdown (#12309)
PR-12439 - KAFKA-10199: Further refactor task lifecycle management (#12439)
PR-12437 - KAFKA-13769: Add tests for ForeignJoinSubscriptionProcessorSupplier (#12437)
PR-11886 - KAFKA-13730: OAuth access token validation fails if it does not contain the “sub” claim (#11886)
PR-12415 - KAFKA-14079: - Ack failed records in WorkerSourceTask when error tolerance is ALL (#12415)
PR-12438 - KAFKA-13868: Replace YouTube embedded video with links on streams page (#12438)
PR-12422 - KAFKA-13982: Move WorkerConfigTransformerTest to use Mockito (#12422)
PR-12374 - KAFKA-14039: Fix AlterConfigPolicy usage in KRaft (#12374)
PR-12411 - KAFKA-14078: Do leader/epoch validation in Fetch before checking for valid replica (#12411)
PR-12423 - KAFKA-13158: Migrate ConnectClusterStateImpl to Mockito (#12423)
PR-12433 - KAFKA-14093: Use single-worker Connect cluster when testing fenced leader recovery (#12433)
PR-12347 - KAFKA-13919: expose log recovery metrics (#12347)
PR-12427 - KAFKA-10199: Add tasks to state updater when they are created (#12427)
PR-12285 - KAFKA-14001: Migrate streams module to JUnit 5 - Part 1 (#12285)
PR-12408 - KAFKA-14076: Fix issues with KafkaStreams.CloseOptions (#12408)
PR-12397 - KAFKA-10199: Cleanup TaskManager and Task interfaces (#12397)
PR-12365 - KAFKA-14020: Performance regression in Producer (#12365)
PR-12320 - KAFKA-13702: Connect RestClient overrides response status code on request failure (#12320)
PR-12349 - KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance (#12349)
PR-12324 - KAFKA-12699: Override the default handler for stream threads if the stream’s handler is used (#12324)
PR-12387 - KAFKA-10199: Add RESUME in state updater (#12387)
PR-12386 - KAFKA-10199: Add PAUSE in state updater (#12386)
PR-12420 - KAFKA-13769: Fix version check in SubscriptionJoinForeignProcessorSupplier (#12420)
PR-12405 - KAFKA-13572: Fix negative preferred replica imbalanced count metric (#12405)
PR-10964 - KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
PR-12370 - KAFKA-13785: [10/N][emit final] more unit test for session store and disable cache for emit final sliding window (#12370)
PR-12265 - KAFKA-13968: Fix 3 major bugs of KRaft snapshot generating (#12265)
PR-12398 - KAFKA-14062: OAuth client token refresh fails with SASL extensions (#12398)
PR-12388 - KAFKA-14013: Limit the length of the reason field sent on the wire (#12388)

ksqlDB¶

40d8003d - fix: Replace regex in CommandParser with a split by space
cc0501d8 - fix: Simplify regex in DdlDmlRequestValidators to avoid catastrophic backtracking
aefe6338 - Fix log4j-extensions
08853cd3 - Fix log4j-extensions
PR-9821 - Fix: migrations tool without basic auth works again
PR-9818 - Fix: default to get authHeader if using basic
PR-9812 - Fix: catch TopicAuthorizationExceptions when getting topic config
PR-9717 - Fix: change password-based configs to Type.PASSWORD
6cffbe92 - Feat: properly use Kafka consumer in multi thread environment
a74891c1 - Refactor: fixed checkstyle errors
38122a2f - Feat: Print Topic sets auto.offset.reset as default
PR-9731 - Fix: show streams/tables should not return names in back tick
4ec1bc23 - Refactor: removed executePrintQueryWithVariables
56133e1c - Feat: Using fixed poll timeout instead of disconnect check interval
fc3e81c8 - Refactor: Remove PrintResponseWriter and move write to PrintSubscriber
abe2ebd6 - Refactor: vcrfxia’s feedback, comments, unused code, moving things
7e384445 - Refactor: small refactoring, luca-filipponi PR feedback
ae274326 - Refactor: use Publisher instead of QueryPublisher everywhere
1b968cff - Feat: Modify QueryStreamHandler so it handles Print Statements
993dc778 - Feat: Extend QueryEndpoint /KsqlServerEndpoints to create printPub…
facadf54 - Feat: Implement BlockingPrintPublisher extending BasePublisher
42f8f6dc - Feat: Add PrintResponseWriter and implement PrintResponseWriter
48d986a7 - Feat: Implement PrintSubscriber extending BaseSubscriber
PR-9713 - Bump netty tcnative version to 2.0.54.Final
PR-9702 - Feat: Allow alternative nullable representations also for protobuf_nosr
PR-9708 - APIF-3117: Unpin Scala version.
PR-9697 - APIF-3122: Unpin Protobuf version.
PR-9694 - Fix: null check of query plan when restoring
8ee15c2c - checkstyle
b42fb444 - revise: Victoria’s comments
PR-9676 - Fix: prevent null values in runtimeID cmd records from throwing errors
6f6f1843 - checkstyle
0adcb0b7 - missing changes and make spotbugs happy
169b0878 - Feat: Alternative representations for nullable Protobuf fields
fb01cab4 - revise: Sergio’s comments
909f8d11 - revise: Alan’s comments
a1e25d70 - Fix: also skip basic authentication for authentication.skip.paths
PR-9667 - style: fix checkstyle error - Line is longer than 100 characters
PR-9640 - Feat: Support Bytes to UUID UDF Function
PR-9646 - Fix: remove spurious consistency error message (2)
PR-9637 - Fix: Improving error messages
PR-9645 - Fix: remove spurious consistency error message
d911d5f9 - Fix: copy Avro data using physical schema to deal with optional fields
PR-9619 - Fix: null handling in headers
PR-9626 - Feat: allow base64 strings to be coerced to bytes
PR-9622 - Fix: fix tag pattern in doc
PR-9588 - Fix: dont throw persistent query limit errors for non-queries
PR-9612 - Fix: link fix
PR-9561 - refactor: remove usage of the org.apache.kafka.connect.runtime library
PR-9585 - Fix: show command must list the exact names of sources
PR-9594 - Fix: give hint when source name has extra/lacks dquotes
PR-9593 - Fix: enhance error messages when source has extra/lacks double quotes
PR-9573 - Fix: enhance error messages when source has extra/lacks double quotes
PR-9531 - Fix-9243: improving error messages
PR-9524 - Feat: improve syntax error messages
5e07e8c7 - Fix: Ignore schema id while generating materialized schema #9516
PR-9551 - MINOR: add static terminate cluster command and id
PR-9443 - Fix: Add shaded jar for java-client and document its use
PR-9534 - Feat: add source connector property to CREATE SOURCE statements
1b327c61 - Fix: Ignore schema id while creating internal schemas #9163
PR-9458 - Docs: adding newline to allow correct script execution
PR-9522 - Feat: Add TRUNC function
PR-9453 - Feat: add stop co-ordinates to NodeLocation
PR-9512 - Feat: expose cleanup policy in create statements
PR-9514 - Artifactory migration for release_stabilization.py
PR-9369 - Fix: disable SR ID_COMPATIBILITY_STRICT for Protobuf to allow serialization with external references
PR-9504 - Feat: add a customer log4j MetricAppender to track log level stats
PR-9496 - Feat: hard delete internal schemas for persistent queries.
PR-9497 - Fix: don’t confuse flag on with old queries
PR-9493 - Feat: Variadic TopK that can select other columns
PR-9420 - Fix: track lag by queryId rather than applicationId
PR-9481 - Feat: Support Object as the type parameter for a UDAF variadic column argument
f9864add - minor: Update error message for retention config
PR-9482 - Fix: handle windowed queries in YATT
PR-9451 - Feat: Support variadic arguments in the middle of a function signature
PR-9317 - Feat: add command topic migration config and migration logic
PR-9463 - Fix: allow assert values command to check partial columns
PR-9470 - Fix: only use the internal topic id for the shared runtimes without a
b09f94d6 - minor: Make retention config as optional & validate it
PR-9433 - Fix: replace struct schemas with missing connect name
PR-9448 - Fix: cors handling for precondition server
PR-9452 - Fix: Do not clean up when in degraded mode
14ec76ab - minor: Remove unused code
ccfd63c8 - minor: Address review comment
790ef920 - minor: Address review comments
PR-9442 - Fix: apply custom error handler to KsqlStatementException (MINOR)
132c2220 - minor: fix checkstyle
ba164390 - Fix: Validate retention config for exisiting topics
cb3c3ea0 - minor: fix checkstyle
PR-9423 - Fix: Ensure lag reporter doesn’t crash
PR-9424 - Fix: turn off internal topic for transient queries
PR-9409 - Fix: don’t block event loop in PullQueryQueue
PR-9402 - Feat: add support for JSON arrays
PR-9407 - Fix: Update default to not delete consumer groups
PR-9410 - Fix: topologies with internal topics can not use get sources
PR-9238 - Fix: Avoid new schema version registration on INSERT VALUES
PR-9408 - Fix: With shared runtimes, ALTER SYSTEM would cause paused queries to resume.
fde710ae - Fix: Use schema_id while serializing keySchema during INSERT
PR-9383 - Feat: add CORRELATION UDAF
0f699264 - Feat: Add config to set topic retention in the WITH clause #5148

REST Proxy¶

ab17aa29 - Fix the overzealous conflict resolution
5be3406e - KREST-9942: Remove SnakeYaml from dependency management
PR-389 - Cherry pick KREST-1553 for duplicate log fix
PR-385 - Backport KREST-8335 to branch 7.2.x
60edebfe - Changed all configs to dynamic configs
PR-1083 - Fix build breakage caused by KAFKA-14334
PR-1116 - Cherry-pick: fix test (#1115)
69ed0cd9 - remove incorrect synchronization
08035fb1 - remove incorrect synchronization
PR-1110 - Add server-common dependency to avoid the issue of org.apache.kafka.s
PR-1108 - Set kafka-schema-registry version in dependency mamanagement
7a7f5dcd - Remove unnecessary ${project.version} tag
PR-1102 - Fix transitive dependency of kafka-schema-registry
PR-1101 - Upgrade dependency org.yaml_snakeyaml
PR-1097 - KREST-8639: Only call get on topic manager provider once
PR-1095 - Krest 8353 per lkc rate limiting admin endpoints
PR-1096 - Make sure SchemaRegistryRestApplication check for leader election finish
PR-1093 - Address a coincidence causing a build breakage
PR-1090 - KREST-8518: Add dry-run/validate-only versions of some topic APIs
PR-1084 - KREST-3009: Add ability to update partition count on a topic
PR-1077 - Remove obsolete Avro test
PR-1075 - Fix the line longer than 100 characters issue
PR-1072 - KREST-7107: Shouldn’t provide overly detailed info or stack traces in responses -
PR-1051 - NPE when post to topics endpoint without payload
PR-1061 - Log Describe log Dirs and log the topic name causing a 404 anyway
PR-1059 - KREST-7275: update openapi changes for batch acl
PR-1035 - Replaced deprecated method
PR-1053 - KREST-7005: use a sensor object per request - but shared per LKC id
PR-365 - Adapt Test to handle Exception class changing its parent
ed1ab64b - Make errorSensor final
0918cd98 - Remove Rest429CustomResponseHandler, and - Use MetricsResourceMethodApplicationListener logic for recording the metrics of 429
f7f3835e - Make sure that fourTwoNineHandler only bound to the right application
94f41bcf - KREST-8429: refactor to separate 429 metrics from logging
75aa92ab - KREST-8429: Address review comments
283fd540 - KREST-8429: remove LKC tags from 429 metrics
686de13e - KREST-8429: alert on 429s
a518e923 - KREST-8391: add comment per review
7b4b32fe - KREST-8391: swap order of dos rate limiters
PR-362 - APIF-2959: Add support for per-listener SSL config.
PR-361 - Add ApplicationServer.getSslContextFactories method.
PR-359 - CIAM-2346: Override-able Request Log Format
PR-360 - APIF-2959: Fix SslConfig.getClientAuth.
PR-357 - APIF-2959: Prepare rest-utils API for per-listener SSL configs.
6a7c4069 - Use LINEAR instead of CONSTANT bucket sizing for latency percentiles.

Schema Registry¶

PR-2627 - For Maven plugin, ensure metadata/ruleSet subject names are decoded
PR-2626 - Allow rule actions to differ per rule
PR-2623 - Upgrade deps for azure, cel, jsonata
909c2f98 - DGS-xxxx: Update azure-identity to 1.8.2
52e6bc28 - DGS-xxxx: update Msal4j dependency to bring in latest json-smart
PR-2620 - Add CSFLE tests for bytes
PR-2619 - Upgrade Tink to 1.9.0
PR-2616 - DGS-7005 Convert socket timeouts to RetriableException for converters
PR-2613 - Add param for CEL executors to ignore guard separator
PR-2610 - Update OpenAPI spec with rules, metadata
PR-2607 - Convert Avro utf8 objects to string for CEL
f45fe28e - DGS-6698: Remove SnakeYaml from the dependencyManagement
PR-2602 - DGS-6929 Ignore invalid Avro defaults for source connectors
ad72fd78 - Fix merge issue
PR-2599 - DGS-6516 Add normalize query param to compatibility APIs
PR-2597 - Refactor duplicate tag handling for field rule executors
PR-2593 - Enhancements to DlqAction
PR-2594 - Add support for optional guard for CEL expressions
PR-2590 - DGS-6701 Fix getSchemaById to match subject if passed
ed4ec914 - DGS-6698: Remove SnakeYaml from the dependencyManagement
PR-2589 - DGS-6613 Ignore invalid Avro defaults in Avro Converter
6bb74171 - Fix test due to merge
PR-2588 - DGS-6603 Fix default conversions in JsonSchemaConverter
PR-2586 - Allow field xforms to still run if no tag is specified
PR-2585 - Revert behavior of DGS-6522
PR-2584 - DGS-6522 Disallow empty subject names during schema registration
PR-2582 - Reset default for leader election delay to false
PR-2577 - DGS-6373: Ignore leading dot when merging Protobuf custom options
PR-2575 - DGS-6331: Handle javaType for oneOfs during JSON deserialization
PR-2572 - DGS-6319: Handle field encryption rules that apply to the same tags
PR-2570 - DGS-6267: Changing logs to debug in SR
PR-2568 - DGS-6306: Allow multiple oneofs in Protobuf converter
PR-2566 - Minor: Add jackson annotation to SchemaEntity.java
PR-2560 - Add config whether to delay leader election
PR-2559 - Add tag APIs to ParsedSchema
PR-2552 - Upgrade Tink to 1.8.0
PR-2550 - DGS-6192: Include default ctx when using subjectPrefix w/wildcard ctx
PR-2546 - Adding createBrokerConfig changes
PR-2533 - DGS-5624: SR Oauth client config : Make Identity pool and logical cluster id as optional
PR-2544 - Add rule params
PR-2540 - DGS-6075: Add explicit definition of snakeyaml.
PR-2543 - DGS-6073: upgrade com.azure.azure-identity to 1.7.3
PR-2531 - Add support for ruleSet handlers
PR-2542 - DGS-6072: Upgrade google-api-client to 1.35.2
PR-2539 - DGS-6071: Handle nested messages in Protobuf custom options
PR-2538 - Add check to relookup schema after population/normalization
PR-2532 - Handle OneOf in Protobuf Path
PR-2530 - DGS-6032: Externalize Kafka group configurations
PR-2527 - DGS-6033: Allow rule names to be qualified with subject in configs
PR-2528 - Add doc field to Rule
PR-2525 - DGS-6033: Allow multiple rules of the same type
PR-2523 - DGS-6023: Add Protobuf converter config to not generate index for unions
PR-2522 - DGS-6022: Add Protobuf converter config to generate struct for nulls
PR-2517 - MINOR cherry-pick Protobuf validate optimization to 7.0.x
PR-2519 - DGS-6014: cherry pick Maven plugin enhancement to 5.5.x
PR-2518 - DGS-6014: Make url decoding in Maven plugin more lenient
PR-2516 - DGS-5908: Qualify extension fields from Protobuf descriptor
PR-2515 - DGS-5959: Update Maven plugin for metadata/ruleSet
PR-2512 - DGS-5215: Add integration tests for CSFLE [skip secret scan]
PR-2514 - DGS-5950: Allow optional map key in Avro
PR-2510 - DGS-5907: Allow CMK to be passed via metadata
PR-2509 - DGS-5909: When normalizing Avro, ensure defaults are valid
PR-2505 - DGS-5908: Qualify extension fields when normalizing Protobuf
PR-2504 - Add more CEL executor tests
PR-2501 - DGS-5897: Support return Avro schemas with all refs resolved
PR-2500 - DGS-5896: Allow metadata/rules to be passed to console producer
PR-2496 - DGS-5019: Add error tests for CSFLE
PR-2497 - DGS-5576: Change sensitive field type to TYPE.PASSWORD
PR-2495 - DGS-5850: Support adding schema tags to record
PR-2494 - DGS-5851: Add header support to console producer/consumer
PR-2470 - DGS-5210: Add Support to Copy ParsedSchema with Given Tags [skip secret scan]
PR-2490 - DDGS-5816: Track key for use in value serdes
PR-2482 - DGS-5018: Add client-side field-level encryption [skip secret scan]
PR-2487 - DGS-5804: Add a ConfigProvider to read SR metadata
PR-2486 - DGS-5803: Add size/ttl configs for caches holding latest schemas
PR-2483 - DGS-4826: Validate rulesets sent via REST API
PR-2481 - DGS-4826: Allow updating only metadata/ruleSet during registration
PR-2435 - DGS-5335: Encode sensitive metadata properties at rest
PR-2469 - DGS-5796: Initiate leader election after resources have been set up
PR-2475 - Fix handling of Protobuf repeated options
PR-2471 - Preserve metadata associated with primitive types during normalization
PR-2447 - SUP-49: Improve Error reporting on incompatibility for better usability
PR-2462 - DGS-3254: Support for separate internal and external certs
PR-2468 - DGS-5567: Handle Protobuf map options in toCanonicalString()
PR-2467 - Changed the header name of pool id
PR-2465 - DGS-5514: Implementation for new config parameters
PR-2460 - DGS-5544: Support nulls when using optional fields in proto2
PR-2459 - DGS-5400: Support subjectPrefix containing wildcard context and subject
PR-2457 - DGS-5514: Enhance config APIs with additional parameters
PR-2456 - Derive schema maven plugin 7.3.x
PR-2455 - Bug fix for union types, added test for same and removing unused imports
PR-2450 - Pin Snakeyaml version
PR-2448 - DGS-5475: Add error handling for rules
PR-2443 - DGS-5423: Check subject when looking up ID across contexts
PR-2431 - DGS-5117: Derive Schema Maven Plugin
PR-2441 - Minor: change OAuth config string
PR-2440 - Update the Snakeyaml dependency
PR-2418 - DGS-4996: Deriving Schema from Messages - Avro
PR-2439 - Add support for use.latest.version for deserializers
PR-2409 - DGS-3721: SaslOAuthCredentialProvider
PR-2434 - Fix build breakage caused by KAFKA-14334
PR-2428 - DGS-4823: Handle Jackson annotations during object transform
PR-2371 - DGS-3396: Add deletedOnly=true to SR list APIs
PR-2421 - DGS-5220: Handle nested extend decls in Protobuf
PR-2413 - Skip set-compatibility goal if skip flag is set
PR-2417 - Rename rule body to expr, add onSuccess/onFailure
PR-2415 - DGS-5254: Make timeouts configurable when forwarding requests
PR-2414 - DGS-5253: Handle cycles when resolving references
PR-2401 - DGS-4995: Deriving Schema from Messages - Protobuf
PR-2344 - DGS-3719, DGS-3720: Added OAuthCredentialProvider and CachedOauthTokenRetriever
PR-2411 - DGS-5243: Remove static from JsonSchemaData.toConnectData
PR-2402 - Add rule client functionality
PR-2407 - MINOR: Fix OpenAPI deleteSubjectConfig example
PR-2405 - DGS-4754: Handle empty record default at field level
PR-2398 - Add metadata/rules to schema format
PR-2374 - DGS-4813: Deriving Schema from Messages - JSON
PR-2372 - Migrate PowerMock to Mockito in RestServiceTest
PR-2366 - Allow specifying readerSchema on KafkaAvroDeserializer
PR-2361 - DGS-4419: Adding validation error report in json serialization
PR-2363 - Adding timer in onJoinPrepare

Commercial Features¶

Confluent Server¶

OBSTel-189 - Include process.roles in the MetricsMessage for C3 (#9618)
KAFKA-14887: FinalizedFeatureChangeListener should not shut down when
(cherry picked from commit e64436dbfb4f8cbaf529f36c068747e452e189de)
KSECURITY-1556: Updated jetty version
KAFLAESS-1743: replica entity invalidation fix
[KMETA-658] Add ClusterBalanceManager to BrokerServer colocated startup
OBSTel-189: Add Kafka process roles as part of the KafkaServerProvider’s resource attributes
Reduce remove brokers history from 2 weeks to 1 day
DGS-6911: Properly close schema validation interceptor
KSECURITY-1460, KSECURITY-1466: Updated snakeyaml, json-smart and nim
KGLOBAL-3170: Deflake testFetchRequestPartitionMaxSize in ClusterLinkFetcherThreadTest (#9467)
KSECURITY-1459: Updated golang.org/x/net mod.
DGS-6911: Fix memory leak by closing schema validation interceptor
KSECURITY-1455: Updated golang.org/x/text mod.
KSECURITY-1453: Updated version of golang.org/x/sys
Use RemoteConfigurationRequest.Builder to avoid issues with shading (
API-665: Allow named listeners to passthrough to Kafka HTTP Server.
KAFKALESS-1412: Fix SBC status API for KRaft controller
cherrypick KAFKA-14816: Only load SSL properties when issuing cross-worker requests to HTTPS URLs
KSECURITY-1399: Add custom_security_config as parameter in KafkaService
KMETA-745: Support –config file in metadata shell
Backport “Minor: add logging to inLock / inWriteLock to catch leaked locks (#8594)” to 7.4.x
Made Request Pipelining disabled by default in 7.4.x
KMETA-478: Enable ZooKeeper audit logs by default
KMETA-714: disable migration from ZooKeeper while Confluent tiered storage is enabled
Fix version comparison in kafkatests (#8922)
KMETA-726: Allow config records before topic record
KAFKALESS-1633: Update MovementExclusionGoal to check for original brokers (#8791)
Fixing the Identation error
KMETA-700: Re-enable ZkMigrationIntegrationTest#testDualWrite
DGS-4197: Fix schema validation not working under KRaft
KGLOBAL-2530: Disallow ZooKeeper to KRaft migration if cluster contains topic placement topics
KMETA-668: Halting the process if Authorizer fails to startup (#8871)
CONFLUENT: enable FFF in FetchFromFollowerIntegrationTest
KMETA-702: Disable migration when cluster link metadata is present in the Zookeeper mode.
KAFKA-14676: Include all SASL configs in login cache key to ensure clients in a JVM can use different OAuth configs (#13211)
KSECURITY-885: Add additional parameter to pass configs overrides for controller nodes
Disable ZkMigrationIntegrationTest.testDualWrite until it can be fixed
KAFKA-14458: Introduce RPC support during ZooKeeper migration #13028
KMETA-693: Fix missing MetadataEncryptorFactory references
KAFKALESS-1591: Fix merging of partition and replica metrics when replica metrics are missing
KSECURITY-917: Updated AclAuthorizerTest to run tests for both zk and kraft (#8464)
Move BrokerHealth APIs to CloudAdmin
KAFKA-14457: Controller metrics should only expose committed data
KSECURITY-1001: Close old rest client when creating new one (#8785)
Backport part 2 of KAFKA-14446
Backport part 1 of KAFKA-14446
KSECURITY-897: Make expiration timeout for ZooKeeper ACL change notification path configurable
KSECURITY-1005: [7.4.x]Enforce kafka-client-plugins consumers to use the same snakeyaml version
Delete kafka-cells-admin.sh script from 7.4.x
KENGINE-287: RPCProducerIdManager should not wait on new block
KMETA-682: StandardAuthorizer uses findRule and findRulesByResourceTy
KC-2328: Update yaml.v2 to 2.2.4
METRICS-4722: - Clean up reference to TR when closing the remote config source
KAFKA-14601: Improve exception handling in KafkaEventQueue #13089
KSECURITY-885: Add zk_sasl dummy method to NoZookeeperService
KSECURITY-965: Update git version 1.13.0
Moves cells admin APIs to a CloudAdmin interface which lives in a separate package.
METRICS-4611: Update telemetry dependency version (#8371)
KSECURITY-953: Update azure-identity to 1.7.3
KAFKALESS-1621: Disable FFF metrics processing in SBC
KSECURITY-884: Update RbacCluster.java bootstrapServers() function to use “EXTERNAL” type listener.
KSECURITY-981: Add FeatureZNode zk node path to secure root paths
Bump version to 3.4.0
KDATA-560: Extend Durability Audits to KRaft protocol and metadata topic
CIAM-2581: - Robustify additional roledef file specification
KGLOBAL-2261: Add mirror topic and link coordinator state to updateMetadata request.
Test fixes
KDATA-668: Improve AuditJob logs
KMETA-633: Unable to get a PhysicalClusterMetadata instance when in KRaft mode
KMETA-629: check authorizers in controller startup
KAFKALESS-1545-invalidated-window-metrics
KFS-490: - Admin roles updated to support FlinkResourcePool
KDATA-661: Periodical Audit Run is missing after the job scheduled on last day of year takes more than 24 hours
Make Content-Type header configurable for Telemetry HttpExporter
TRAFFIC-4283: Add RBAC configuration for PrivateLinkAttachment and PrivateLinkAttachmentConnection resources
Use Byte for cell state and Short for cell sizes
KSTORAGE-2635: print leader epoch in TierMessageFormatter output
Moves CellLoad class to common package
KSTORAGE-2416: remove step to delete discarded FTPS file
KENGINE-321: CPU regression from FFF metrics
Add ‘broker’ to cells requests in order to support KRaft cells APIs
KPERF-568: build arm64 images
QEC-8060: Adding trogdor build process in semaphore to be automatically kickstarted after kafka build process,
KENGINE-314: Fix a NPE when FetchSession close.
Adds more cells integration tests
CIAM-2463: - Make and test CCloudAdmin role to allow for Org0 Apikeys
Disable logging fetch requests in slow log
KGLOBAL-2447: Make OAuth client-side configs configurable for cluster links in Cloud
Update persistence of component code in BrokerHealthZNode
KDATA-643: DA Compaction Debugger Tool
KAFKA-14612: Make sure to write a new topics ConfigRecords to metadata log iff the topic is created (#13104)
KAFKALESS-1516: Add capacity metrics for the CapacityGoal hard goals
KAFKA-14601: Improve exception handling in KafkaEventQueue #13089
Kdata 601 ttps integrate with restore
KDATA-365: Add support for deletion of tier topic snapshots in CLM
KDATA 655: Fixing output for on demand audit jobs
Use cellid as Integer instead of string
KAFKALESS-1540: Refresh least-loaded brokers on a successful CapacityGoal replica move
DGS-5187: Propagate broker default config change to catalog
KAFKALESS-1540: Try to rebalance down to a lower threshold when a capacity goal triggers
KAFKALESS-1547: Introduce exponential backing off on cluster requests
KGLOBAL-2486: Changing RPO metric reporting from Percentile to Value
Minor Short circuit accumulated bytes calculation in DelayedFetch
Add CLI integration for TenantPlacementAdvisor
QEC-8060: Adding trogdor build process in semaphore to be automatically kickstarted after kafka build process
KSTORAGE-2549: separate tier fetch and local fetch metrics
KSECURITY-928: Added code to expose granular error codes to user.
Disabling hanging test testNetworkThreadTimeRecorded
KAFKALESS-379: Avoid creating a new tree set of brokers
KGLOBAL-1647: Prevent excessive logging when ClusterLinkSyncAcls fails to create multiple ACLs on destination cluster
KDATA-656: Fixing context as part of on demand DA
KREST-8654: fix issue curl process stuck for RestProduce soak worker
KSECURITY-182: Add test and made some changes to support ACL sync in cluster link with resourceId flag enabled
Request Pipelining POC v2
Explicit linux/amd64 build for M1 local workflow.
CONFLUENT: rename rack state to zone alignment for response bytes tag
KPLATFORM-1532: Add QueuesProxy role for kafka queues
KSECURITY-903: Add KRaft to LdapAuthorizerTest.java
CIAM-2533: - Ability to specify additional roledef files for cc-rbac
KAFKALESS-1546: For leader replicas, fill in FFF metrics with a zero value
KAFKA-14458: Introduce RPC support during ZK migration #13028
KPERF-516: Include EOF response in disconnectedClient metrics
TCP-109: add cloud_new_networklink_permissions.json, add networklink resource type
Adding function definitions to NoZKService
CONFLUENT: Minor cleanups to RbacClusters
KDATA-655: Fix summary generation for on demand DA
KENGINE-313: Observers should be selectable as preferred read replicas
KENGINE-302: Add trace level logs when recording CROSS_RACK egress
Bumping version for encoding of BrokerHealthZNode
cc-trogdor: building image respects local Kafka version
KPLATFORM-1096: Remove cc-rollingupgrade-ctl.
KDATA-647: DA for compaction enhancement
KSECURITY-912: - Fix race condition
KGLOBAL-2461: Account for promotion policy when removing observer from ISR
KSECURITY-898: Update tests in ce-audit module to run on KRaft mode
KSECURITY-801: adding try/catch to DetailedRequestAuditLogFilter
KSECURITY-741: Fix for High CPU usage on Confluent Platform RBAC after AuthCache restructuring
KSECURITY-903: Update LDAP tests to run on KRaft clusters
Kcfun 557 cli for describe cells tenants
Allow for use of Global ducktape params to set the default metadata quorum
Moves PartitionPlacementStrategy.java to org.apache.common in order to be reusable within tools/
KAFKALESS-1334: SBC’s FFF Adoption
KSECURITY-905: Changes to add identity info in token processing error logs.
KCFUN-583: Wire cell load API to SBC cluster model
KSTORAGE-2456: log compaction with transactions property tests
KDATA-573: Implementing on-demand Durability audits
MINOR Fix Noisy Log in setTieringEnabled
Adds various cell commands to complete the CLI
KDATA-384: Add support for Azure in InspectTieredObjects tool
KAFKA-14435: KRaft - Fix allow.everyone.if.no.acl.found config behavior for StandardAuthorizer
KSECURITY-801: Kafka Produce/Consume audit logging
KGLOBAL-2456: Clear partitionsWithNewHighWatermark in ClusterLinkFetcherThread
KCFUN-665: Update KCFUN constant_soak workload to FFF
KAFKALESS-1532: Fix NPE with replica count on a broker is null
KCFUN-252: Adds max replicas per broker limit to CreateTopicPolicy
KAFKALESS-1467: Improve error reported by CDBE::computeEvenClusterLoadPlan
KCFUN-665: Soak workload for fetch from follower
KAFKA-14532: Correctly handle failed fetch when partitions unassigned
KGLOBAL-2453: Fix CL partition max size in fetch request.
Cherrypick KAFKA-14417: Address incompatible error code returned by broker from`InitProducerId`
KSECURITY-882: Support CP MDS/RBAC on KRaft clusters
KAFKALESS-1239: Fix NPEs in EvenClusterLoad state machine during remove/add broker, GVD operations
KAFKALESS-1528: Do NOT invalidate metrics when removing a broker
cherry-pick: KAFKA-14379: Consumer should refresh preferred read replica on update metadata
KENGINE-304: Avoid choosing degraded replica as preferred read replica
KCFUN-602: Implement the basic producer id throttle mechanism
KCFUN-689: Validate Server cets in ConfluentTrustManager
KGLOBAL-2442: Disallow cluster link deletion when mirror topics are in PENDING_STOPPED state
KSTORAGE-2577: Dump headers of tier state files using DumpTierPartitionState tool
KGLOBAL-1852: Reject cluster link request on Confluent Cloud unauthenentication
Added describe permission to DataSteward and DataDiscovery
KPERF-457: Slowlog Sampling
KAFKALESS-1124: Add size() to the EventQueue and size metric for SbcEventQueue
KGLOBAL-2432: Return external endpoint for link coordinator when describing links
Adding Admin APIs for Leadership Priority API
Propagate broker health changes through UpdateMetadataRequest
Adding import to DistributedHerderTest which was preventing build
KSTORAGE-2069: minor change: changed debug to info so that it can show up in opensearch
Removing WorkerWithTopicCreationTest
Update connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/DistributedHerderTest.java
KGLOBAL-2450: Add cluster links to kraft shell
KPERF-425: Enable creating Netty client SSL engine
KGLOBAL-2427: Fix and test tenant connection metric for reverse connections
Minor: Disable the telemetry reporter integration test
Fixing failing test cases in WorkerTest
CNK-1867: Apply latest num.partitions value for new topics without override in ZooKeeper-mode
DGS-5558: removed describe permission for CCA and Operator
CONFLUENT: Upgrade Netty to 4.1.86
KMETA-270: Generate snapshots from metadata shell
KSTORAGE-2394: Add TierFetcher Non Retriable Exception Meter
KDATA-361: Metrics for Tier Topic Snapshots
SBC: Don’t expire replica entity if topic partition is present
KCFUN-696: Add a lower bound to the negative value for TokenBucket
KAFKA-9038: [WIP] Allow creating partitions for topics partitions not in reassignment
KGLOBAL-2415: handle missing topic on Traffic store startup
CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
DP-9030: - Switch to codeartifact repo
CONFLUENT: Use single audit log provider for MDS and its hosting broker
KGLOBAL-2431: Disallowing mirror topic creation for a link in FAILED state
Disabling tests which are causing jenkins build to hang
Run EndToEndAuthorizationTests with KRaft quorum servers
Deduplicating Connect log events
KGLOBAL-2438: Fixing naming for failed broker links
KAFKALESS-1523: Use effective goals when passing goalsConfig from triggerEvenClusterLoadTask
KSTORAGE-2116: add an internal min.segment.ms config to clamp segment.ms
KAFKA-14435: Fix allow.everyone.if.no.acl.found config behavior for StandardAuthorizer
KAFKALESS-1415: Introduce tenant aware goal class
KGLOBAL-2358: converted CCLOUD_HOST_SUFFIXES to a config for fedramp
Make numMessages a long type to increase the capability of workload
CONFLUENT: Change cdc topic load timeout config name and it’s default value
Cherry-pick KAFKA-14372: Choose replicas only from ISR for preferred read replica
Add an interface to propogate broker health status to the metadata cache
Add CCloudIdentityProviderAdmin to delete identity providers/pools for deactivated orgs
KAFKALESS-1241: Add configurable min.free.space threshold to balance for DiskCapacityGoal
Remove use of “authorizer-properties” from EndToEndAuthorizerTest
KDATA-637: Move compaction validation to a separate alert type
KAFKA-14386: Return TopicAssignment from the ReplicaPlacer (#12892)
Cherry pick KAFKA-14417: Producer doesn’t handle REQUEST_TIMED_OUT for InitProducerIdRequest, treats as fatal error
DGS-3472: Emit incremental and snapshot events for topic metadata in KRaft mode
CONFLUENT: Replace deprecated usage of new Long
Cherry pick KAFKA-14334: Complete delayed purgatory after replication (#12783)
DGS-5508: Catch all Exception in CacheBuildEvent
KAFKALESS-1247: Fix main ZooKeeper thread freezing in the integration test
KAFKALESS-1506: Add metrics around the proposal execution
KAFKALESS-1283: Improve the incremental CPU goal’s observability
KCFUN-666, KCFUN-595: Implements rest of cells APIs with adminClient and their integration tests
KAFKALESS-1511: Movements from hard goals shouldn’t be restricted by oscillation prevention
Alter Broker Health API - ZooKeeper [Part 1]
KAFKALESS-1283: Introduce metrics for IncrementalResourceDistributionGoal
KCFUN-583: Derive cell load from SBC cluster model
DGS-5493: Moved DD and DS to different namespace
KCFUN-634: Add tool to give cell rebalancing recommendations
DGS-5248: Adding SR client OAuth config for Schema Validation. (#8137)
Add ce-licensing libs to broker classpath
CONFLUENT: Cluster link fetcher should complete fetches when new messages are replicated
Remove ForwardingAdmin overrides of methods that were deleted in recent commit
CONFLUENT: disable RocksDBMetricsRecorderTest
Kcfun 558 cells admin cli fw
KENGINE-233: fetch from follower metric: detect unknown/same/cross rack egress
KSTORAGE-2303: Keep SBC enabled and processing metrics when demoting a broker
KDATA-582: FTPS snapshot deletion
KPLATFORM-1350: Initializing Broker Health Manager in KRaft mode
KSTORAGE-2472: use delayed deletion for KRaft stray partition deletion
KMETA-523: Ensure sensitive configs are encrypted in KRaft snapshots
KCFUN-595: Implements tenant APIs: adminClient.assignTenantsToCell and adminClient.deleteTenants
KCFUN-666: Plumb DescribeCells and DescribeTenants API and their integration tests
KAFKA-14422: Consumer rebalance stuck after new static member joins a group with members not supporting static members
KAFKALESS-761: Don’t multiply CPU usage by 100
Ignore throttling test
KGLOBAL-1587: Implementing RPO metric for cluster linking
KAFKA-14009: Rebalance timeout should be updated when static member rejoins
KSECURITY-783: Update Async auth timeout behavior
KFS-337: Add flink and network to ConfluentServerCrnAuthority constants
Fix testWorkerTracingEnabled by verifying connectorClass called 3 times is expected
KAFKALESS-1276: Add ReplicaMetricsAggregator
KAFKALESS-1169: Calculate replica CPU from metrics
KAFKALESS-1170: Switch CPU and NW_OUT resource utilization when reloc
CONFLUENT: Fix Kafka management audit log exception metric
KAFKALESS-1402: Disable invalid goal sequences for incremental balancing
Fix NPE in WorkerTest
KSTORAGE-2303: Use LD flag to control whether SBC is enabled when demoting broker
KAFKA-14307: Controller time-based snapshots (#12761)
KAFKALESS-1209: Consolidate test methods creating metrics for a single replica to use ReplicaLoad.Builder/ReplicaLoad
KCFUN-607: Incorporate replication factor in tenant to cell assignments
KAFKALESS-1209: Remove legacy DeterministicLoad#createLoad(cpu, nwIn, nwOut, disk) method
KSTORAGE-2527: GCS checksum implementation should handle ByteBuffer with non zero position
AUTHN-1449: Added new error for issuer cache lookup error.
KAFKALESS-1209: Extend unit tests to set Resource#PRODUCE_IN metric values
Nits for checkstyle
Initial changes
KMETA-387: Confluent Repair Mode (formerly Kafka Repair Mode)
AUTHN-945: Minor fix for decoding of latency records in auth topic
EVENTS-1464: Depend directly on events-schema rather than copying protos
KDATA-596: Create KRaftLogSnapshotGenerationEvent for KRaft DA
Clarify description for broker config
DGS-5318: add Topic + Read/WriteCatalog operations to Org/Env Admin
Increase vagrant aws timeout for system test
KSTORAGE-2303: Set strategy IGNORE for demoted broker in SBC
Add ce-audit missing proto fields to align with events-schema
KDATA-592: Create KRaftNodeStartUpEvent for KRaft DA
KPLATFORM-974: AlterLeadershipAPI max brokers configuration
KDATA-356, KDATA-357, KDATA-358, KDATA-359: Tier topic snapshots
KGLOBAL-2024: Validate JSON filters with alter configs in kafka-cli
Fix build in ForwardingAdmin w/ describeCell overridden method
KAFKALESS-1376: Integrate GoalOptmizationHistory with E2E balancing workflow
KAFKALESS-1376: Add GoalOptimizationLogger
Disable broken tests
KAFKALESS-1367: Skip soft goal execution when incremental balancing is enabled
KPLATFORM-1332: Allow Network Health Manager to be enabled in KRaft mode
KCFUN-642: Implement cell load resolver classes for TenantPlacementAdvisor
Fixing build errors. WorkerTest was fixed based on changes from KAFKA-14346 PR
KSTORAGE-2303: MovementExclusionGoal should honor eligible source or destination
Extract TenantPlacementAdvisor related metrics to shared enum
KMETA-508: Checking mv when creating new cluster link records
KGLOBAL-2389: NettySslEngineFactory to support trustmanager algorithm
KCFUN-600: Setup ProducerIdQuotaManager to manage client throttle
KGLOBAL-2023: Controller support to remove cluster link references in ACLs on removing cluster link.
KAFKALESS-1376: GoalOptimizationHistory
Upgrade cc-base docker image to v20.6.0-jdk-17 and cc-base-alpine docker image to v2.16.0
CNKAF-108: Improve log message for dynamic certificate update
KSTORAGE-2310: Adds misclassification detection logic for stray partition deletion
KAFKALESS-1404: Add metric to track v2 enablement
KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
AUTHN-1409: Added support for identity in v2 schema audit logs.
KGLOBAL-1501: Fix error message propagation and improve error message when mirror topic name is not expected
DP-9370: - use cc-service-bot to manage Semaphore project
KDATA-591: Create KRaftLogStartOffsetChangeEvent for KRaft DA
KAFKALESS-1376: Add SuspendedTopicPartitionPool
KAFKALESS-1438: Log reason when interrupting the Executor
KSECURITY-789: Upgrade to snakeyaml version 1.32
Increase test timeout to reduce flakiness of the test
KDATA-590: Create KRaftHWMChangeEvent for KRaft DA
KAFKALESS-1376: Add TopicPartitionMovement
KAFKALESS-1439: Log full exception stack trace at DEBUG when a plan fails to compute during retry
KGLOBAL-2378: added default value for routes CDC topic
CIAM-2462: - Make new CCloudAdmin role to enable KSQL Healthchecking
METRICS-4145: Migrate TelemetryReporter to OpenTelemetry
AUTHN-1365: Update KsqlAdmin role def
KGLOBAL-2290: Add the internal topic pattern for Cluster Link Auto-mirroring
METRICS-4647: Deduplicate RemoteConfiguration POJOs
CONFLUENT: Minor updates in parsing user metadata record
KCFUN-619: -add-cell-load-rpc-message
KSTORAGE-2069: Enable Idempotent Producer for Tier Topic Internal Producer
Reducing the gradle cache expiry time
KAFKALESS-1386: Clear metrics not belonging to leader/follower
Support dynamic configuration of create-topic/alter-config/create-cluster-link policies in kraft
KCFUN-476: Tenant partition availability metrics for KRaft controllers
AUTHN-1001: Add support for using providerId to fetch token verification key.
CONFLUENT: Allow holes in tag ids
Fix the KafkaConfigTest.scala test failure
KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
KAFKALESS-829: Drop all metrics windows post SBC rebalance
KPLATFORM-1238: Network Health Manager incoming traffic should only consider requests that use the broker bootstrap lb
DA Compaction PerfFixes 1
MINOR: fix stray partition integration test
AUTHN-1032: Add Identity field to authentication_info in audit logs.
KCFUN-668: Auto-Tuning tenants with unlimited quota based on order of
KENGINE-223: Adding metrics to check truncation below HWM and new leader LEO less than HWM
KAFKALESS-1149: SBC v2 windows
CONFLUENT: Fix AuditLogProtobufV2ToJsonTest
CDMUM-2135: Update AuthenticationInfo to include identity_pool and identity
AUTHN-1353: KsqlAdmin user should be able to manage its ApiKey
KSTORAGE-2512: add latency metric for TierObjectStore.getObject call
KAFKALESS-1415: Update cluster model to have tenants by id
KGLOBAL-2355: adher to matching the key format(networkId:pkcId)
KGLOBAL-2209: add mTLS client auth configs to link config using broker
AUTHN-1134: Add new Create Ops
KSTORAGE-2506: fix FileTierPartitionState.segments(from, to) to handle ‘from > endOffset’ case properly
Kcfun 594 create cell manageability rpc api
AUTHN-1173, AUTHN-1174, AUTHN-1175 compute and emit SDS pipeline record latency Kafka-metrics
KENGINE-254: Prevent fetching from follower without a leader epoch
KGLOBAL-2252: Fix minor nits found during demo
KGLOBAL-2274: alter config on every multitenant listener on ssl certif
KCFUN-641: Implement load function classes for TenantPlacementAdvisor
Delete RaftSnapshotWriterTest.java
KAFKALESS-1415: Introduce tenant class for tenant aware goal
KCFUN-551: & KCFUN-552: Implicit cell creation integration and topic/partition integration. pr-image
Update cluster-linking-soak-destination-kraft.yaml to pkc-do0nk1
KSTORAGE-2509: record throttling time for tier compaction
Fix forreplicaAssignment
KGLOBAL-1726: Perform additional validation before truncating mirror partitions below high watermark
KMETA-484: Get rid of metadataQuorum/log path in metadata shell
CIAM-2424: - Name RestClient threads for stackdump identification
KAFKA-14337: Correctly remove topicsWithCollisionChars after topic deletion (#12790)
KMETA-387: Kafka Repair Mode
KSECURITY-794: Upgrade to protobuf version 3.19.6
KSECURITY-766: Update multitenant authorizer to use user metadata store
KMETA-438: Introduce old ClusterLinkDescription
KDATA-550: - Allow old segment deletion for other partitions if current partition is in is in isFrozenLogStartOffsetState
KCFUN-650: Readd KCFUN-581 compute per-IP throttle based on adjusted g
CC-18181: Fix connect tests broken due to WorkerSourceTask change
KMETA-438: Add link coordinator information describe cluster links co
KSTORAGE-2507: Ensure dynamic update to log.deletion.throttler.disk.free.headroom.bytes works well
MINOR: fix negative TierLogToClean.uncleanableBytes value
Param key bytes testing
KDATA-461: CLM handle the case when encryption key is in an invalid state
KDATA-558: Add restore metrics for tracking number of failures during subtasks
MSG-100: Add kafka-queues lib in classpath
KSECURITY-751: Create Store for User Metadata
KMETA-465: Fixing ClusterLinksImage to write out linkMode
KAFKALESS-1167: Add hard goals to incremental goals list
KDATA-570: Add support for durability audit to do
Remove fetch from follower parameter from createBrokerConfig and add
CDX-484: Fix unit test
CDX-484: Add StreamShareMetadataRead role for accessing catalog servic
Log average key size observed during the compaction
KAFKALESS-1210: Delete sample store topic if exists
CPKAFKA-9173: Disable colocated KRaft upgrade from CP 7.1/7.2
KMETA-368: Leader will respond to the partions with leader epoch changed.
KCFUN-643: Lazy evaluation for dynamic quota
KGLOBAL-2035: Unblock local follower fetch requests when records are appended to mirror leader logs
KDATA-589: use override option in revert restore copy
KAFKALESS-1394: Increase timeout for aborting executions to 5m
KCFUN-551, KCFUN-552: Creates CellControlManager and TenantControlManager.
KPERF-521: A couple minor fixes to offset commit batching
KAFKALESS-1362: Refactor MetricDef to use KafkaMetricDef instead of String as its interface
CONFLUENT: Add userResourceId existence check in AuditLogUtils.v2SchemaPrincipal
KGLOBAL-2277: increased timeout to resolve flakiness
KDATA-487: compact topic restore feature branch
KGLOBAL-1583: add network_id to Authenticate audit events for cluster linking
Fix minor build issues in connect
Add missing curly brace in WorkerSourceTaskTest.java
KAFKALESS-1337: Use Time#hiResClockMs() instead of System#currentTimeMillis() in ResourceDistributionGoal
Adding dependencies
Adding log4j
Fixing includes
Added new role for schema registry Healthcheck
KENGINE-238: Add fetch from follower egress metrics to support SBC
KSTORAGE-2479: Replace putBuf with putBuffer in TierObjectStore interface
KCFUN-75: PROXY support for Java client
Adding exactly_once_source argument in test_file_source_and_sink
Updating the docuemntation of atomicMoveWithFallback to match the implementation.
Increase the retries during periodical audit
KGLOBAL-2273: Make ClusterLinkManagerTest.testClusterLinkConfigReencryption more reliable
DP-8824: Use JDK 17 node label if for pull request
KDATA-577: Create KRaftLeaderChangeEvent for KRaft DA
KAFKALESS-1209: Introduce a ProduceInboundCapacityGoal goal and a PRODUCE_IN Resource
KDATA-318: Implement the cloud API
KMETA-458 / KAFKA-14292: controlledShutDownOffset should only be updated if set to -1
KAFKA-14296: Partition leaders are not demoted during kraft controlled shutdown (#7745)
KFS-237: Add new KsqlAppAdmin role
KAFKA-14296: Partition leaders are not demoted during kraft controlled shutdown
KSTORAGE-2435: Config changes to support different hashing logic for offset map
CPU-105: add customplug rbac for connect-plugin-service
CONFLUENT: Fix testSslCipherAlter to work with Java 11 and newer
KGLOBAL-1707: Set config source correctly for cluster link configs
METRICS-4649: Refactor Remote Configuration activeFilters Semantics for 7.3.x
METRICS-4649: Refactor Remote Configuration activeFilters Semantics
KAFKALESS-1387: add OpenTelemetry support for telemetry metrics in SBC
KDATA-355: Add FTPS snapshot integration test
AUTHN-1135: Add providerId to JwtIssuer records
AUTHN-1276: Renamed identity pool constants as per spec.
KMETA-402: Enable audit log provider start up in kraft controller authoriser
AUTHN-1199: Create new error messages based on jose4j error codes.
KMETA-431: KRaft Controllers should crash after failing to apply any metadata record (#12709)
Only prepend LKC if hostname starts with ‘-‘
METRICS-736: Log Error for audit log of consumption on audit log topic
KSTORAGE-3: Unit tests for log recovery when local segments are lost
KGLOBAL-998: Add Kafka CLI support for DescribeClusterLinks
KMETA-451: Allow broker registration with older confluent.metadata.version
KSTORAGE-2277:Smart ISR shrink when a broker has degraded storage
Fix testHealthCheckTotalTimeMs test failure
KMETA-448: Fix auto leader balancing of linked partitions
KENGINE-263: add valid rack set config for FFF
KGLOBAL-2033: Consolidate on one reason code instead of having both a reason code and an exception in LinkStateInfo
KPLATFORM-1150: fix more jdk 17 test failures
Exclude non-fips dependency(bcprov-ext-jdk15on) from trogdor project (#7673)
Exclude non-fips dependency coming from trogdor project
KMETA-448: Fix auto leader balancing of linked partitions
Exclude fips and non-fips bouncycastle dependency jar from connect packages
CONFLUENT: Fix response object when authorization fails
KGLOBAL-2246: Fix the order of creation and deletion of link ACLs during startup or cluster link config update.
KSTORAGE-2304: Add broker load percent to local topic for SBC
Add engine team as code owner for transaction and group coordinator
CDX-469: RBAC changes for stream sharing SR subjects
CONFLUENT: Pass RequestId to forwardedContext in handleEnvelopeRequest
KGLOBAL-2037: Cluster link fetching is not resumed after failure.
Cherrypick excluded bc jdk15 (non fips) from :ce-broker-plugins as it uses non-fips bc jars during compile time to 7.3.x
KMETA-430: Support old confluent.metadata.version feature levels
KMETA-436: Fix compatibility break with default principal schema
Tier State Machine Refactor in Follower Fetch Path
KSTORAGE-2442: MINOR - Memory Usage Fixes for Property Tests
Exclude bc jdk15 (non fips) from :ce-broker-plugins as it uses non-fips bc jars during compile time
KPERF-483: Add HealthCheckTotalTimeMs metric
KSTORAGE-2463: Do not start tier compaction if there are old segments waiting to be tiered
CONFLUENT: Update logs in ConfluentAuditLogProvider
KGLOBAL-2143: Return error message containing all racks that dont have enough brokers during topic creation validation
KGLOBAL-2122: Validate there are enough brokers to satisfy updated topic placement
Add new soak clients CLI to expand a Kafka cluster
KDATA-554: Update restore timing metrics to include additional time due to retries
KAFKALESS-1370: Skip partitions with null leader
KPERF-391: Validate topic name collisions through the MetadataCache
KAFKALESS-1368: Test for topic metrics availability before accessing it
KSTORAGE-2442: Reuse OffsetMaps across CompactionTasks
KDATA-503- DA support idempotent producer.
KSTORAGE-2410: Add ResetTierPartitionState tool
KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
KC-2290: Promote when dynamic network health manager mitigation is disabled
Fix typo in ensure script for KRaft provisioning
KMETA-405: compatibility for kafka-storage –release-version in ensure
Add unit and integration tests
KGLOBAL-2216: Handle SocketTimeoutException in ClusterLinkConnectionChecker when checking TCP connection
KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
KGLOBAL-2215: Limit timeout in CreateClusterLinks
Add formatter for client quotas
KGLOBAL-1578: validate that security protocol of a cloud to cloud link
KAFKALESS-1300: Fix reading goals configs from originals, add integration tests
KGLOBAL-2218: Eliminate stronger validation on ACL sync enable and filter flags.
KPLATFORM-1150: fix jdk 17 test failures
KGLOBAL-2144: Rename LinkError to ClusterLinkError and move it to common package
KDATA-181: Move existing DA test package to Kafka from unit.kafka
DGS-4573: Added more control plane role bindings for schema registry
KSTORAGE-2369: retry with doesNotExist option when GCS upload fails
KMETA-229: Confluent’s CreateTopicPolicy must handle batches correctly in KRaft mode
KGLOBAL-124: Convert Java UUID to Kafka Uuid for CL
KAFKALESS-1210: Remove version from RawMetricType and its usages
KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
METRICS-4564: Add new TelemetrySubmitter role
Added metrics to monitor bad timestamps
KDATA-192: Log internal partition in DumpDB tool
dpennaqueiroz/metrics 4620 - Remote Config bug fixes (#7495)
MINOR: refactor name for producer state metrics helper
dpennaqueiroz/metrics 4620 - Remote Config bug fixes
KAFKALESS-1219: Make progress updates refresh the rebalanced topics only
KAFKALESS-1300: Set SBC goals to static values when dynamically deleted
KPLATFORM-1109: Adding BrokerHealth Status metrics for kc-1786
KGLOBAL-2203: Fix MRC assignment logic so rack positioning in placeme
Add log message when skipping deletion
KCFUN-149: Add new listener ports to Confluent internal list
KGLOBAL-2203: Fix MRC assignment logic so rack positioning in placement file is honored
KENGINE-245: Add metric for timing between duplicate batches
Add log message when skipping deletion
KSECURITY-731: Add KAFKA_CLUSTER to scope.resources hierarchy in audit logs
KCFUN-310: Broker-side Proxy Protocol V2 Support
KSTORAGE-2411: Handle offset map error when calling LogCleaner.validateLog
Telemetry Reporter Remote Configuration (#6585)
KGLOBAL-1577: Added CCloud mTLS trust manager
AUTHN-1173: Add LatencyRecord key and value
[KCFUN-468] Add Failed Auth Metric tagged by IP
Bump version to 3.3.0
KSTORAGE-2116: clamp segment.ms to higher minimum in CCloud
CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
RCCA-8564: log a warning if LDAP login fails becuase of network issues
CIAM-2290: Upgrade bc fips drivers
KSECURITY-726: Push sessionId and requestId to request logs
KSECURITY-579: Only populate the metrics for enabled security mechanisms for the selector.
KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error messa
KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
KSECURITY-704: Added metric to show BlockedQ size, logs at aysnc auth cancellation call points.
CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
KENGINE-246: Metric to measure frequency of nonzero sequences when no producer state is present
FileTierPartitionState metadata snapshots
MINOR: refactor safeSwapForRestoredState
KGLOBAL-1812: Fix thread leak in cluster linking test. (#7188)
KSTORAGE-2415: remove discarded tier metadata file during FTPS init
Add DynamicConfig to enable/disable MetadataCollector
Increase timeout, correct error message returned for addBroker test
KC-2284: Network Health Manager continues trying to demote when dynamic configuration disabled mitigation
KGLOBAL-1797: Remove rack mixing feature flag (#7421)
CC-17987: Remove logging of sensitive records
MINOR: delete discarded FTPS file in safeSwapForRestoredState
KAFKALESS-1123: Add logs to help with diagnosing when submitting tasks to ccrunner
KGLOBAL-1797: Remove multi-region cluster rack mixing feature flag
Correcting the sox logging command for zk that got incorrectly copied
Build and release ZooKeeper image as part of promotion
KAFKALESS-1167: Incremental rebalance support for ResourceDistributionGoal
EVENTS-1022: Deprecate events-schema logical cluster status fields
CONFLUENT: implement ControllerLoadTime metric for KRaft
KSTORAGE-2402: Fix byteEndOffset parameter of TierObjectStore.getObject call
KGLOBAL-1854: Add more identifying tags for link count and unavailable link count
KGLOBAL-2141: Add link error code and message to CreateClusterLinksResponse
Network health manager should be started after Replica manager
KENGINE-229: Partition.isReplicaIsrEligible is misused in Partition.maybeIncrementLeaderHW
Disable failed test CLIENTS-2714
KGLOBAL-2133: Improve link error messages
CIAM-2305: Add CCloudGrandfatheringRoleBindingAdmin role
KENGINE-212: batch transaction requests.
KENGINE-211: Add a metric to record the avg latency for a transaction to transit from PreCommit to Commit.
Migrate connect tests to KRaft
Remove the html end tag from upgrade.html
KENGINE-236: Make producer ID expiration ms a dynamic config
KCFUN-497: Allow decimal values for IP connection rates [pr-image]
MINOR: move Records.sizeInBytes from fetch request callback
KSECURITY-150: Add exception stack trace in LdapAuthenticateCallbackHandler for binding failure
KGLOBAL-2127: retry for metrics to catchup before giving up in test
KAFKALESS-1210: Remove serialization/deserialization for partition/broker metric samples
KAFKALESS-817: Set RF = -1 for internal topics if placement constraint is set
Convert soak cluster app to go modules
KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case
CDMUM-2116: Make the JwtIssuerStatic constructor public for initialization using jwks
AUTHN-1038: Enhanced logging (client and server) for erroneous cases.
Re-arrange clm Kraft tests
Remove the html end tag from upgrade.html
KSTORAGE-2308: delayed stray partition deletion logic
KAFKALESS-1209: Refactor BrokerCapacityInfo constructor into a Builder and replace usages
KPLATFORM-477: Making the metric LeadershipNetworkHealthStatus a number
KMETA-290: Metadata shell supports snapshot and log
CDMUM-2116: Make the JwtIssuerStatic.builder to be public
KGLOBAL-1803: reset lastCaughtupTime when mirror leader appends to its
KSTORAGE-2378: Fix TierMetadataDebugger to accept additional consumer configuration
KGLOBAL-2042: Add LinkState to ListClusterLinks RPC
KSTORAGE-1924: add kraft mode for TierEpochStateReplicationTest
KAFKA-14015: Reconfigure tasks if configs have been changed for restarted connectors in standalone mode(#12568)
KPLATFORM-477: Metric to combine leadership info with network health
KSECURITY-470: Add support for enum values in kafka request audit logs
CONFLUENT: Set exposure field to AuthenticationInfo in request audit logs
KSECURITY-602: Add request and connection id to authentication and authorization audit logs
KGLOBAL-2045: Future replica append need not redo epoch validation
Don’t include Server info as an HTTP response header in the Connect REST API (#7264)
KGLOBAL-2044: Tag unavailable link metric with all reason codes
CCLOG-2002: Don’t include Server info as an HTTP response header in the Connect REST API
Add required perms for sox logging
Log cc-kafka-init and cc-zookeeper latest
KGLOBAL-1751: Add enhanced cluster link creation validation
KPLATFORM-1012: Fix the start up order of Network Health Manager
KGLOBAL-1775: Authenticate NetworkId is allowed to talk to this cluster
KC-2276: Add BrokersExcludedForReplicaPlacementCount metric in KRaft
KGLOBAL-1803: reset lastCaughtupTime when mirror leader appends to its log
KGLOBAL-1754: Add cluster link unavailable info to ListClusterLinksResponse and DescribeClusterLinksResponse
AUTHN-1166: Allow schema registry binding Scopes
SBC Make ‘invalid.replica.assignment.retry.timeout.ms’ description match
CONFLUENT: ConfluentServerAuthorizer fixes to work with new authorizer API
Ensure DQ is backwards compatible by ignoring unknown format
KENGINE-173: Add metric for number of producer IDs
Cherrypick KAFKA-14097: Separate configuration for producer ID expiry (KIP-854)
Minor: Fix SOX versions
Add Javadoc to undocumented public APIs in SBC
KSECURITY-599: - Don’t filter out schema registry healthcheck LSRCs
KSECURITY-577: Update log level in RestClient
KSECURITY-573: Push management audit events to confluent-audit-log-events-v2 topic
KPERF-462: Add sequenceId to each incoming request in each KafkaChannel
KAFKALESS-1273: Dynamic Configuration of Incremental Balancing
KENGINE-224: Add PartitionsTotalSize metric to GroupCoordinator/TransactionCoordinator
Bump cc-base from v18.9.0-jdk17 to v18.15.0-jdk-17
SOX logging migration
CP 7.3 - Telemetry Reporter Remote Configuration [METRICS-4195][METRICS-4186][METRICS-4189]
Gave API key permissions to schema registry roles
[METRICS-4576] KafkaServerMultipleExportersSubsetPartitionerTest fix
Add Metrics as Codeowner for Subset Partitioner #7152
KCFUN-373: KRaft support for launching MultiTenantQuotaConsumer
Disk Utilization Detector
KAFKALESS-1210: Delete Kafka sample store
Artifactory migration
KAFKALESS-1248: Enable testSelfHealingWithIgnoredBrokersPresentWithReplicaPlacements
Disable co-located KRaft tests
CONFLUENT: Add original files from confluent/master
KAFKALESS-1254: Enable two tests in BrokerFailureDetectorTest
KMETA-247: Bind multiTenantSaslSecretsStore on Controller Server.
Fix some lingering test issues
CONFLUENT: Add original files from confluent/master
KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Control
KGLOBAL-1812: Fix thread leak in cluster linking test.
CONFLUENT: Add original files from confluent/master
KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Controller
KGLOBAL-1777: retry finding a link coordinator instead
KDATA-360: Add feature flag and cloud API for TTP snapshots
KSECURITY-595: Enhanced AuthenticationException class to include reasonCode and errorInfo
KGLOBAL-1812: fixed review comments from PR#7184
KGLOBAL-1812: give chance for every resource in harness to close
MINOR: Add the “shell” node back to metadata shell
KSECURITY-577: Update log level in RestClient
KGLOBAL-1979: log error code when updating linked leader epoch fails
KSECUTIRY-582: Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
RCCA-8435: Remove Explicit Checks in Number of Brokers and Partitions in Subset Partitioner
KCFUN-166: Reduce tenant sensor expiration
KGLOBAL-1958: Fix regression in updating corrupted cluster link configs
KSECURITY-155: Add Request_Id, Connection_Id to correlate with Authorization/Authentication/Request Audit Events
KAFKALESS-830: Add follower fetch rate at broker level
KAFKALESS-830: Set replication bytes in/out for topics
KGLOBAL-1750: Add cluster linking connection checker
KGLOBAL-1867: Fix the NPE exception in isAutoMirrorTopic() check when the link is failed
KSECUTIRY-582: Add direct dependency on com.squareup.okhttp3.okhttp 4.9.3
KGLOBAL-1952: Attempt to shutdown both clusters in CL tests even if one fails to avoid thread leak impacting other tests
KAFKA-14144: Compare AlterPartition LeaderAndIsr before fencing part
KDATA-509: Tier Topic Partition Snapshot (TTPS): FlatBuffers and serialization wrapper
KMETA-329: Workaround for AlterPartition regression on stale controllers
KAFKA-13785: [10/N][emit final] more unit test for session store and
cherrypick KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZooKeeper mode
KSTORAGE-2275: Add Windowed Avg/Max/Counter Storage Probe Metrics
Fix typo in offset writer thread config name
KAFKA-14144: cherry-pick
KAFKA-14114: Add Metadata Error Related Metrics
KAFKA-14104: Add CRC validation when iterating over Metadata Log Records (#12457)
KAFKALESS-1267: Deliver BALANCER_ENABLED event in ccrunner thread
METRICS-4507: Add cluster linking metrics to Telemetry Reporter
Remove TelemetryReporter dependency on core introduced in KMETA-295
KPERF-469: Fix regression in Follower Throttle Condition Check
Bump go version to 1.16.15
Upgrade ducktape to 0.11.1
Request filter should be applied independently on each request thread
KAFKA-14114: Add Metadata Error Related Metrics
KMETA-329: Workaround for AlterPartition regression on stale controllers
KSTORAGE-2293: storage performance metrics - fetch
KAFKALESS-1255: Add Javadoc and rename variables to contain their measurement unit
Add muckrake mapping for 7.3
KAFKA-14107: Upgrade Jetty version
KAFKA-14114: Add Metadata Error Related Metrics
KCFUN-138: ClientRequestQuotaManager jmh benchmark
KSECURITY-478: migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775

Security¶

Removed From Supported Operations to follow RBAC Rules
Added fix for unit test
Fixed import
Changed the design of RBAC for Exporters
Removed extra line changes
Added RBAC for Schema linking Phase 1
Remove logging statement
Removed bcfips from common
Pin bouncycastle fips dependency in confluent-security-plugins
Exclude bcpkix-jdk15on as compile dep and add as test dependency
SEC-5350: Update netty-codec-http to 4.1.86.Final
Fix createBrokerConfig due to Kafka merge
Adding null check in AuthorizationFilter operation method
DGS-5605: Remove logging statement for idToken
Changed operation to compatibility read.
Update confluent-schema-registry-security-plugin/src/main/java/io/confluent/kafka/schemaregistry/security/filter/AuthorizationFilter.java
Moved comment and updated variable names
Added RBAC for /schemas/ids/1/subjects and versions endpoints
Adding parameter to createBrokerConfig
DGS-5542: Incorporated code review comments and removed the unused parameter schemaRegistryResourceOperation
DGS-5542: SchemaId validation for numeric value in authorizer filter
Update confluent-schema-registry-security-plugin/src/main/java/io/confluent/kafka/schemaregistry/security/filter/AuthorizationFilter.java
Changed from 5 operations to 1
Updated tests
Added null when schema id does not exist
Changed the logic of authorizeSchemaIdLookup
Changed to Short
Adding OAuth config support for Schema validation
Exclude bcpkix-jdk15on as compile dep and add as test dependency
Pin bc-nonfips dep as a test dependency
Pin bouncycastle fips dependency in confluent-security-plugins
Removed bcfips from common
Excluded bcfips jars
Removed From Supported Operations to follow RBAC Rules
Added fix for unit test
Changed the design of RBAC for Exporters
Fixed import
Removed extra line changes
Added RBAC for Schema linking Phase 1
AUTHN-1087: Explicit define netty versions in kafka-rest plugin
DGS-3396: introduce LookupFilter for list apis
AUTHN-1087: Upgrade vertx to 4.3.2
AUTHN-1087: Upgrade vertx to 3.9.14
AUTHN-1325: Update bc-fips version to 1.0.2.3
Fix bouncycastle issue
CIAM-2416: Extract org resource id from JWT token
DGS-5208: remove fetchFromFollower param from CreateBrokerConfigs
Fix createBrokerConfig call
Exclude bouncycastle non-fips jar from connect packaging
Fixed tests
Added invalid principal check
Fix a NoClassDefFoundError caused by a missing dependency
Fixed checkstyle error
Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas

Secret Registry¶

Exclude Non-FIPS dep and use bc-fips dependency
Remove unused variables
Update RestService.java
CCLOG-2349: remove request body from debug logs
Refactor: use variable name for argument
Add missing parameter in test case
Fixing createBrokerConfig
Exclude Non-FIPS dep and use bc-fips dependency
MINOR: Update createBrokerConfig arguments
Update createBrokeConfig
Update CODEOWNERS
Handle illegal reflection access under java 9+

Control Center¶

RCCA-9557: - Fix broker flapping issue 7.0
Update Docker image tag in integration test to be compatible with 7.4.x
Use getTokenLifetimeMs from Util
Update Docker image tag for integration test
Use Docker image tag 7.1.6
Increase wait start time for control center in test
Make sure Control Center stops properly
Change PORT to avoid conflict with TelemetryReporterIntegrationTest
Change to make this compatible with 6.2.x
KC-2261: timeout websocket connection via jwt token
Update CODEOWNWERS to match latest definition from master
MMA-13013: Override SSL Stores priority
Refactoring- made code more functional and improved readability.
Updated NotFoundException message in scenario where topicName provided returns 0 no consumergroupdata.
Updated code based on PR feedback.
Updated /{consumerGroupId} API to also support topicName queryParam.
Improving test cases with scenarios of single consumergroup covering multiple topics. Updated /{consumerGroupId} to not support topicName queryParam, it is not required.
Adding unit test for CachedConsumerOffsetsResource
MMA-12987, MMA-12439: Fix ControlCenterSecureIntegrationTest Test
adding support for topicName in query param to return only consumer groups for that topic
MMA-12439: Fix ControlCenterSecureIntegrationTest Test
MMA-12980: Fix ConsumeToSocketSerdeTest Test
Pin bc-fips version correctly
MMA-12439: Update Websocket configuration for Control Center if BasePath supplied
Reading password using getPassword instead of getString, this fixes the casting exception
MMA-12895: Add New APIs for querying KRaft mode in KafkaResource
Adding a config to force armeria health check to use HTTP1
Fixing failing test case, due to merge issue
Fixing check style error, removing unused import
Renaming Health_check to health check
MMA-12911: Propagate numPartitions and replication factor from CreateTopicsResult
MMA-12912: Disable offset commits, don’t supply group ID
Adding config based functionality to force http1 for armeria health checks
MINOR: Update repo to use mvn-wrapper to speed up builds
MMA-12804: Re-enable failing tests in SslUtilsTest
Fix codeowners to make c3 default ownens as well
Cherry pick 7746 codewoner
RCCA-7746: CCloud RBAC user unable to view messages from UI
KEXP-349: experiences code owners
RCCA-7746: CCloud RBAC user unable to view messages from UI
MMA-12804: [Temp] Ignore tests due to bouncy castle fips dep [7.2.x, master]
MMA-9023: memory bounding for streams applications [5.4.x, master]
MMA-12768, MMA-12764, MMA-12765: Fix test TelemetryReporterIntegrationTest
MMA-12762, MMA-12624: Add testng dependency
APIF-2959: Use base SSL config for proxy servlets.
Fix Failing Tests due to CC-18163
MMA-12556: Build failure: org.json added checked exception in JSONObject
Add OmgRawDeserializer for use by KafkaApi
Using empty string instead of null to avoid nullpointer exceptions
Improved java doc
Renamed extraPath to urlPath to make the changes more understandable
Checkstyle fixes
Added method for extracting any path present in the url, and added before the healthcheck path and after the healthyUrlsMap urls

MQTT Proxy¶

Migrate test to OpenTelemetry
Remove the pin of netty version to use the (current) version defined in common
Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
Upgrade netty

Metadata Service¶

MMA-13145: Update launchdarkly
API-665: Accept named listeners when deciding if MDS is enabled.
clean testng
[Fix build] Drop down to testNG 7.5 - which is the last version to support jdk8
Fix org.jetbrains dependency
Adapt 7.1.x to testNg 7
clean up surefire
CIAM-2615: - Bump TestNG to 7.7.0
NotNull
Fix NotNull import
Update CP RBAC tests to run on KRaft Clusters
CIAM-2579: update pgsql
CIAM-2572: Add PreStop hook to deployment for graceful shutdown
Add sgscheduleradmin User devel
CIAM-2353: moved all cloud tests to cc-rbac.
CIAM-2581: - Robustify additional roledef file specification
CIAM-2463: - Make and test CCloudAdmin role to allow for Org0 Apikeys
CIAM-2577: Turned off test RbacExtractorMetricsTest
CIAM-2352: Moved some unit tests from module rbac-api-server to cc-rbac
Add sgscheduleradmin User in stag
CIAM-2557: Upgrading base OS for Docker image
CIAM-2504: Propagate publish attributes through call stack
AUTHN-1569: Create trust service db r/w user for prod
CIAM-2420: Add excludePrincipal filter to v2 public list endpoint
KPLATFORM-1531: Add kafkaqueuesadmin user in prod
CIAM-2533: - Ability to specify additional roledef file for cc-rbac
AUTHN-1569: Create trust service db r/w user for stag
AUTHN-1569: create read/write account for trust-service devel
KPLATFORM-1531: Add kafkaqueuesadmin user in stag
Fixed build error relate to FIPS jar for 7.0.x branch
Add schema.sql
Add kafkaqueuesadmin user in devel
CIAM-2527: Lock timeout applied at transaction level for RBAC DB
BUGFIX: “Added describe permission to DataSteward and DataDiscovery”
Use test branch
Bumped prod version to 0.612.0
Bumped devel version to 0.612.0
AUTHN-860: dbmigrate identitypool access for datasci user [PROD]
AUTHN-898: - Enable Trust-Extractor Lifecycle Publish
AUTHN-860: dbmigrate identitypool access for datasci user [STAG]
CDMUM-2408: Add role bindings to flowserviceadmin for identity providers/pools deletion [PROD]
CDMUM-2408: Add role bindings to flowserviceadmin for identity providers/pools deletion [STAG]
CIAM-2475: Allow deletion of rbac role bindings for SR Scope
Increase buffer size for Proxy Servlet
AUTHN-860: dbmigrate identitypool access for datasci user [DEVEL]
CIAM-2507: Add CHC Query Time Metric
CIAM-2507: Increase number of connection in DB connection pool
CDMUM-2408: Add role bindings to flowserviceadmin for identity providers/pools deletion
CIAM-2524: - Re-run all grandfathering scripts
Bump extractor version in prod
CIAM-2495: - Fix KSQL grandfathering queries
Promoted 0.598.0 of cc-rbac to devel
CIAM-2495: - Prod rbacgrandfatheringadmin needs CCloudKsqlRoleBindingAdmin
CIAM-2495: - Devel rbacgrandfatheringadmin needs CCloudKsqlRoleBindingAdmin
CIAM-2495: - Stag rbacgrandfatheringadmin needs CCloudKsqlRoleBindingAdmin
CIAM-2483: Updating to add some logging to cached client
CIAM-2513: Fix CCA SR migration script
CIAM-2483: Add a cached client wrapper
CIAM-2520: Changed maximum number of rbac role bindings to 4000.
CIAM-2506: - Round 2 : Fix SR grandfathering issues found from dry run
Bump MDS version in Devel
Update Launchdarkly
Enabled chcRespnseCache stats
CIAM-2505: CHC Query Counter
CIAM-2494: - Fix the SR grandfathering queries
DGS-5493: Moved DD and DS to different namespace [Updating MDS tests]
CDMUM-2403: CDMUM-2130 Fix getAuthenticationinfo compile issue on the RefreshIdentityProviderKeys api
AUTHN-1012: Handle empty post and patch requests
AUTHN-1291: Create public endpoint for manual jwks refresh
AUTHN-1465: - move ld check inside try catch in case of CHC errors
AUTHN-1226: Enforce 7 day max jwks key refresh period
trust-extractor: throttle CHC requests using local cache
CIAM-2402: Grandfathering script: check ISA field for ‘t’ instead of true
remove accessToken duplicate code
fix string comparion issue
CDMUM-2130: Audit log identity(full crn) for all identityPool/user/serviceAccount access
CIAM-2490: - Fix migration script to include internal service accounts for connectors and schema validation
CIAM-2489: - Split SR and KSQL apikey grandfathering script directories
CIAM-2230: Add CCA SR migration scripts
CIAM-2489: Service accounts with KSQL Apikeys should be grandfathered too
CIAM-2402: Remove owner_type restriction
Minor: resolve manualRepublishError in trust-extractor sds publish path
RCCA-9784: Promote to prod
RCCA-9784: Promote to devel
Add trust-extractor LD flag
Minor: update pool dao to have correct message sequence ids
Fix broken test + adjust manualRepublish to include sds count
Minor: pass provider id to JwtIssuerKey constructor
DGS-5150: Insert sr healthcheck rolebinding
RCCA-9784: Temporarily remove delete of old style JwtIssuer records
APIF-3122: Unpin protobuf version.
AUTHN 1263 - SDS Trust Service Periodic Publish Integration
Move from artifactory for helm
Insert schema registry healthcheck rolebinding
CIAM-2480: Deleting role bindings that are invalid from database to unblock SR test
Promoted 0.564.0 of cc-rbac to prod
AUDIT-1230: Upgrade the java library dependency to v1
CIAM-2462: - ksqscheduleradmin can grant KsqlAdmin for healthchecks
CIAM-2468: - grant flowserviceadmin more roles in prod
Revert Extractor Based JWKS Manual Refresh
AUTHN-1134: Move create audit to parent resource
KFS-251: - added ksqlscheduleradmin to prod
CIAM-2374: Added feature that default page size can be set by LD
Update java chc client
CIAM-2426: Add tests for enforcing checking on org resource id.
CIAM-2230: Create README for RBAC Phase 3 migration scripts
CIAM-2305: Remove grandfatheringadmin whitelist & x-org check from rbac create/delete
DP-9370: - use cc-service-bot to manage Semaphore project
DGS-5219: Delete zombie rolebindings for connect systemtests
Promote CC RBAC Devel to hal-319, helm-553 to include cascade delete
Update chc java client version
CIAM-2305: Exclude rbacgrandfatheringadmin from cross org check
CIAM-2462: - added migration for ksqlscheduleradmin role in devel
CIAM-2462: - added migration for updating role on the ksqlscheduleradmin user
CIAM-2305: Use KsqlAdmin and users._organization_id in KSQL/SR migration scripts
CDX-484: seed for cdx metadata read role for prod
Promote CC-RBAC to Devel to use 0.539.0 to cascade delete resource rolebindings
MMA-12624: Fix testng dependencies
CIAM-2462: - Test Ksql roles
CIAM-2198: Delete Resource Rolebindings fix
CIAM-2468: grant flowserviceadmin permission to delete internal role b
AUDIT-1202: change methodNames in auditLogs and modify version
Added second LD flag, fixed RejectionExecutionException error
Upgrade cc-base to v20.6.0-jdk-17
CIAM-2258: - sds healthcheck v2
CIAM-690: Implement pagination for V2 RBAC Public API list endpoint
CIAM-2464: - Fix master not finding new Protobuf method
CIAM-2258: - extractor - wire up LatencyRecord emitter and CHC
CIAM-2422: Enforce checking on calling org resource id has to match with target org resource id
Applied migration
Renamed migration file to align with prod changes
CIAM-2346: Better log format for jetty request logs’
RCCA-9483: OrgAdmin Reactivation for Halliburton
AUTHN-812: Log to db jwks refresh errors
CIAM-2343: - Validate JWT Token generation using cluster id/name
Unsert schema registry healthcheck rolebinding
MINOR: Promote To Devel 0.539.0
AUTHN-812: generate new jooq objects
AUTHN-812: Make stag/prod migrations for jwks error field
CIAM-2148: Retry failed consumer seek-back in Extractor Listeners
APIF-2959: Use base SSL config for proxy writer.
AUTHN-541: fix: Add clientIp to audit logs (app-sec recommendation)
CIAM-2427: CHC Client Exception Counter
AUTHN-812: Make devel migration for jwks error field
CDX-484: seed for cdx metadata read role for devel
CDX-484: seed for cdx metadata read role for stage
KFS-251: - added ksqlscheduler admin user on stag
CIAM-2320: Decrease role bindings that we publish
CIAM-2414: Support inclusive Lookup of Managed rolebindings for v2alpha1
CIAM-2376: Configured pgaudit for PROD
CIAM-2402: Modify SR api key migration script to support internal SAs
Update pull_request_template.md
AUTHN-1285: Enforce get using provider/org upon read/upd/del
CDX-484: fix role bindings for cdx stream share
CDX-484: Add role bindings for StreamShareMetadataRead role
CIAM-2392: Bump PSQL Version
CIAM 2417: Revert to normal tables
CIAM-2376: Create pgaudit DB role
CDX-469: add db seed for prod environment
DGS-5157: add_schemaregistryvalidationadmin_schemaregistryconnectadmin_role for prod environment
CDX-469: Fix bad ids and seed StreamShareSRRead role binding
CIAM-2376: Added pgaudit config
CIAM-2409: Fix Issue with Org Keys containing null values at the end
CIAM-2124: Update the Extractor to use Batch API
CIAM-2376: Created pgaudit role rds_pgaudit
DGS-5155: add_schemaregistryvalidationadmin_schemaregistryconnectadmin_role for devel environment
CIAM-2396: - Deploy KsqlAdmin to devel
Remove authentication team as codeowner from rbac specific parts
CIAM-2376: Configure pgaudit log
CIAM-2337: Seeking for unassigned partitions cause seek exception
CIAM-2376: Added specific db parameters for MDS
modify tests to include schema registry healthcheck role added
CIAM-2389: Update topics to history tables to include deactivated records
CIAM-2373: Switch to GetPhysicalClusterIdAndK8sClusterIdResponse endpoint from chc
CIAM-2343: - Validate JWT Token generation using cluster id/name
AUTHN-1135: Fix sequencing of provider records
AUTHN-1265: Add force refresh mechanism
CIAM-2307: Write Stage 3 schema registry tests in V2CloudResourceOwnerTest
Stag migration version
Add change to test file
Add_schemaregistryvalidationadmin_schemaregistryconnectadmin_role_binding
RCCA-9245: Undelete rbs deleted in CIAM-1859 w/principals having only inactive admin bindings
CIAM-2376: Created rds_pgaudit db role
AUTHN-1135: Add providerId to JwtIssuer record
CIAM-2265: Handling of null offset data at start bug
CIAM-2338: CHC lifecycle publish type time meter bug
CIAM-2369: Upgrade cc-rbac and extractor version to the latest
CDMUM-2268: Add more unit test cases for public apis using oauth
CDX-469: add db seed for stag environment
AUTHN-992: Create manual jwks refresh endpoint
CIAM-2035: Add KSQL/SR api key migration scripts
CDX-469: add role bindings for SR and stream catalog integration
CIAM-1859: Delete crufty BASIC resource level role bindings for active users
Update CHC object to allow unknown fields
RCCA-9083: Reactivate role bindings deleted by Newswav org delete
AUDIT-1170: Nikhil/audit 1170 app sec suggestions
AUTHN-541: trust-service: Expose principal user id to customer
Upgrade auditlog-emitter-java version
Remove useless varieable
Audit log identityPool and full crn of external user in trust service
CIAM-2263: Refactor tests to be independent of each other and write to different topics
Update the identity provider in response to be identity_provider_id
CIAM-2320: Change RBAC orm DB query to handle exact and descendant scopes
AUDIT-1110: Auditing cc rbac crud
CIAM-2330: Fix Issue with data inconsistency CHC
CIAM-2291: Enable using the physical cluster mapping from CHC messages
Bump postgresql from 42.3.3 to 42.4.1
AUTHN-1212: Skip loading audit config for legacy pkc rbac
EVENTS-1022: Update events-schema to latest
CIAM-2213: - Extract a RepeatedJobRunner from existing Org0 healthcheck
Update sds global Kafka lkc/pkc ids for devel and prod
CIAM-2305: Add MDS local tests for CCloudGrandfatheringRoleBindingAdmin
Updated cc-rbac to 0.465.0(225)
CIAM-2313: A LD Flag for CHC lifecycle
CIAM-2305: Seed rbacgrandfatheringadmin rb dbmigration [devel, stag, prod]
CIAM-1309: Fix undelete endpoints spec
RCCA-9749: Restored Org Admin role for user u-l670z2
Fix fmt issue
Rename algorithmWhitelists
Print identity claim
Add non-sub identity claim test
Use JwtIssuerStatic
Add constraints into the authenticator
Build authenticator on-demand
Fix test failure and syntax issue
Add more assertion in test cases
Add OAuth authentication API to validate OAuth token and corresponding identity pool
CIAM-2214: Add API to get all K8 ids from CHC
CIAM-1692: Delete crufty rolebindings associated with deleted users
CIAM-2318: Deleted RBAC role bindings for deleted clusters
AUDIT-1163: Add clearingHouse api-keys in config
Add more logging to CHC to understand consumption bug and also publish duration
Fix flaky tests due to multithreaded environment for chc listeners
CIAM-2033: Authorize request body validator
Fix the org_resource_id naming
Use underscore in ValidateOAuthRequest and ValidateOAuthResponse variable
CIAM-1931: Meters to capture Extractor consumer/producer failures to consume records from Events/Chc Kafka
Artifactory migration
CIAM-2295: Gating Kafka publish to only be done on lifecycle publish
Update Dockerfile
AUTHN-1147: Skip cts-sdk-go deploy
CIAM-2292: Update cc-rbac/rbac-extractor deployment order to stag->devel->prod
AUTHN-1147: Auto generate trust service cts-sdk-go
CIAM-2288: Temp Workaround: To prevent publishing KSQL and SR rolebindings for new Kafka Cluster Lifecycle flow
CIAM-2285: Upgrade Chc version
CIAM-2243: CLI: Create and Delete command fail with 404 Role Not found for SR/KSQL
AUTHN-965: - Make build work with old and new jvms
CIAM-2212: Publish SDS role bindings to SDS Sync Pipeline instead of Kafka Sync Pipeline
CIAM-1974: Env rolebinding publish for new LC creation
CIAM-1896: Added metrics for LD used by RBAC quotas
Used cluster.id instead of creating new one cloudCluster.id
CIAM-1829: Org rolebinding publish for new LC creation
CIAM-2217: Default to JSON type if accept all header present
Updated v2list type
CIAM-2236: RBAC resource to roles UI endpoint
Fixed yaml ref
CIAM-2240: Refactor CHC and Sync Listeners and Fix tests
Specified content return in yaml
Returns list of role bindings, updated test clarity
Fixed line length errors
Fixed whitespacing
Returns role bindings that were undeleted and throws a 400 error for valid requests but no undeletion
CIAM-1965: Publish K8 ID for RBAC Crud Changes for KSQL/SR with K8 ID from CHC
CIAM-2246: Enforce LD auth.rbac.limits.enable on org resource id for better cont
Updated test to reflect edge cases
Fixed naming and wider tests for undelete endpoints
Update UserUndeleteRequest.java
Update ScopeUndeleteRequest.java
Update release.svg
Update mothership-seed.sql
Update mk-include-git-hash
Updated rbacdb scope tests
Updated rbacdb tests
Automated newReason for both endpoints
Automatic newReason generated, user no longer passes it in
Remade tests with cleaner code, touched up formatting, changed undeleteRequest to userUndeleteRequest
Whitespace issues fixed
Caught malformed reason field at api level
Caught malformed JSON reason field error
Fix tabbing issue on yaml file
Updated v2alpha1.yaml
Changed crnString variable to ‘crn’
Updated tests to reflect that endpoint now checks for valid orgIds and parameters
Cleaned up spacing and added safety comments
Changed validators for scopeUndelete
Added validators and removed debug print lines
Test no authorizor
Add debug lines
Debug roles endpoint, testing docker image creation
Remove debug print statement
Added undelete by scope
Update RbacOrmDbService.java
Update DuplicateRequest.java
Created and tested user undelete function

Replicator¶

used spotbugs version variable instead
use spotbugs annotations instead of findbugs
resolving compile issue
KGLOBAL-3224 removed find bugs usage
remove aws-maven plugin to unblock the downstream validation for common: RCCA-10412, APPSEC-2551
Fixed deprecated method usage
Changed to 5 sec to honor TASK_SHUTDOWN_GRACEFUL_TIMEOUT_MS_CONFIG
Changed default to 120 sec to match topic config sync
KGLOBAL-3060: added consumer poll timeout ms property
Resolve compile issues created by change in signature due to KAFKA-14346
KGLOBAL-2436: Don’t seek to beginning on pause for schema translator
REPL-2055-: Log WakeupException at debug level