Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 7.3.2¶

Released February 2023

Commercial Features¶

Security¶

Security Fixes

Version 7.3.1¶

Released December 2022

Community Features¶

Common¶

PR-492 - APIF-3112: Upgrade jmx_prometheus_javaagent to 0.17.2.
f5764727 - Merge conflict resolution.
PR-489 - APIF-3117: Upgrade scala to 2.13.10.
PR-485 - APIF-2957: Upgrade Jackson and Protobuf versions.

Kafka¶

PR-845 - CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
PR-842 - DP-9030: Use the new withGradleFile closure
PR-12809 - [KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)
PR-829 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
PR-12836 - KAFKA-14282: stop tracking Produced sensors by processor node id (#12836)
PR-816 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
PR-12794 - Revert “KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)” (#12794)
PR-12790 - KAFKA-14337; Correctly remove topicsWithCollisionChars after topic deletion (#12790)
PR-12765 - KAFKA-14316; Fix feature control iterator metadata version handling (#12765)
PR-12747 - KAFKA-14300; Generate snapshot after repeated controller resign (#12747)
PR-12676 - KAFKA-14209 : Integration tests 3/3 (#12676)
PR-12741 - KAFKA-14296; Partition leaders are not demoted during kraft controlled shutdown (#12741)
PR-12736 - KAFKA-14292; Fix KRaft controlled shutdown delay (#12736)
PR-12709 - KAFKA-14275; KRaft Controllers should crash after failing to apply any metadata record (#12709)
PR-12634 - KAFKA-14225; Fix deadlock caused by lazy val exemptSensor (#12634)
e23c59d0 - Bump version to 3.3.1
1780f266 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
4b35f247 - Bump 3.3 branch to 3.3.1-SNAPSHOT

ksqlDB¶

PR-9713 - Bump netty tcnative version to 2.0.54.Final
PR-9708 - APIF-3117: Unpin scala version.
PR-9697 - APIF-3122: Unpin Protobuf version.

REST Proxy¶

PR-1075 - Fix the line longer than 100 characters issue
PR-1072 - KREST-7107 Shouldn’t provide overly detailed info or stack traces in responses -

Schema Registry¶

PR-2467 - changed the header name of pool id
PR-2460 - DGS-5544 Support nulls when using optional fields in proto2
PR-2459 - DGS-5400 Support subjectPrefix containing wildcard context and subject
PR-2456 - Derive schema maven plugin 7.3.x
87defb47 - Exposed the method to schema registry client to use in security-plugins
PR-2450 - Pin Snakeyaml version
PR-2443 - DGS-5423 Check subject when looking up ID across contexts
PR-2441 - * Minor: change OAuth config string
PR-2440 - Update the Snakeyaml dependency
PR-2409 - DGS-3721 SaslOAuthCredentialProvider
PR-2371 - DGS-3396: Add deletedOnly=true to SR list APIs
8e5ce168 - Unpin Protobuf version.
PR-2421 - DGS-5220 Handle nested extend decls in Protobuf
PR-2415 - DGS-5254 Make timeouts configurable when forwarding requests
PR-2414 - DGS-5253 Handle cycles when resolving references
PR-2344 - DGS-3719, DGS-3720 Added OAuthCredentialProvider and CachedOauthTokenRetriever
PR-2407 - MINOR: Fix OpenAPI deleteSubjectConfig example
PR-2405 - DGS-4754 Handle empty record default at field level

Commercial Features¶

Confluent Server¶

remove extraneous }
CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
DP-9030 - Switch to codeartifact repo
merge from ccs:3.3 to ce:7.3.x
DGS-5248 Adding SR client OAuth config for Schema Validation.
KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error messa
Increase vagrant aws timeout for system test
Upgrade from Scala 2.13.8 to 2.13.10
Upgrade to snakeyaml version 1.32
Upgrade jackson-databind version to 2.13.4.2
MINOR: TierArchiver - improve logging for cancelled tasks
KGLOBAL-2143: Return error message containing all racks that dont have enough brokers during topic creation validation
KGLOBAL-2122: Validate there are enough brokers to satisfy updated topic placement
Upgrade to protobuf version 3.19.6
CONFLUENT: Revert protobuf upgrade change
CIAM-2424 - Name RestClient threads for stackdump identification (#7887)
CPKAFKA-9173: Disable colocated KRaft upgrade from CP 7.1/7.2
Bump version to 3.3.1
KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
Bump 3.3 branch to 3.3.1-SNAPSHOT

Security¶

DGS-5542: Incorporated code review comments and removed the unused parameter schemaRegistryResourceOperation
DGS-5542: SchemaId validation for numeric value in authorizer filter
fixed indentation
Update AuthorizationFilter.java
Changed from 5 operations to 1
updated tests
added null when schema id does not exist
changed the logic of authorizeSchemaIdLookup
changed to Short
dummy commit for new build
- Adding OAuth config support for Schema validation
Exclude bcpkix-jdk15on as compile dep and add as test dependency
Pin bc-nonfips dep as a test dependency
Pin bouncycastle fips dependency in confluent-security-plugins
Removed bcfips from common
excluded bcfips jars
Removed From Supported Operations to follow RBAC Rules
Added fix for unit test
Changed the design of RBAC for Exporters
Fixed import
Removed extra line changes
Added RBAC for Schema linking Phase 1
AUTHN-1087: Explicit define netty versions in kafka-rest plugin
AUTHN-1087: [7.0.x only] Enforce Netty versions in kafka-rest plugin
DGS-3396: introduce LookupFilter for list apis
AUTHN-1087: Upgrade vertx to 3.9.14
AUTHN-1325: Update bc-fips version to 1.0.2.3
Fix bouncycastle issue
Exclude bouncycastle non-fips jar from connect packaging
Revert “Exclude bcpkix-jdk15on as compile dep and add as test dependency”
Revert “Pin bouncycastle fips dependency in confluent-security-plugins”
Revert “Removed bcfips from common”
fixed checkstyle error
Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas

Secret Registry¶

Exclude Non-FIPS dep and use bc-fips dependency

Control Center¶

RCCA-7746: CCloud RBAC user unable to view messages from UI
MMA-9023: memory bounding for streams applications [5.4.x, master]
[MMA-12762][MMA-12624] Add testng dependency
Fix Failing Tests due to CC-18163
Revert “MMA-12206 upgrade io.netty:netty-codec-http”

MQTT Proxy¶

Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
Upgrading netty

Metadata Service¶

Update Launchdarkly
APIF-3122: Unpin protobuf version.
MMA-12624 Fix testng dependencies
CIAM-2392 Bump PSQL Version

Replicator¶

KGLOBAL-2436: Don’t seek to beginning on pause for schema translator
REPL-2055-: Log WakeupException at debug level

Version 7.3.0¶

Released October 2022

Community Features¶

Common¶

aeca780b - Update Jolokia version in pom.xml
PR-472 - Upgrade Netty to 4.1.79.Final
PR-471 - fix checkstyle
PR-468 - APPSEC-1393: Fix dependency in disk-usage-agent [6.2 and 7.0]
PR-464 - APPSEC-1393:Migrate from confluent-log4j to reload4j [5.4.x – 7.0.x]
PR-465 - APPSEC-1393: Confluent log4j to reload4j [7.1.x Only]
PR-467 - APPSEC-1412: Upgrade JUNIT in 5.4.x only
PR-442 - Introduce Pull Request Reviewers

Kafka¶

eefe8671 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
PR-12642 - KAFKA-14207; KRaft Operations documentation (#12642)
PR-12681 - KAFKA-14259: BrokerRegistration#toString throws an exception, terminating metadata replay (#12681)
9d1f9f77 - Bump version to 3.3.0
PR-12628 - KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
PR-12664 - KAFKA-14243: Temporarily disable unsafe downgrade (#12664)
PR-12653 - KAFKA-14240; Validate KRaft snapshot state on startup (#12653)
PR-12658 - KAFKA-14233: Disable testReloadUpdatedFilesWithoutConfigChange first to fix the build (#12658)
PR-12655 - KAFKA-14238; KRaft metadata log should not delete segment past the latest snapshot (#12655)
PR-12570 - KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
PR-12625 - KAFKA-14222; KRaft’s memory pool should always allocate a buffer (#12625)
PR-12626 - KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626)
PR-12603 - KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603)
PR-12624 - KAFKA-14215; Ensure forwarded requests are applied to broker request quota (#12624)
b2639c8d - Remove the html end tag from upgrade.html
PR-12597 - KAFKA-14205; Document how to replace the disk for the KRaft Controller (#12597)
PR-12596 - KAFKA-14203 Disable snapshot generation on broker after metadata errors (#12596)
PR-12617 - KAFKA-14216: Remove ZK reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc (#12617)
PR-12618 - KAFKA-14217: app-reset-tool.html should not show –zookeeper flag that no longer exists (#12618)
PR-12609 - KAFKA-14198; swagger-jaxrs2 dependency should be compileOnly (#12609)
PR-12584 - KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584)
PR-12604 - KAFKA-14188; Getting started for Kafka with KRaft (#12604)
PR-12599 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (#12599)
PR-12598 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (server side) (#12598)
PR-12595 - KAFKA-14204: QuorumController must correctly handle overly large batches (#12595)
PR-11783 - KAFKA-14143: Exactly-once source connector system tests (#11783)
PR-12586 - KAFKA-14200: kafka-features.sh must exit with non-zero error code on error (#12586)
PR-12578 - KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case (#12578)
PR-12533 - KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Controller (#12533)
PR-12294 - KAFKA-13990: KRaft controller should return right features in ApiVersionResponse (#12294)
PR-12571 - KAFKA-14187: kafka-features.sh: add support for –metadata (#12571)
PR-12565 - KAFKA-14183; Cluster metadata bootstrap file should use header/footer (#12565)
PR-12513 - KAFKA-14177: Correctly support older kraft versions without FeatureLevelRecord (#12513)
PR-12103 - KAFKA-13850: Show missing record type in MetadataShell (#12103)
PR-12551 - KAFKA-14178 Don’t record queue time for deferred events (#12551)
PR-12469 - KAFKA-13914: Add command line tool kafka-metadata-quorum.sh (#12469)
PR-12508 - KAFKA-13888; Implement LastFetchTimestamp and in LastCaughtUpTimestamp for DescribeQuorumResponse [KIP-836] (#12508)
PR-12518 - KAFKA-14167; Completion exceptions should not be translated directly to error codes (#12518)
PR-12517 - KAFKA-13940; Return NOT_LEADER_OR_FOLLOWER if DescribeQuorum sent to non-leader (#12517)
PR-12491 - KAFKA-14148: Update ResetOffsetsDoc (#12491)
PR-12514 - KAFKA-14154; KRaft controller should return NOT_CONTROLLER if request epoch is ahead (#12514)
PR-12274 - KAFKA-13959: Controller should unfence Broker with busy metadata log (#12274)
PR-12506 - KAFKA-14154; Return NOT_CONTROLLER from AlterPartition if leader is ahead of controller (#12506)
PR-12498 - KAFKA-13986; Brokers should include node.id in fetches to metadata quorum (#12498)
PR-12184 - Fix the rate window size calculation for edge cases (#12184)
PR-12487 - KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode (#12487)
f3cf6db3 - KAFKA-14114: Add Metadata Error Related Metrics
PR-12396 - KAFKA-14051: Create metrics reporters in KRaft remote controllers (#12396)
PR-12403 - KAFKA-13166 Fix missing ControllerApis error handling (#12403)
PR-12467 - KAFKA-14129: KRaft must check manual assignments for createTopics are contiguous (#12467)
PR-12447 - KAFKA-14124: improve quorum controller fault handling (#12447)
c2422f63 - Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
PR-12489 - KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing partition epoch (#12489)
PR-12457 - KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
PR-12440 - KAFKA-14107: Upgrade Jetty version (#12440)
PR-12483 - KAFKA-14136 Generate ConfigRecord for brokers even if the value is unchanged (#12483)
ba219265 -: upgrading netty to v4.1.79.Final #754
PR-12429 - KAFKA-14089: Only check for committed seqnos after disabling exactly-once support in Connect integration test (#12429)
PR-12415 - KAFKA-14079 - Ack failed records in WorkerSourceTask when error tolerance is ALL (#12415)
PR-12374 - KAFKA-14039 Fix AlterConfigPolicy usage in KRaft (#12374)
PR-12411 - KAFKA-14078; Do leader/epoch validation in Fetch before checking for valid replica (#12411)
PR-12433 - KAFKA-14093: Use single-worker Connect cluster when testing fenced leader recovery (#12433)
PR-12347 - KAFKA-13919: expose log recovery metrics (#12347)
PR-12408 - KAFKA-14076: Fix issues with KafkaStreams.CloseOptions (#12408)
PR-12365 - KAFKA-14020: Performance regression in Producer (#12365)
PR-12349 - KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance (#12349)
PR-12421 - Revert “KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)” (#12421)
PR-12420 - KAFKA-13769 Fix version check in SubscriptionJoinForeignProcessorSupplier (#12420)
PR-12405 - KAFKA-13572 Fix negative preferred replica imbalanced count metric (#12405)
PR-10964 - KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
PR-12265 - KAFKA-13968: Fix 3 major bugs of KRaft snapshot generating (#12265)
PR-12398 - KAFKA-14062: OAuth client token refresh fails with SASL extensions (#12398)
PR-12390 - KAFKA-14055; Txn markers should not be removed by matching records in the offset map (#12390)
PR-12381 - KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection (#12381)
PR-12296 - KAFKA-13996: log.cleaner.io.max.bytes.per.second can be changed dynamically (#12296)
PR-12359 - KAFKA-13983: Fail the creation with “/” in resource name in zk ACL (#12359)
PR-12091 - KAFKA-12943: update aggregating documentation (#12091)
PR-12297 - KAFKA-13846: Follow up PR to address review comments (#12297)
PR-12337 - KAFKA-10199: Remove main consumer from store changelog reader (#12337)
PR-12360 - KAFKA-14032; Dequeue time for forwarded requests is unset (#12360)
PR-12379 - KAFKA-10199: Remove call to Task#completeRestoration from state updater (#12379)
PR-12224 - KAFKA-13943; Make LocalLogManager implementation consistent with the RaftClient contract (#12224)
PR-11782 - KAFKA-10000: Integration tests (#11782)
PR-11784 - KAFKA-13228; Ensure ApiVersionRequest is properly handled KRaft co-resident mode (#11784)
PR-11894 - KAFKA-13613: Remove hard dependency on HmacSHA256 algorithm for Connect (#11894)
PR-12376 - Upgrade Netty and Jackson versions [KAFKA-14044] (#12376)
PR-12372 - KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
PR-12371 - KAFKA-14035; Fix NPE in SnapshottableHashTable::mergeFrom() (#12371)
PR-12204 - [9/N][Emit final] Emit final for session window aggregations (#12204)
PR-12329 - KAFKA-14010: AlterPartition request won’t retry when receiving retriable error (#12329)
PR-12139 - KAFKA-13821: Update Kafka Streams WordCount demo to new Processor API (#12139)
PR-12293 - KAFKA-13963: Clarified TopologyDescription JavaDoc for Processors API forward() calls (#12293)
PR-12312 - KAFKA-10199: Expose tasks in state updater (#12312)
PR-12279 - KAFKA-10199: Commit the restoration progress within StateUpdater (#12279)
PR-12269 - KAFKA-13966 Prepend bootstrap metadata to controller queue (#12269)
PR-12291 - KAFKA-13987: Isolate REST request timeout changes in Connect integration tests (#12291)
PR-12209 - KAFKA-13930: Add 3.2.0 Streams upgrade system tests (#12209)
PR-11781 - KAFKA-10000: Per-connector offsets topics (#11781)
PR-10738 - KAFKA-6945: KIP-373, allow users to create delegation token for others (#10738)
PR-12298 - KAFKA-13998: JoinGroupRequestData ‘reason’ can be too large (#12298)
PR-12304 - KAFKA-13880: Remove DefaultPartitioner from StreamPartitioner (#12304)
PR-12226 - KAFKA-13890: Improve documentation of ssl.keystore.type and ssl.truststore.type (#12226)
PR-12263 - KAFKA-13939: Only track dirty keys if logging is enabled. (#12263)
PR-12161 - KAFKA-13873 Add ability to Pause / Resume KafkaStreams Topologies (#12161)
PR-12206 - KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
PR-12287 - KAFKA-13846: Use the new addMetricsIfAbsent API (#12287)
PR-12248 - KAFKA-13958: Expose logdirs total/usable space via Kafka API (KIP-827) (#12248)
PR-12181 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2) (#12181)
PR-12250 - KAFKA-13935 Fix static usages of IBP in KRaft mode (#12250)
PR-12121 - KAFKA-13846: Adding overloaded metricOrElseCreate method (#12121)
PR-11473 - KAFKA-13436: Omitted BrokerTopicMetrics metrics in the documentation (#11473)
PR-11780 - KAFKA-10000: Exactly-once source tasks (#11780)
PR-12140 - KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)
PR-12264 - KAFKA-13967: Document guarantees for producer callbacks on transaction commit (#12264)
PR-11779 - KAFKA-10000: Zombie fencing logic (#11779)
PR-12267 - KAFKA-13947: Use %d formatting for integers rather than %s (#12267)
PR-12197 - KAFKA-13929: Replace legacy File.createNewFile() with NIO.2 Files.createFile() (#12197)
PR-12067 - KAFKA-13780: Generate OpenAPI file for Connect REST API (#12067)
PR-12180 - KAFKA-13917: Avoid calling lookupCoordinator() in tight loop (#12180)
PR-12270 - KAFKA-10199: Implement removing active and standby tasks from the state updater (#12270)
PR-12245 - KAFKA-13410; Add a –release-version flag for storage-tool (#12245)
PR-12240 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (#12240)
PR-12235 - KAFKA-13945: add bytes/records consumed and produced metrics (#12235)
PR-11778 - KAFKA-10000: Use transactional producer for leader-only writes to the config topic (#11778)
PR-12159 - KAFKA-13933: Fix stuck SSL unit tests in case of authentication failure (#12159)
PR-12170 - KAFKA-13875 Adjusted the output the topic describe output to include TopicID & se… (#12170)
PR-12238 - KIP-835: metadata.max.idle.interval.ms shoud be much bigger than broker.heartbeat.interval.ms (#12238)
PR-12005 - KAFKA-13803: Refactor Leader API Access (#12005)
PR-12210 - KAFKA-13930: Add 3.2.0 to core upgrade and compatibility system tests (#12210)
PR-12246 - KAFKA-13718: kafka-topics describe topic with default config will show segment.bytes overridden config (#12246)
PR-11776 - KAFKA-10000: Add new preflight connector config validation logic (#11776)
PR-12191 - KAFKA-12657: Increase timeouts in Connect integration tests (#12191)
PR-12136 - KAFKA-13773: catch kafkaStorageException to avoid broker shutdown directly (#12136)
PR-12084 - KAFKA-13845: Add support for reading KRaft snapshots in kafka-dump-log (#12084)
PR-12183 - KAFKA-13883: Implement NoOpRecord and metadata metrics (#12183)
PR-721 - Update CODEOWNERS
PR-12225 - KAFKA-13946; Add missing parameter to kraft test kit ControllerNode.setMetadataDirectory() (#12225)
PR-10830 - KAFKA-12902: Add unit32 type in generator (#10830)
PR-12187 - KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (#12187)
PR-12062 - KAFKA-13833: Remove the min_version_level from the finalized version range written to ZooKeeper (#12062)
PR-12200 - KAFKA-10199: Implement adding standby tasks to the state updater (#12200)
PR-12035 - KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so (#12035)
PR-12190 - KAFKA-13923; Generalize authorizer system test for kraft (#12190)
PR-12160 - KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD (#12160)
PR-12109 - KAFKA-13863; Prevent null config value when create topic in KRaft mode (#12109)
PR-12150 - KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
PR-12050 - KAFKA-13830 MetadataVersion integration for KRaft controller (#12050)
PR-12165 - KAFKA-13905: Fix failing ServerShutdownTest.testCleanShutdownAfterFailedStartupDueToCorruptLogs (#12165)
PR-12162 - KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs (#12162)
PR-11748 - KAFKA-12635: Don’t emit checkpoints for partitions without offset-syncs (#11748)
PR-12164 - Update note on upgrade from log4j to reload4j (#12164)
PR-12087 - KAFKA-13851: Add integration tests for DeleteRecords API (#12087)
PR-11916 - KAFKA-12703; Allow unencrypted private keys when using PEM files (#11916)
PR-12135 - KAFKA-13785: [7/N][Emit final] emit final for sliding window (#12135)
PR-11969 - KAFKA-13649: Implement early.start.listeners and fix StandardAuthorizer loading (#11969)
PR-11775 - KAFKA-10000: Add all public-facing config properties (#11775)
040b11d7 - KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
PR-12108 - KAFKA-13862; Support Append/Subtract multiple config values in KRaft mode (#12108)
PR-12131 - KAFKA-13879: Reconnect exponential backoff is ineffective in some cases (#12131)
PR-12085 - KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
PR-12010 - KAFKA-13793: Add validators for configs that lack validators (#12010)
PR-11983 - KAFKA-13763: Refactor IncrementalCooperativeAssignor for improved unit testing (#11983)
PR-710 - : Update jackson packages to 2.13.2
PR-12049 - KAFKA-10888: Sticky partition leads to uneven produce msg (#12049)
PR-12028 - KAFKA-13804: Output the reason why broker exit unexpectedly during startup (#12028)
PR-11773 - KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618) (#11773)
PR-12127 - KAFKA-13785: [8/N][emit final] time-ordered session store (#12127)
PR-12128 - KAFKA-10199: Implement adding active tasks to the state updater (#12128)
PR-12029 - KAFKA-13815: Avoid reinitialization for a replica that is being deleted (#12029)
PR-12106 - KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
PR-709 - : Update jackson packages to 2.13.2
PR-12100 - KAFKA-13785: [6/N][Emit final] Copy: Emit final for TimeWindowedKStreamImpl (#12100)
PR-12072 - KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
PR-12111 - KAFKA-13865: Fix ResponseSendTimeMs metric in RequestChannel is removed twice (#12111)
PR-11955 - KAFKA-12380 shutdown Executor in Connect’s Worker when closed (#11955)
PR-12096 - KAFKA-13794: Fix comparator of inflightBatchesBySequence in TransactionsManager (round 3) (#12096)
PR-12064 - KAFKA-12841: Remove an additional call of onAcknowledgement (#12064)
PR-12092 - KAFKA-13834: add test coverage for RecordAccumulatorTest (#12092)
PR-12090 - KAFKA-13852: Kafka Acl documentation bug for wildcard ‘*’ (#12090)
PR-12075 - KAFKA-13841: Fix a case where we were unable to place on fenced brokers in KRaft mode (#12075)
PR-12066 - KAFKA-13834: fix drain batch starving issue (#12066)
PR-11703 - KAFKA-13588: consolidate changelogFor methods to simplify the generation of internal topic names (#11703)
PR-12030 - KAFKA-13785: [5/N][emit final] cache for time ordered window store (#12030)
PR-12052 - KAFKA-13799: Improve documentation for Kafka zero-copy (#12052)
PR-12004 - KAFKA-10095: Add stricter assertion in LogCleanerManagerTest (#12004)
PR-12063 - KAFKA-13835: Fix two bugs related to dynamic broker configs in KRaft (#12063)
PR-11993 - KAFKA-13654: Extend KStream process with new Processor API (#11993)
PR-11681 - KAFKA-8785: fix request timeout by waiting for metadata cache up-to-date (#11681)
PR-12033 - KAFKA-13807: Fix incrementalAlterConfig and refactor some things (#12033)
PR-11945 - KAFKA-13769: Explicitly route FK join results to correct partitions (#11945)
PR-12055 - [MINOR] Update upgrade documentation for 3.2 (#12055)
PR-12036 - KAFKA-13823 Feature flag changes from KIP-778 (#12036)
PR-10472 - KAFKA-12613: Fix inconsistent validation logic between KafkaConfig and LogConfig (#10472)
87aa8259 - KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
PR-12031 - KAFKA-13651; Add audit logging to StandardAuthorizer (#12031)
PR-12018 - KAFKA-13542: Add rebalance reason in Kafka Streams (#12018)
PR-12043 - KAFKA-13828; Ensure reasons sent by the consumer are small (#12043)
PR-11948 - KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
PR-11939 - KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)
PR-12006 - KAFKA-13794: Follow up to fix producer batch comparator (#12006)
PR-11998 - KAFKA-13801: Kafka server does not respect MetricsReporter contract for dynamically configured reporters (#11998)
PR-11842 - KAFKA-13687: Limiting the amount of bytes to be read in a segment logs (#11842)
PR-11997 - KAFKA-6204 KAFKA-7402 ProducerInterceptor should implement AutoCloseable (#11997)
PR-11974 - KAFKA-13763: Improve unit testing coverage and flexibility for IncrementalCooperativeAssignor (#11974)
PR-11995 - KAFKA-13782; Ensure correct partition added to txn after abort on full batch (#11995)
PR-11991 - KAFKA-13794; Fix comparator of inflightBatchesBySequence in TransactionManager (#11991)
PR-11965 - KAFKA-13778: Fetch from follower should never run the preferred read replica selection (#11965)
PR-11981 - KAFKA-13791: Fix potential race condition in FetchResponse#`fetchData` and forgottenTopics (#11981)
PR-11941 - KAFKA-13749: CreateTopics in KRaft must return configs (#11941)
f68f1a97 - Add muckrake mapping for 7.2 release
baf8976f - Add muckrake mapping for 7.2 release
PR-11978 - KAFKA-13786: Add a note in`control.plane.listener.name` doc (#11978)
PR-11950 - KAFKA-12875: Change Log layer segment map mutations to avoid absence of active segment (#11950)
PR-11829 - KAFKA-13785: add processor metadata to be committed with offset (#11829)
PR-11928 - fix: make sliding window works without grace period (#kafka-13739) (#11928)
PR-11953 - KAFKA-13772: Partitions are not correctly re-partitioned when the fetcher thread pool is resized (#11953)
PR-11971 - KAFKA-13783; Remove reason prefixing in JoinGroupRequest and LeaveGroupRequest (#11971)
PR-11963 - KAFKA-13777: Fix potential FetchResponse#responseData race condition issue (#11963)
PR-11908 - KAFKA-13748: Do not include file stream connectors in Connect’s CLASSPATH and plugin.path by default (#11908)
PR-11743 - KAFKA-13660: Switch log4j12 to reload4j (#11743)
PR-11962 - KAFKA-13775: - Upgrade jackson-databind to 2.12.6.1 (#11962)
PR-11967 - Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
PR-11869 - KAFKA-13719: Fix connector restart cause duplicate tasks (#11869)
PR-11966 - KAFKA-13418: Support key updates with TLS 1.3 (#11966)
PR-11923 - KAFKA-6718: Add documentation for KIP-708 (#11923)
PR-11942 - KAFKA-13767; Fetch from consumers should return immediately when preferred read replica is defined by the leader (#11942)
PR-11760 - KAFKA-13600: Kafka Streams - Fall back to most caught up client if no caught up clients exist (#11760)
PR-11949 - KAFKA-4801: don’t verify assignment during broker up and down in testConsumptionWithBrokerFailures (#11949)
PR-11946 - KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (#11946)
PR-11805 - KAFKA-13692: include metadata wait time in total blocked time (#11805)
PR-11940 - KAFKA-13689: optimize the log output of logUnused method (#11940)
PR-11920 - KAFKA-13672: Race condition in DynamicBrokerConfig (#11920)
PR-11926 - KAFKA-13714: Fix cache flush position (#11926)
PR-11933 - KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (#11933)
PR-11892 - [Emit final][4/N] add time ordered store factory (#11892)
PR-11932 - Revert “KAFKA-7077: Use default producer settings in Connect Worker (#11475)” (#11932)
PR-11912 - KAFKA-13752: Uuid compare using equals in java (#11912)
PR-11796 - KAFKA-13152: Replace “buffered.records.per.partition” with “input.buffer.max.bytes” (#11796)

ksqlDB¶

PR-9468 - remove cc-docker-ksql from downstream builds
PR-9567 - fix: use resolved configs in precondition checker
PR-9394 - fix: fix regex used to extract queryId from threadId metrics tag
PR-9392 - fix: move udf loading to run before the precondition checker
PR-9391 - fix: make sure to close clients from precondition checker
PR-9388 - fix: compare topics not sources
PR-9393 - fix: use internal topic config for transient queries too
PR-9389 - refactor: remove AVRO_SCHEMA_ID & SCHEMA_ID from QTT historical plans
PR-9378 - feat: Add support for four and five column arguments to UDAFs
PR-9361 - feat: UDAFs with multiple/variadic args
PR-9366 - feat: Add log, power, and cbrt UDFs
PR-9351 - refactor: combine yatt input and output nodes into one topic node
PR-9341 - fix: make api client recognize ddl warnings better
PR-9360 - refactor: Materialized to MaterializedFactory
ef65f924 - Addressed Jim’s comments
PR-9336 - fix: Map invalid casts to null.
3d2a56f1 - Updated documentation for detailed processing log in KsqlDB.io project
PR-9337 - bugfix: remove log4j from the classpath (#9334)
PR-9321 - fix: allow YATT to insert into and check contents of DDL sources
e986f668 - fix: Create a KsqlSerializationException class
56dddbb1 - fix: classify KsqlSerializationException as USER error based on topic(KSE-1045)
PR-9327 - Bump changelog version heading to 0.27.1
PR-9130 - fix: use JsonSchemaConverter to support JSON anyOf types
PR-9314 - fix: Allows functions which return maps to be dereferenced again.
6f656c0a - fix spotbugs
PR-9283 - feat: Adding ksqlDB Query Status metric.
a700c7ec - refactor: rename getAuthToken to getAuthHeader
PR-9300 - feat: refresh service context and topic client in precondition checker
PR-9272 - refactor: Migrate legacy UDAFs to use current annotations
PR-9203 - feat: Support pausing/resuming persistent queries
dcfe7941 - fix: Return proper status code for QPS ratelimit.
PR-9277 - fix: DESCRIBE FUNCTION failing for annotated UDAFs with initial args
PR-9255 - fix: change auth token provider to accept token strings instead of principals
PR-9260 - fix: Excludes Guava from Guava-retrying in order to manage Guava depe
PR-9246 - MINOR: improve error message for missing key
PR-9248 - fix: Removing reverted configuration org.apache.kafka.streams.Streams
PR-9239 - fix: add getAuthToken method to AuthenticationPlugin interface
PR-9141 - feat: enable new emit-final implementation
PR-9225 - fix: change consumer_group_member_id tag to just member to match Druid label name
PR-9213 - feat: Added numerous trigonometric UDFs
PR-9209 - fix:CAST function works with ISO-8601 timestamps with a trailing ‘Z’
PR-9215 - fix: convert topic tag name and add consumer group member id tag to ThroughputTotalMetrics
PR-9211 - fix: change group name and extend CumulativeSum in ThroughputMetricsReporter
PR-9168 - feat: introduce ATTR aggregation function
PR-9205 - fix: reset collector before reconfiguring
PR-9180 - fix: add BYTES support for KAFKA format
PR-9186 - fix: Allows results from CAST to compared.
PR-9167 - fix: ambiguous reference to close issue
PR-9134 - feat: cull the list of API consumable/editable properties
PR-9144 - fix: move misplaced query-level configs to the correct list
PR-9145 - fix: revert default /query-stream Content-Type to application/vnd.ksqlapi.delimited.v1 from application/vnd.ksql.v1+protobuf
PR-9127 - fix: Fixes a few null handling bugs
PR-9103 - feat: add ProtoBuf as a content type for pull queries over /query-stream endpoint
PR-9045 - feat: add metric for query restarts
PR-9120 - feat: Support all wildcard (*) on struct reference syntax
PR-9105 - feat: clean up processing log metric
PR-9107 - feat: add support for assert statements to migration tool
PR-9099 - feat: add assert methods to java client
PR-9035 - feat: add metric that’s emitted when processing log emits an error
PR-9096 - feat: automatically build confluent cloud image on every master merge
PR-9036 - fix: re fetch streams for each materializationProviderBuilder
PR-9091 - feat: add ASSERT SCHEMA statement
PR-9078 - Add PROTOBUF_NOSR
5423da9d - refactor: Fix checkstyle & make naming consistent
PR-9086 - feat: assert not exists topic
be09c0a6 - refactor: Set supportedArgs with string & bytes for max/min agg functions
PR-9072 - fix: classify SR missing subject and access rights query errors as USER errors
PR-9066 - feat: add ASSERT TOPIC command
030f2147 - feat: enable max/min udaf for string & bytes data types
f2877e8d - fix: classify KsqlFunctionException as USER error
9e9d10e7 - fix: throw KsqlFunctionException while aggregating in sum udaf #9052
1bb24c31 - feat: migrate java client to use application/vnd.ksql.v1+json format
PR-9047 - fix: INSERT/VALUES on a stream with SCHEMA_ID/SCHEMA_FULL_NAME fails
PR-9026 - feat: support checking preconditions before starting core app
PR-9040 - fix: use the engine’s KsqlConfig to build queries
PR-9038 - fix: INSERT fails when serializing Proto/Avro nested Structs
PR-9041 - build: exclude reload4j
PR-9032 - fix: register state listener after restarting runtime
PR-8986 - feat: allow aggregations without group bys
PR-9028 - fix: remove double quotes from json_records function
PR-8933 - fix: Create stream fails when multiple Protobuf schema definitions exist
PR-9023 - fix: include header columns when injecting schemas
PR-8918 - fix: Guard null struct dereferencing inside function calls
PR-8984 - fix: INSERT VALUES fail when SR schema has a non-default name
PR-9014 - fix: fail validation on create connector if connector already exists
PR-8923 - fix: shared runtimes calculate cache size for validation properly
PR-8999 - fix: move create connector validation to validate phase
PR-8998 - fix: remove ErrorEntity and throw on connector error instead
PR-8983 - Revert “feat: Allow to plug-in custom error handling for Connect serv
PR-8977 - Improved/fixed aggregate function error messages.
PR-8949 - feat: allow STREAMS with no key
PR-8926 - fix: Repartition RHS of a FK join if it uses SR schema
PR-8973 - fix: wait longer while waiting for expected spq
PR-8947 - revert: consistency APIs

REST Proxy¶

PR-1049 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-1044 - APIF-2768: Workaround for “Failed to bind to 0.0.0.0/0.0.0.0:9998” error.
87452021 - Run mvn spotless:apply
PR-1039 - KREST-2655 Simple version of disconnect
PR-1006 - KREST-4932: Add produce record rate-limited metrics.
PR-1023 - KREST-5637 Use a Meter as recommended by observability and the right Metrics Object
PR-1024 - Fix Checkstyle empty catch block error.
PR-1022 - KREST-4067 Introduce import control to Kafka REST.
PR-978 - KREST-4591 add topic recreate logic to cluster test harness
PR-985 - KREST-4687 avro consume still fails
PR-1009 - KREST-5732 Refactor to allow us to override producer metrics in ce-kafka-rest
PR-1014 - KREST-5830: Create a lazy wrapper around MappingIterator for Produce Action.
PR-1012 - KREST-5637 Move to cumulative sum for billing metrics
PR-1008 - KREST-5637 Add count based byte metrics and move tracking earlier
PR-1002 - KREST-5385: Add error_code to produce responses.
PR-1003 - Remove ProducerPool.
PR-1001 - Remove KafkaRestContext#getProducerPool.
PR-350 - Cherry-pick https://github.com/confluentinc/rest-utils/pull/349 to 5.4.x
3d1250af - APIF-2739: Upgrade Jetty to 9.4.48.v20220622.
d93674fb - Do not create a new sensor for error count
88556559 - Set sensors to be expired in 1 hr
ff3595a0 - DGS-4220: fix request tag based metrics
PR-343 - Update CODEOWNERS for APIF team
PR-320 - APIF-2714: Switch from confluent-log4j to reload4j (5.4.x)
PR-329 - APIF-2705: Update jersey version to 2.36.
PR-316 - MMA-12033 Fix the connections limits test
PR-315 - KREST-4977 Allow limiting the number of active connections.
PR-310 - KREST-4450 500 error when topic not present

Schema Registry¶

NOTE: DGS-4389 added support for Protobuf custom options, which may change the behavior of schema lookups. To retain the old behavior in the Protobuf serializer, set schema.format=ignore_extensions.

PR-2399 - DGS-5084 Ignore compat check in IMPORT mode
PR-2395 - DGS-4971 Handle map types with enhanced.protobuf.schema.support
PR-2389 - MINOR: Update ErrorMessage description
PR-2388 - MINOR: Add ErrorMessage OpenAPI descriptions
PR-2387 - MINOR: Add OpenAPI description for deleteGlobalConfig
PR-2385 - Add OpenAPI operation tags
PR-2378 - Migrate PowerMock to Mockito in RestServiceTest (#2372)
PR-2381 - Adding timer in onJoinPrepare
0a9fb7d8 - Renamed DocumentedName to RootResource.java
PR-2376 - DGS-4768 Fix reserved ranges for Protobuf enums
646cf301 - Added DocumentedName to RootResource.java
PR-2373 - DGS-4724 Qualify names and merge maps when normalizing custom options
2fd5e6e9 - Remove deprecated methods in SchemaRegistryMetric
PR-2364 - Adding timer in onJoinPrepare
PR-2359 - DGS-4395 Fix message indexes of normalized Protobuf with map
PR-2357 - DGS-4389 Add support for Protobuf v2 extensions
PR-2353 - [DGS-4361] Added “/schemas/ids/{id}/schema” endpoint
PR-2351 - DGS-4358 Fix NPE in Protobuf converter for null map value
PR-2346 - Optimize sync call
PR-2345 - Add leader change listeners
PR-2342 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-2339 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
PR-2336 - DGS-4249 Fix name resolution during Protobuf normalization
PR-2331 - DGS-4172 Bound size of Avro datumReader/Writer caches
PR-2329 - DGS-4163 Pass topic to schema formatters
PR-2328 - DGS-4162 Handle refs at different levels in Protobuf converter
PR-2327 - DGS-1648 Allow Protobuf msg fullname to be passed for console producer
PR-2326 - DGS-4134 Add config to ignore default for nullable fields
PR-2313 - DGS-3862 Upgrade to spotbugs 4.7.0
PR-2305 - Change everit-json-schema coordinates to reflect artifact published on Central
PR-2289 - Added Custom Annotation
PR-2262 - Remove static reference over java.util.Random
PR-2198 - Set-compatibility Goal
PR-2197 - Adding folder support for Test Local Compatibility maven plugin

Commercial Features¶

Confluent Server¶

METRICS-4649 Refactor Remote Configuration activeFilters Semantics for 7.3.x
KMETA-451; Allow broker registration with older confluent.metadata.version
KMETA-448; Fix auto leader balancing of linked partitions
Exclude non-fips dependency(bcprov-ext-jdk15on) from trogdor project (#7673)
Exclude fips and non-fips bouncycastle dependency jar from connect packages (#7668) to 7.3.x
Back port fix for clm test to 7.3.x
Cherry-pick excluded bc jdk15 (non fips) from :ce-broker-plugins as it uses non-fips bc jars during compile time to 7.3.x
KMETA-436; Fix compatibility break with default principal schema
KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
Remote Config bug fixes (#7495)
KGLOBAL-2203: Fix MRC assignment logic so rack positioning in placeme
Add log message when skipping deletion
Cherry-pick KGLOBAL-1812:cb3b61c3c7e2d60b17feb2deb6580dfd2fec4973 to 7.3.x
Cherry-pick KGLOBAL-1777:4d6485ea189094f7c1935de786c607034f91538b to 7.3.x
Bump version to 3.3.0
CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
RCCA-8564: log a warning if LDAP login fails becuase of network issues
CIAM-2290: Upgrade bc fips drivers
KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
KGLOBAL-1812: Fix thread leak in cluster linking test. (#7188)
Increase timeout, correct error message returned for addBroker test
KGLOBAL-1797: Remove rack mixing feature flag (#7421)
Cherry-pick KGLOBAL-2127:72723ca0606d24a3894b58d4fad3eace9b0a07a0 to 7.3.x
CONFLUENT: implement ControllerLoadTime metric for KRaft
Cherry-pick Subset Partitioner Fix to 7.3.x
KENGINE-229; Partition.isReplicaIsrEligible is misused in Partition.maybeIncrementLeaderHW
KENGINE-212: batch transaction requests.
KENGINE-211: Add a metric to record the avg latency for a transaction to transit from PreCommit to Commit.
Remove the html end tag from upgrade.html
KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case
KMETA-290 Metadata shell supports snapshot and log
KGLOBAL-1803 reset lastCaughtupTime when mirror leader appends to its
Don’t include Server info as an HTTP response header in the Connect REST API (#7264)
Add Javadoc to undocumented public APIs in SBC
Update log level in RestClient
CP 7.3 - Telemetry Reporter Remote Configuration [METRICS-4195][METRICS-4186][METRICS-4189]
KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Control
RCCA-7224: Revert CIAM-1419 for 7.3.x
Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
KGLOBAL-1952: Attempt to shutdown both clusters in CL tests even if one fails to avoid thread leak impacting other tests
KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing part
KMETA-329; Workaround for AlterPartition regression on stale controllers
cherrypick KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode
Fix typo in offset writer thread config name
Fix 7.3.x compilation
KAFKA-14114: Add Metadata Error Related Metrics
KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
[METRICS-4507] Add cluster linking metrics to Telemetry Reporter
KAFKA-14107: Upgrade Jetty version.
KAFKA-14114: Add Metadata Error Related Metrics
Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
KMETA-321: Disable integration tests with co-located KRaft mode
Avoid sending partial updates during LDAP group manager startup
KSTORAGE-2280: Added a compaction CPU Utilzation Metric
KDATA-484: Adding stateBeforeDeletion field
[AUTHN-1097] Disable subject claim requirement in OAuth tokens
KENGINE-219: Fix AlterPartition idempotency
DGS-4302: ZKTopicMetadataCollector should avoid throwing exceptions
[minor] Update README with PR docker image push
Revert “CNKAF-1132: Record & unrecord total-throttle-time metric (#25
KAFKALESS-1247: Fix CPU metric for non-Linux system and ConfluentTelemetryReporterSamplerIntegrationTest timeout issue
KAFKALESS-737: Add system test for triggerEvenClusterLoad command
SBC: Add Resource Optimization Detector framework
Change CruiseControlMetricsProcessor log levels
Build and push dirty images to GAR nonprod repo
KAFKA-6945: KIP-373, allow users to create delegation token for others.
KGLOBAL-1576 Refactor SaslServerAuthenticator auth to relay NetworkRegionId to SaslServer
KAFKALESS-1261: Use newly added follower fetch rate metric
Upgrading netty to v4.1.79.Final #754
AUDIT-1139: Reducing the verbosity of NBKE
Add metrics for monitoring of Kafka Management events
KGLOBAL-1732: Update SBC to handle sync replicas and observers on the same rack
Address review comments
Address Yash Mayya’s comments around duplicate tests in AbstractWorkerSourceTaskTest and WorkerSourceTaskTest
Upgrading netty version to 4.1.79.Final
Upgrading aws-java-sdk-s3 to v1.12.268
KAFKALESS-1261: Add fetch count metric at topic level
Hotset Size Based Retention Breach Deletion of Compacted Segments
Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 - remove Operation annotation
Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 KAFKA-10000: Integration tests
Cherrypick 7098f04c3d5a30a6c16291dd78aa98694ce56e0b - fix WorkerSourceTaskTest
Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 - Fix missing tracer invocations
Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 and fix tests and refactor builder related changes in Worker
Cherrypick AK commit 6853d63e4de03d679978add576aa0977cecc053a - Fix test failures related to producerConfigs and adminConfigs
Remove updateConnectorConfig method
Fix MethodLength related to DistributedHerder
Cherrypick AK commit 603502bf5fb78983434a1a44ccc15a49ef6942b0
Cherrypick AK commit a110f1fe852ae8c958a8c64b0736a9bb0617338e - Merge header configs along with producer, consumer and admin configs
KAFKALESS-738: Even cluster load plan system tests
Cherrypick KAFKA-13803
Fix compilation issue in AbstractHerder, ReassignPartitionsCommand and use latest sink/source config from AbstractHerder’s ce-kafka/master
Revert “KAFKA-10000: Add new preflight connector config validation logic (#11776)”
Modified NonKafkaLogicalClusterMetadata parseLCM to be in sync with r
KAFKA-13649: Implement early.start.listeners and fix StandardAuthoriz
Cherry pick KAFKA-13474 on master to 6.1.x
CONFLUENT: add code for deleting the Acls from the pod by passing BOOTSTRAP_SERVERS_CONFIG, lkcID and principals
KREST-6986: Only add stats to sensor if metrics are not already registered
KGLOBAL-1724: Wrong source topic name in DescribeMirrorsResponse.
KDATA-454: add raft test annotation for clm test
KGLOBAL-1786: Trodgor task for consumer group operations and listing offsets
Sync this version of proto file
CIAM-2169: Add KsqlCluster: Describe to EnvMV & CCMV
Exponential backoff for automatic alter leadership calls
KMETA-295: RuntimeException in TelemetryReporter init on KRaft controllers
KAFKALESS-1216: Add ReplicaEntity and ReplicaMetricSample classes
KAFKALESS-1167: Configuration of incremental balancing
[METRICS-4509] Add Consumer Lag Offsets Metric to Telemetry-Reporter Whitelist
CIAM-1503: Ability to de-code message headers of auth-topic
AUTHN-1074: Support hierarchy for provider/pool
KCFUN-199: Convert dynamic quotas tests to KRaft
KAFKALESS-1217: Move TopicPartition to PartitionInfo
KGLOBAL-1727 : add NetworkRegionId tag to CL SaslAuthenticate request
AUTHN-974: Separate identity provider and pool resource types
KCFUN-506: Improve the quota allocation algorithm by capping at broker limit
KAFKA-14020: Performance regression in Producer (#12365)
KAFAKALESS-752: CLI for ComputeEvenClusterLoadPlan
KAFKALESS-1258: Use ConfigurationsImage during KRaft SBC startup and introduce in-memory BalancerEnabledConfig and consolidate/abstract SBC enablement there
KAFKALESS-1268: Add metrics for Databalancer engine
Added resourceId flag in aclCommand to list the ACLs in new format using Kafka CLI
KAFKALESS-1270: Pause for a bit while Executor reservation acquirement aborts another executor run
Update tenant transformations for kafka management audit logs
SD-628: Added resource type and roles for Stream Designer pipelines
CPKAFKA-8929 Disabling KRAFT failing test
[METRICS-4508] Add Kafka Controller Preferred Replica Imbalance Count Metric
KSTORAGE-2284, KSTORAGE-2285, KSTORAGE-2286: measure log append rate / data size / latency
KCFUN-495: Make Dynamic Quota reconfigurible
[METRICS-4503] Ignore Topology Change if Preferred Partition Leader Doesn’t Change for RandomBrokerPartitionSubsetPartitioner
KGLOBAL-1730: Add support for replica placement file with sync replicas and observers on the same rack
ReplicaManager should use brokerState instead of isShuttingDown to fence partitions followed by shutting down broker
Fix connect_rest_test.py after introduction of new source configs
DGS-4151:Update Rolebindings for some SR/DG related roles/operations
Update CODEOWNERS
CONFLUENT: Add metadata team to .github/CODEOWNERS for the old controller
CONFLUENT: Avoid materializing collection in AbstractFetcherManager to compute sum
AUTHN-1036: Change poolId principal prefix to always show User:
KSTORAGE-1696: Non contextual or confusing tiering logs seen frequently in Confluent Platform
KGLOBAL-1584: Add time to stop mirror topic metric
KAFKALESS-1247 Temporarily disable ConfluentTelemetryReporterSamplerIntegrationTest.testSampler
CIAM-2156: Add UI viewing permissions to SRResourceOwner, SRDeveloperX roles
KGLOBAL-1658: Add source topic id to kafka-mirrors –describe output
KC-2195: Implementing an HTTP API on the KRaft Controllers to check the quorum health
AUTHN-881: Added support for poolId in authz audit event.
KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
KAFKALESS-734: Backend implementation for ComputeEvenClusterLoadPlan
KDATA-476 Minor Remove unneeded wrap/unwrap in Option
KAFKALESS-1254: Disable BrokerFailureDetectorTest.testLoadFailedBrokers
CIAM-1518: KSQL modeled as a “cluster” for RBAC cloud
Disable failed test CLIENTS-2345
Disabled failed test CPKAFKA-6522
CIAM-2178 Split Role Def File for SDS into KSQL SDS and SR SDS
(JIRA ID : SEC-3593) CP LDAP - Enable Configurable Case Sensitivity for Authorisation
AUTHN-974: Add providerId to pool events
KGLOBAL-1769: logging node identifier in the error message and making
KSTORAGE-2267: lingering storage metrics on deleted logs
KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
KAFKALESS-1248: Temporarily disable testSelfHealingWithIgnoredBrokersPresentWithReplicaPlacements
KAFKALESS-1222: Enable BrokerFailureDetectorTest.testPartialClusterFa
DGS-3944: Add metrics for ZKMetadataCollector
KAFKALESS-1230: Enable ReplicaPlacementSelfHealingTest
KDATA-480 Abstract out Retry Policy from ObjectStoreUtils class
[skip secret scan] KGLOBAL-1366: Enable CL system tests in KRaft mode
Collect garbage collection metrics in TelemetryReporter [METRICS-4470]
INIT-599 - Allow OrgAdmins to delete the whole Organization in cloud_rbac_roles
[KPERF-454] Batch optimization for committing consumer group offsets.
KMETA-83 Support for StandardAuthorizer benchmark
DP-8085 - Migrate to Semaphore self-hosted agent
KMETA-185: Explicitly start metric reporters in remote KRaft controllers
Update CODEOWNERS for ce-metrics
KMETA-249; Ensure linux metrics collected on remote controllers
Add Cloud resource type of CLUSTER_LINK
KDATA-392: restore system test to support kraft
KAFKALESS-1227: Add NPE handling and consider all detection goals ski
CIAM-2083: Move SDS rbac roles into separate json
KSTORAGE-2279: Txn markers should not be removed by matching records in the offset map
Properly gather partition information when detected topics with incon
KGLOBAL-1351: Fix Incorrect prefixed-destination-link-count
Bug fix for system tests.
KGLOBAL-1085: Use default timeout for stop mirror topic.
KMETA-239 Fix missing ControllerApis error handling
Extract and Introduce libs for SDS engine to make authnz decisions outside of Kafka server
DGS-3331, DGS-3332 Get topic config change and snapshot in ZK
KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
KAFKALESS-754: Allow altering SBC goals configs (confluent.balancer.rebalancing.goals and confluent.balancer.triggering.goals) dynamically
Add detailed audit log integration test
Fix for KafkaAuthStoreTest.testCacheFailureStatus
MINOR: ignore consecutive handleMigration and trackInitLeader calls in TierDeletedPartitionsCoordinator
KMETA-186 Fix AlterConfigPolicy usage in KRaft
KAFKALESS-1221: Ensure SBC does not compute plans when reassignments exist
KGLOBAL-1649: Compatibility is broken for createClusterLink requests in KRaft mode.
KSTORAGE-2258: implement bucket storage probe metrics
KSTORAGE-2137: enable FTPS cleanup in ce-kafka system tests
[AUTHN-954] Add temp fields to IdentityPoolValue
KAFKALESS-1218: Use Linux system cpu utilization
KAFKALESS-733: Add ComputeEvenClusterLoadPlan Kafka admin API
KAFKALESS-839 exclusion-aware ReplicaPlacementGoal
fixed import order
KCFUN-506: Set a minimum value for reported quota consumption
KENGINE-194: Topic IDs not added to in sync fetcher pool
KMETA-213: Fix NPE caused by missing null check in SnapshottableHashTable::mergeFrom()
resolved failing tests
KAFKALESS-1227: Disable test_topic_rebalance for ZK
Adding tenant partition availability metric
KCFUN-386, KCFUN-392, KCFUN-253: Setting a hard limit on number of partitions and topics per cluster. Partial update # of topics and partitions for in-flight requests
Fix for test testWriterReelectionBeforeProduceComplete
Fix for test testWriterReelectionBeforeProduceComplete.
KAFKALESS-1222: Disable BrokerFailureDetectorTest#testPartialClusterFailure in ZK mode
KAFKALESS-1221: Ignore reassignemnts cancel plan computation test
KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs
KSTORAGE-2232: respect endOffset parameter when building offset map
KAFKALESS-1207 Fix numBrokers created in onPrem case at ClusterModelPBTUtils
KMETA-131 Cluster Linking metadata.version support
AUTHN-908: Disable trust policy cache
KSTORAGE-2249: update MergedLog.read to throw NotLeaderOrFollowerException on spurious OffsetOutOfRangeException
reverted the LocalLog info logging that came from AK as its already logged as part of MergedLog
Change DP schema request field config key default behavior
CNKAF-1195: Don’t compute plan while reassignments are present
KAFKALESS-1189: Fix SbcUpdateMetadataEvent to not override old metadata or get stuck in a loop
Add support for slow logs in request logging
reverted manually applied change from Unified as the logic to increment log start offset is different in MergedLog so this change is not needed
CONFLUENT: Move license validator after startup completion
KGLOBAL-1613: Persistent connection is not available (#6679)
KGLOBAL-1614: Transform ACL binding filter in ClusterLinkSyncAcls when in multi-tenant env and add ACL migration semantics integ test to MultiTenantClusterLinkTest
CONFLUENT: Log at debug level when pid unavailable
Integrate Kafka management events with audit log provider
KGLOBAL-1559: Do not fetch the metadata information under MetadataManager’s lock since the call is blocking.
KGLOBAL-771: Source Initiated Links for KRaft.
KAFKALESS-879: Get rid of capacityFor method
KAFKALESS-1202: Log error from EvenClusterLoadStateManager whenever registering an event with exception
KGLOBAL-1507: Filter out _schemas during cluster linking auto-mirroring
KGLOBAL-1613: Persistent connection is not available
KAFKALESS-1133: Add log to capture rack aware failure
Modify DP schema request field to config key
Refactor CLM tests
KDATA-432: fix tier state fence restore test for Azure
KPLATFORM-543: Move startup completion to end of startup sequence
KGLOBAL-1611: Handle ClusterLinkDisabledException in ClusterLinkAutoMirroring when determining if mirror topics need to be filtered
KGLOBAL-1473: converted ClusterLinkDestConnectionManagerTest and ClusterLinkAutoMirroringTest from easyMock to mockito
KSTORAGE-1965: Not all bytes were read from the S3ObjectInputStream
KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2)
KSTORAGE-2221: persist bounded base offset info of segments into FTPS
KMETA-203 Add “confluent.metadata.version”
KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
DGS-3504:Role changes for DG Catalog RBAC
KDATA-180: Add internal durability audit metrics
Kdata 448 refine restore rto
KAFKALESS-732: Introduce ComputeEvenLoadPlan Protocol
CONFLUENT: EasyMock -> Mockito conversion for most ce-kafka core tests
Maintain mapping of userId to resourceId after deletion of API Key
CIAM-1621: Make RBAC binding scopes support a tree structure
Remove events from ce-audit as code owner
RCCA-5913: provide exceptional use utility for mutation of topic ids
MINOR: Fix broker load tests in MultiTenantKafkaIntegrationTest
KGLOBAL-1561: Close reverse connections when deleting source side link.
Fix CLM liveness metric
KAFKALESS-751: Introduce CLI for trigger even cluster load
Cherry-pick KAFKA-13935
KDATA-455: address TopicIdPartition is None when rolling segment
KGLOBAL-1050: Remove cache in ClusterLinkSyncTopicsConfigs task
KAFKALESS-1186: Fix race condition in initializing BalancerStatusTracker
MINOR: Convert oauth tests
Add capability to enable trace record schema through DataPreview request
Add principal to authentication failure audit log
KCFUN-485: Memory leak in ThrottledChannelReaper
KGLOBAL-1559: Resolve the deadlock between cluster link manager and cluster link metadata manager interactions.
KAFKALESS-1177: Detect inconsistent replication factor by logging and emitting a metric
Support dynamic config for resourceId in TenantAclProvider
KDATA-388: RPO metrics for restore
KGLOBAL-1451:Remove retry-time-based mirror failure when source topicids are known
KC-2238: Reduce max.block.ms for telemetry producers in system tests to reduce broker shutdown time
KSTORAGE-1699: Auto disable segment deletion throttler during low free disk space
[AUDIT-1015] Fix producer emit operation future value to complete with true on success.
Readd metrics plumbing for request handler avg idle percent metric.
KGLOBAL-1485: Fail fast for persistent connection to non-coordinator
CCLOG-1790 Connector Developer roles should be able to access metrics
QEC-7888: Ensure that cluster link deletions in progress are completed on broker restart
KAFKALESS-731: Correct the log for the EvenClusterLoad status value being verified.
KAFKALESS-1109: Awakened events should execute before those in-queue
KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft
[skip secret scan] KAFKALESS-731: Update even cluster load state manager as design rework
MINOR: Add AWS ECR setup to image building section
KAFKALESS-1110: Handle rebalance drizzles with fixed-but-not-violated trigger goals.
Bump telemetry client version to 3.163.0 for minor logging fix
MINOR: Convert more integration tests
Authn-526: Kafka AuthN rate and Latency Metrics per saslMechanism
MINOR: SslCertificateIntegrationTest conversion
Make resource id support config as dynamic config
Fix exception handling in RBAC writer coordinator
Remove reference to cc-deployer.mk
AUTHN-813: Replace trust policy antlr parser with cel parser
KCFUN-463: dynamic config to allow client supplied acks setting to be overridden [skip secret scan]
KCFUN-481: Remove SocketServer state for disconnected channels when IP throttling expires
KAFKALESS-525: Test both flavors of shouldShutdown in AddBroker tests.
KAFKALESS-837: Rename Broker#State to Broker#Strategy
Revert “CIAM-1419: Restructure DefaultAuthCache to lookup access rules for principal (#4758)”
Revert “CONFLUENT: Rename DataPlaneAuthCache class name to CloudAuthCache (#4885)”
Update source and serviceName CRN format for authentication failure events
Revert “KDATA-404: New fields, and structure to be added to Durability Database”
KDATA-450 CLM support for Azure object store
KCFUN-128: Decouple recordThreadIdleRatio from request path
KDATA-421: Add missing azure backend for tier storage system tests
Test failure ‘KafkaService’ has no attribute ‘ACL_AUTHORIZER’
Artifactory Migration
MINOR: fix prefer.tier.fetch config for compacted topic in log_compaction_test system test
KC-2144: Revert “AUTHN-280: Java client: implement async auth (#4659)”
KC-2202 Disable kraft cluster linking system tests 7 2
KENGINE-181: add recoverable partitions to GroupMetadataManager metrics
KAFKALESS-1063: Allow more flexible leadership exclusion reasons.
KGLOBAL-1547: Bump ClusterLinkRecord version since we have added a non-nullable link mode
KGLOBAL-1415: Increase cluster link reconnect backoff max value
MINOR: Add metadata team as code owners
MINOR: Log PROXY protocol address when auth fails
KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (KMETA-108)
CIAM-2043 Separate permission for OwnKafkaClusterApiKey
KGLOBAL-1394: Broker fails to start due to deleted cluster link (#6509)
KSTORAGE-2060: Ignore missing file during log dir deletion
[MMA-5228] allow operator to describe all topics
CPKAFKA-8728, CPKAFKA-8729, CPKAFKA-8584 oauth system test fix
KCFUN-112: Use a separate run method for interbroker network threads
KGLOBAL-1394: Broker fails to start due to deleted cluster link
[skip secret scan] KDATA-348: New events from Tier Metadata Snapshot Initiate and Complete
KGLOBAL-1546: Fix race condition in acls method in StandardAuthorizerData
KGLOBAL-1486: Deflake ClusterLinkTest.test_offset_migration_early_destination_group_start
Efficiency Metrics for CLM
Part7: EasyMock to mockito migration for CL tests
KGLOBAL-1480: Part6 easyMock to mockito conversion
KC-2202 Disable CL in KRaft mode for CP 7.2
KMETA-149; Ensure forwarded requests are sampled for logging
KGLOBAL-1506: Keep CreateClusterLinkPolicy state in sync with metadata log
KMETA-160 Add Confluent records to metadata shell
KC-2223: Disallow enabling SBC with KRaft in CP 7.2 release
CloudClusterMetricsViewer should be able to view Connector Metrics
Rename kafka config of user resource id support
KDATA-353: Add feature flag configuration for FTPS snapshots and dynamic support
Make the sasl handshake and mechanism max receive sizes configurable
KC-2202; Disallow TS and KRaft in 7.2.x
KMETA-172: Ensure partition epoch bumped before ISR expansion
DGS-3640: Include leader epoch for MetadataImageListener::onLeaderUpdate
KGLOBAL-1419: Add KRaft support for storing cluster link IDs with ACLs
graduate MetricsViewer roles to public namespace
KGLOBAL-1478: Part4 convert from EasyMock to Mockito
Add view permission on cluster for connector roles
KGLOBAL-1479: Part5 easyMock to mockito conversion
[skip secret scan] Log consolidated final states of brokers
KC-1907; Audit log wiring for the KRaft controller
retention_stress_test: reliably spread writes across partitions
MINOR: disable test case testBasicRetention in kraft mode
KAFKALESS-529: Reset create time on new broker removal operation
KGLOBAL-1490: Fix bug with mapping of link id to topics
KGLOBAL-1475: Part3 convert from EasyMock to Mockito
KGLOBAL-1474: Part2 convert from EasyMock to Mockito
CONFLUENT: add more validation during KRPC deserialization
retention_stress_test: follow up formatting fix
CPKAFKA-5840: tier deletion test doesn’t produce data for the expected duration
KGLOBAL-1489: Enable AlterMirrorsRequestTest for KRaft mode
KAFKALESS-1076: Do not process altered exclusion events on inactive databalancer
Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
build: kafka system tests should support larger EBS volumes for newer instances (#6371)
Upgrade Gson
KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD
KGLOBAL-1471: Shut down ClusterLinkMetadataThread more gracefully
KGLOBAL-1357: Enable testSourceTopicRecreateDetectionUsingTopicIds in KRaft mode
retention_stress_test: long does not exist in python3
Set hostname verifier to accept everything when identifica
added support for userResourceID in delete Acls [skip secret scan]
DGS-3330, DGS-3333 and DGS-3471, run one active TopicMetadataCollector in a cluster
KGLOBAL-1481: Fix issue where ClusterLinkClearLinkReference does not run on startup in KRaft mode
KGLOBAL-1483: Source topic id is not populated in create mirror topic request.
KDATA-352: Cloud API for FTPS Snapshot upload
CIAM-1793 MetricsApi is available to new DataplaneRoles
Set hostname verifier to accept everything when identification algorithm is empty
AUTHN-833: Pass sub and azp as separate validatedExtensions
KAFKALESS-792: Stop SBC should interrupt SBC startup
EVENTS-991: Add events team as codeowners of ce-events modules
Add support for extracting AuditLog Entries from Kafka Request Events
EVENTS-989: enable use of subset partitioner in events exporter
build: kafka system tests should support larger EBS volumes for newer instances
KGLOBAL-1418: Use admin client for creating/deleting ACLs in ClusterLinkClearLinkReference
Update cloudevent encoding config doc
CPKAFKA-7262: fully support 7.0 and 7.1 in upgrade, downgrade tests
KSTORAGE-2180: upgrade flatbuffers to 2.0 to support M1 macs
KAFKA-13879: Reconnect exponential backoff is ineffective in some cas
KL-903/934: Enable EvenClusterLoadStatusTest#testBrokerRebalanceWithSelfHealing and SelfHealingAfterEnableTest#testBrokerRebalance
KSTORAGE-2171: do not adjust baseOffset for compacted segment
KC-2089: Ensure MZ cluster availability during a network issue (Add leadership priority calls)
KMETA-16: Add tiered storage topic config change validation to KRaft
[skip secret scan] KMETA-97 and KMETA-98: Fix compatibility issues with the topic CRUD request versions and bugs in mirror state lookup.
KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
KREST-5636 Inject MT secrets store in Kafka HTTP server apps
KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
EVENTS-874: Create deserializer for LogicalClusterMetadata and headers
KSTORAGE-2173: skip unnecessary format migration and cleanup during FTPS init
KSTORAGE-2176: refactor TierTestUtils.deletedSegments
KCFUN-443: Skip reporting empty quota target[skip secret scan]
KCFUN-391: Ignore Dynamic Quota if the cluster wide quota is unlimited[skip secret scan]
KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
KDATA-409: retry when exception of downloading FTPS from kafka pods
KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
AUTHN-792: Fix async authn performance regression
Resolve dependency issue in ce-broker-plugin
CONFLUENT: Convert ce-kafka-client-plugins to JUnit 5
KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
Disable idempotent producer by default in Connect centralized licenses (#6193)
Add different metric for filtering authorized resources vs authz denied
KGLOBAL-1355: Fix ClusterLinkFailureTest.testDestinationHighWatermark for KRaft using buffered produce
KDATA-434: change max wait time for AuditManagerTest
back port fix for scalabe restore test to 7.2.x
Update base image to use adoptium java 17.0.3 instead of correto
Authorizer performance improvements
KAFKALESS-833 PBT for Self-healing in the presence of ignored brokers [skip secret scan]
KAFKALESS-1060 Ensure broker#isAlive is used properly
Disable idempotent producer by default in Connect centralized licenses
Add Connect team as a codeowner for /connect/
KAFKALESS-874: Port the last set of broker removal integration tests to run in KRaft
KGLOBAL-1438: Make confluent.cluster.link.metadata.topic.enable only applicable in ZK mode
KMETA-127; Get quota integration tests working with KRaft
KSTORAGE-2153: Handle Azure object store responses during dns failures to ensure they’re retriable
CONFLUENT: Add error logs to AclAuthorizer
KMETA-70: Support Confluent’s replica placement plugin with KRaft
KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
KAFKALESS-1047: Always acess SBC related Enums by name
Move partition creation request limit to policy
METRICS-4064: Subset Partitioner Strategy to Reduce Telemetry Kafka Connections [skip secret scan]
Add new resourceType and role mapping for RBAC in Health+ API
KSTORAGE-2149: add compacted.topic.prefer.tier.fetch.ms config
KSTORAGE-2156: skip dual compaction validation if there are tierable local log segments
CONFLUENT: Convert rest-authorizer to JUnit 5 and improve its tests
KC-2116: Migrate tier_unclean_leader_election_test to KRaft
Update jackson packages to 2.13.2 version
Fix backward incompatibility issue in license store (#6152)
KGLOBAL-1349: Enable security for cluster link tests with KRaft
Update jackson packages to 2.13.2 version
KSTORAGE-2119: refactor FTPS tools to adapt to cleanup feature
KSTORAGE-1948: Implement FTPS cleanup logic
KGLOBAL-1390: Use admin client instead of local authorizer in ClusterLinkSyncAcls for creating/deleting destination ACLs
KDATA-428: add clusterid parameter in restore system test
CONFLUENT: Record total usage for cluster link quota even if user quota not set
KAFKALESS-1089: Increase reason limit for replica exclusions
KGLOBAL-1297: Revert cluster link sync filter config validation changes
KAFKA-13660: Switch log4j12 to reload4j (#11743)
METRICS-514 Kafka event log
KSTORAGE-2158: exclude internal topics from tier compaction
KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism (#6235)
KCFUN-413: Disable user tagging for Client Request Quota
KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism
Add kafka-eng as CODEOWNER
Fix backward incompatibility issue in license store
KMETA-104: Add internal REST server to KRaft remote controller
KDATA-313: Segment existence and metadata validation checks
KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
Add muckrake mapping for 7.2 release

Security¶

Removed bcfips from common
Pin bouncycastle fips dependency in confluent-security-plugins
Exclude bcpkix-jdk15on as compile dep and add as test dependency
made SchemaRegistryResourceActionKey private again
Made nonGlobal request a separate method to have different implementation in cloud
Backport “Fix a NoClassDefFoundError caused by a missing dependency (#518)” to 7.3.x
- making the SchemaRegistryResourceActionKey and schemaRegistryResourceActionMap protected so that it is visible to the subclasses in cloud plugins repo.
AUTHN-1087: Upgrade vertx to 4.3.2
AUTHN-1085: Fix NPE in debug log
APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
fix: add getAuthHeader method to ksql authorization plugin
trigger test
fix tests
add javadoc
Trigger Build
remove unused import
fix: add getAuthToken method to ksql plugin
fix: Exclude guava dependencies in ksqldb-rest-app
fix: get JwtToken from string instead of JwtPrincipal
Remove redundant config validation in ksql security extension
checkstyle
overrides
rebase to master
add auth token provider to ksql security extensions
KSE-1081: Add conf to enable/disable KSQL-SR permission checks
DGS-3862 Upgrade spotbugs to 4.7.0
SEC-3245: Migrate from confluent-log4j to reload4j
Refactor KSQL authorizer and security extension to allow Cloud authZ implementations
Delete the JAR fetched by MVN wrapper
Revert “Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision”
Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision
KSQL: Wrap RestAuthorizer on a new AuthorizationDecisionMaker class
Addendum
Addressing Sergios comments
Update KsqlSecurityExtensionConfig.java
Remove Kafka REST ProducerPool.
KSE-947: Add ksql.service.name to KsqlSecurityExtensionConfig
KSE-859: Add KSQL config to enable or disable security user impersonation
update codeowners
Prepare for removal of Kafka REST ProducerPool.
feat: add maven wrapper
Added exclusion since MDC class is failing because of reload4j
Minor: Fix KafkaYammerMetrics import

Secret Registry¶

Exclude Non-FIPS dep and use bc-fips dependency
Handle illegal reflection access under java 9+
Implement onJoinPrepare with latest AK change in KAFKA-14024
Migrate from log4j to reload4j for connect-plugin
CC-17641,CC-17644 Bump jetty version
APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
ignore new findbugs 4.7.0 detectors causing secret-registry build to fail
Migrate log4j
Introduce GitHub CODEOWNERS (Pull Request Reviewers)

Control Center¶

[MMA-12484][MMA-12483] Introduce Maven Profile for passing different configuration for JDK8 and JDK17
[MMA-12432] Support Control Center on JDK 17
MMA-12201 remove slf4j from transitive deps
MMA-12212: Upgrade jetty-http in blueway
MMA-12206 upgrade transitive dependency of netty
MMA-11617: allow for trailing slash in path
[MMA-12212][MMA-12211] Bump jetty version in blueway
MMA-12206 upgrade io.netty:netty-codec-http
MMA-12208 Upgrade okhttp
MMA-12201 Exclude slf4j in transitive dependency
MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
MMA-12201: Use reload4j version properties from common
MMA-12201 Add logredactor [7.1.x]
MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
RCCA-7746: CCloud RBAC user unable to view messages from UI
MMA-12201 Replace log4j with reload-4j
MMA-12208 Upgrade okhttp
MMA-12106 Add new api’s added in kafka streams
MMA-12010 Suppress errors due to Spotbugs upgrade to 4.7.0
CIAM-1533 make the default replication factor for Kafka API topic creation configurable
KEXP-349 experiences code owners
Remove Kafka REST ProducerPool.
Prepare for removal of Kafka REST ProducerPool.

MQTT Proxy¶

Fix 7.3.x test dep for MockFaultHandler
Update pom.xml
CC-17620:Remove dependency on slf4j-log4j12 and confluent-log4j

Metadata Service¶

Fix FIPS deps
Remove spaces
Add back newline
Implement writeTransactionMarkers in DelegateConfluentAdmin
CIAM-2304: Add unit tests for SecurityAdmin
Remove unnecessary dep on noop logger
MMA-12388: Upgrade LD SDK to 5.6.4
Revert “Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)”””
Promoted v0.451.0 Extractor to PROD
CIAM-2201 Close HTTP Application before closing authorizer
Merge 7.2.x to master
fix pass
[AUTHN-1000] stag/prod migrations for provider uniqueness
RCCA-8245: Fix for invalid consumer group for sync listener
[AUTHN-1018] Ignore deleted entries on provider/pool lookup
trust-service: Use v0.47.0 of auditlog library which fixes packaging issue
[AUTHN-1000] Make identity provider index based on jwks/issuer uniqueness
CIAM-2219 Re-open DP- 8085 migrate semaphore agent
Deploy version v0.441.0 of both cc-rbac and Extractor to prod
CIAM-2237 - Test and build stability prophylactic measures
[AUTHN-1081] Fix resourcespec parsing
[AUTHN-1075] Handle server errors gracefully
AUTHN-541 trust-service: Use recommended methods while testing audit logs
AUTHN-541 trust-service: Set default=”” for auditlog bootstrap server
[AUTHN-1070] Enable trust-service feature flags by default
CIAM-2139 Refactor Extractor code to rename CloudCacheHierarchy to CloudHierarchyCache
AUTHN-974: Authorize with separate provider/pool resource types
CIAM-2219: MINOR: Add vault secrets for Docker rate limit raise
Improve Build Stability
chore(deps): bump cc-base from v18.6.0-jdk-17 to v18.9.0-jdk-17
AUTHN-541 trust-service: Add some logging while initializing AuditLogger object
AUTHN-541 trust-service: Do not auditlog if not configured via configs
AUTHN-541 trust-service: Use X-B3-Traceid header as requestId
CIAM-1176 - Remove debugging print statement
Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
Update the oauth validation API specs
CIAM-1176: Increase Role Bindings Quota
CIAM-2049 Publish LC RoleBinding with K8 ID for New LC Creation
[AUTHN-968] Add fixed seed to hashcodes
[AUTHN-1041] Add identity pool filter limit to update calls
AUTHN-1074: Update tests to exercise hierarchy for providers/pools
Revert “DP-8085 - Migrate to Sempahore self-hosted agent (#1187)”
DP-8085 - Migrate to Semaphore self-hosted agent
AUTHN-974: Tests to exercise trust service provider/pool authorize
CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
chore(deps): bump docker/prod/confluentinc/cc-base from v16.4.0-jdk-16 to v18.6.0-jdk-17
[JIRA-ID: SEC-3597] - Integration Tests For Configurable Case Sensitivity for Authorisation in CP LDAP
AUTHN-541 trust-service: Accept X-Request-Id as header param for audit logging
AUTHN-541 trust-service: Address review comments
CIAM-2211 Configure the PKC and LKC ID for SDS in config file
Add utc time zone to metadata timestamps
CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
AUTHN-882: Replace principal with pool id
[AUTHN-954] Modify request field names
fixes 6.0.x build from 5.5.x merge
[AUTHN-820] Create AWS IAM Role for Trust service
CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
DGS-4151: Fixed mds test for Org/EnvOperator Describe access on Subje
[AUTHN-948] Rename spec field for identity providers
CIAM-2197 Fix the type of SR AND KSQL LD Flag
CIAM-2183 - MDS code fails validation for a topic name >= 80 characters
CIAM-2183 - MDS code fails validation for a topic name >= 80
AUTHN-989: Fix multithreading race condition where we only retry refreshing once
[AUTHN-997][AUTHN-948][AUTHN-946] Update Trust-Service Metadata
CIAM-1964 Publish RBAC Crud Changes for KSQL/SR with Fixed PKC ID
[AUTHN-949] Ensure that identity pool operations are using the right provider
[AUTHN-997] Rename identity pool’s spec to status
CIAM-2173 Route Kafka and SDS role bindings use RoleDef Files
CIAM-1518 Generalize Cloud Scope to take SR & KSQL
AUTHN-974: Add providerId to pool events
CIAM-2184 Extractor Publishing empty PKC ID if L*C not found in CHC
AUTHN-541 trust-service: Add a unit test to verify AuditLogUtils
AUTHN-541 trust-service: Ignore auditlog for IdentityPrincipal & HealthCheck
AUTHN-541 trust-service: Audit log CRUD APIs for IdentityPool resource
AUTHN-541: trust-service Audit log CRUD APIs for IdentityProvider resource
AUTHN-541 trust-service: Update AsyncReponses to accept audit objects
AUTHN-541 trust-service: Add a util to help build the AuditLogger object
AUTHN-541 trust-service: Register and bind AuditLogger object for tests
AUTHN-541 trust-service: Register and bind AuditLogger object
AUTHN-541 trust-service: Add auditlog config
AUTHN-541 rbac-extractor: Update events-schema lib from v0.73.0 to v0.82.0
AUTHN-541 trust-service: Add events-schema & auditlog-emitter-java dependency
[AUTHN-954] rename data fields for identity pools
CIAM-2130 - Deactivate environment scoped role-bindings for deactivated accounts in Prod
INIT-599 - Allow OrgAdmins to delete the whole Organization
CDMUM-2091 Add DECISION_ENGINE as accepted audience
Promoting cc-rbac to version 0.400.0
CIAM-2171 CHC API for LC -> PKC to use describeLogicalCluster
[PROD][AUTHN-822] Add db user with rds_iam role
[AUTHN-821] enabling RDS IAM Auth on rbac database
[AUTHN-954] publish duplicate fields for IdentityPoolValue
CIAM-2147 Integration With CHC failing with incompatible vertex version
AUTHN-918[PROD]: Add ‘claims.’ prefix to subject_claim to identity pools in database
[CIAM-2077]: Add API to Integrate with CHC Batch Processing
Revert “CIAM-2147 Integration With CHC failing with incompatible vertex version (#1174)”
CIAM-2147 Integration With CHC failing with incompatible vertex version
Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1173)”
CIAM-2147 Update CHC version to fix incompatible vertex version
Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1172)”
CIAM-2147 Update CHC version to fix incompatible vertes version
[AUTHN-973] Modify rbac operation request based on status update #1171
RCCA-7223: Scale down cc-rbac cpd replicaCount from 2 to 1
AUTHN-918[DEVEL]: Add ‘claims.’ prefix to subject_claim to identity pools in database
CIAM-2147 Update CHC version to fix incompatible vertes version
Promoting cc-rbac to version 0.385.0
AUTHN-968: Add logging around 500 error in createIdentityPool
AUTHN-918[STAG]: Add ‘claims.’ prefix to subject_claim to identity pools in database
[CIAM-1973] Add API to fetch K8 ID from CHC
Promoted Extractor to v0.381.0
[CIAM-2131]: Deactivate environment scoped role-bindings for deactivated accounts in Devel
[CIAM-2127]: Deactivate environment scoped role-bindings for deactivated accounts in Stag
CIAM-2146: Promoted Extractor to v0.374.0
[AUTHN-960] Fix create identity providers bug
revert import optimizations
re-add fallback for missing gateway host
remove unnecessary property
revert endpoint prop and set gateway host always
add rds endpoint property
fix bracket
fix style
adding aws dependencies
add semicolons
add back poassword, cleanup
[AUTHN-824] instrument service to connect to DB with RDS IAM Auth
[CIAM-2138]: Updated ConfigMap Value
AUTHN-740: Add better logs for retrying on refresh failure and fix default refresh interval when cache control is not present
Add service.yaml file for rbac extractor
Added getParameterType to error message. Removed getParameterName
CIAM-2078 Add Exception Handling in CHC API
RCCA-6743: Test that we can allow rest client to disable hostname verification
Promoted Extractor version
trust-service: Add a test to validate CRN format
AUTHN-794 trust-service: Fix CRN format
CIAM-2096: Decreased total hc topics from 60 to 20
[CIAM-2138]: Update CHC API to accept host and port instead of API
AUTHN-943: Update quota limit error message for idp and pool
CIAM-2044: Fixing db transaction deadlock caused by delete.
RCCA-6760 Delete invalid rolebindings
Rebase with upstream and fix test
address review comments
AUTHN-918: update rbac extractor to handle the new sub claim prefix
AUTHN-918: enforce subject claim to start with ‘claim.’ prefix in IdentityPool create and update
[AUTHN-894][AUTHN-749][AUTHN-816][AUTHN-817] Update creation/deletion logic
RCCA-7563: Delete crufty Rolebindings for large customer
CIAM-347: Prevent OrgAdmin from self-deleting OrgAdmin role for V2 API
Added getParameterType to error message. Removed getParameterName
CIAM-2096: Decreased total RBAC HC topics to 20 so decreased total RBAC HC role
Added getParameterType and getParameterName to error message
Adding two test cases for pool filters
Added unit tests for different ParamException types.
trust-service: Add a test to validate CRN format
AUTHN-794 trust-service: Fix CRN format
Catch ParamException 400 errors. Renamed file.
Catch ParamException 400 errors.
CIAM-2021 MDS changes to support Tree Binding Scopes
Remove duplicate dependency
Remove cpd docker pull secrets
CIAM-2094: Decrease total HC topics to 20.
Address review comments
Implement LDClient for provider and pool limits in trust service
Skip regular builds for changes in .deployed-versions
[STAG][AUTHN-822] add db user with rds_iam role]
CIAM-2093, RCCA-7359: Set cc-rbac prod version to 110, rbac-extractor prod version to 295 (0.350.0)
CIAM-2093: Update the num of Extractor pods to 0 in cpd
AUTHN-813: Replace trust policy antlr parser with cel parser
Address review comments
Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)””
CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
Integrate JWKS limit with LD
CIAM-2071: Delete invalid role bindings from v2 api (for internal orgs) [prod]
[AUTHN-822] add db user with rds_iam role
Promoted rba-extractor to version 0.340.0 on PROD, DEVEL and CPD
CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
CIAM-2075 Disabling failure check of PKC Header failure for Integration Testing
Implement jwks limit
CIAM-1970: Implement displayName resolution for V2 API
Set cc-rbac prod version to 93 (0.333.0)
Address review comment
address review comment - check filter limit before pool limit
Added tests
CIAM-1863: Fix deployed versions name
CIAM-1929 Added validation on role binding scope.
Enforce resource limit for IDP, pool and filter in trust service
CIAM-1828 Publish PKC Header For RBAC Role binding Changes
Fix Fuzz Test by specifying most specific scope
[CIAM-2040] Add Log Statement to find out if different ResourceTypes for a single role binding are used by customer
CIAM-1863: Update icc-rbac and extractor pipelines to stag->prod->devel->cpd
CIAM-1880 - Runbookize our DBMigrate approach to deleting rolebindings
CIAM-2030: Updated pattern_type to LITERAL on stag and prod
CIAM-2025: Fixed error on access RBAC role_bindings sequences.
CIAM-1523: Start routing traffic to icc-rbac
CIAM-2028 Remove Dataplane LD Flags from extractor
[CIAM-1840] KSQL/SR Extractor Record Header
CIAM-2025: Grant usage on sequence rbac.role_binding_last_change_id_seq to cc_rbac_extractor_0 and cc_rbac_extractor_1
CIAM-2030 - Updated pattern_type to LITERAL for org 0 role bindings
CIAM-2024: Added logic cluster type healthcheck .
CIAM-2027 Update CHC Client Version to support JDK 8
CIAM-1823 Interface with Cloud Hierarchy Client
CIAM-1786 Add ResourcePattern to correctly resolve display name
AUTHN-835, AUTHN-845: Trim whitespaces on user entry
[CIAM-1839] Enable identification of KSQL/Schema Role bindings
CIAM-1902: Inserted 60 role bindings into rbac db for prod
CIAM-1897: Remove client_address field from icc-rbac audit logs
Remove extra semicolon
CIAM-1919: Address issues from icc-rbac ops review
RCCA-6909 - Customer needs role bindings deleted for deleted lkcs again
CIAM-1939 producer close should timeout
Fix dependencies
[AUTHN-747] remove policy version/id
CIAM-1902: Added role bindings to SA for stag.
Fix JWTDebugLoggingTest failure by excluding log4j jars
CIAM-1954 MINOR: Healthcheck to check RUNNING immediately
CIAM-1951: Updated deployment notification.
CIAM-1432: Make icc-rbac audit logs match kafka mds
CIAM-1432: Add icc-rbac devel auditlog config to stag and prod
AUTHN-737, AUTHN-752, AUTHN-720: Fix self links, add rbac.addr
MINOR Add log for manual republish
CIAM-1752: Enable icc-rbac db metrics
[AUTHN-750] [RCCA-6673] Grant permissions for cts schemas to rbac-extractor user
Update CODEOWNERS
AUTHN-582: Add rbac checks for trust service provider/pool CRUD apis
[AUTHN-567] Add validation checks to getJWKS()
CIAM-1912 AuthN to be Codeowners of trust-service
CIAM-1776 Update cc-base to v16.4.0
CIAM-1432: Fix default cloudevent.codec
Create/run migrations for trust-service in stag/prod
CIAM-1544: Remove ce-kafka-version suffix from mds image version
RCCA-6555 Remove role bindings associated with deleted clusters
CIAM-1523: Add audiences config to icc-rbac
[AUTHN-711] Modify prefix for identity pools
Upgrade cc-base image
CIAM-1432: Enable Audit Logging for icc-rbac
AUTHN-619: Add getIdentityPrincipal call

Replicator¶

RCCA-7678: Reverse proxy header check added
KGLOBAL-2126 seek to begining only for non empty partitions list
Add log redactor.
Migrate confluent-log4j to reload4j.
fix upstream build