Confluent Platform Component Changelogs¶
This topic provides changelogs for the individual Confluent Platform components.
Version 7.3.1¶
Released December 2022
Community Features¶
Common¶
Kafka¶
- PR-845 - CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
- PR-842 - DP-9030: Use the new withGradleFile closure
- PR-12809 - [KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)
- PR-829 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
- PR-12836 - KAFKA-14282: stop tracking Produced sensors by processor node id (#12836)
- PR-816 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
- PR-12794 - Revert “KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)” (#12794)
- PR-12790 - KAFKA-14337; Correctly remove topicsWithCollisionChars after topic deletion (#12790)
- PR-12765 - KAFKA-14316; Fix feature control iterator metadata version handling (#12765)
- PR-12747 - KAFKA-14300; Generate snapshot after repeated controller resign (#12747)
- PR-12676 - KAFKA-14209 : Integration tests 3/3 (#12676)
- PR-12741 - KAFKA-14296; Partition leaders are not demoted during kraft controlled shutdown (#12741)
- PR-12736 - KAFKA-14292; Fix KRaft controlled shutdown delay (#12736)
- PR-12709 - KAFKA-14275; KRaft Controllers should crash after failing to apply any metadata record (#12709)
- PR-12634 - KAFKA-14225; Fix deadlock caused by lazy val exemptSensor (#12634)
- e23c59d0 - Bump version to 3.3.1
- 1780f266 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- 4b35f247 - Bump 3.3 branch to 3.3.1-SNAPSHOT
ksqlDB¶
REST Proxy¶
Schema Registry¶
- PR-2467 - changed the header name of pool id
- PR-2460 - DGS-5544 Support nulls when using optional fields in proto2
- PR-2459 - DGS-5400 Support subjectPrefix containing wildcard context and subject
- PR-2456 - Derive schema maven plugin 7.3.x
- 87defb47 - Exposed the method to schema registry client to use in security-plugins
- PR-2450 - Pin Snakeyaml version
- PR-2443 - DGS-5423 Check subject when looking up ID across contexts
- PR-2441 - * Minor: change OAuth config string
- PR-2440 - Update the Snakeyaml dependency
- PR-2409 - DGS-3721 SaslOAuthCredentialProvider
- PR-2371 - DGS-3396: Add deletedOnly=true to SR list APIs
- 8e5ce168 - Unpin Protobuf version.
- PR-2421 - DGS-5220 Handle nested extend decls in Protobuf
- PR-2415 - DGS-5254 Make timeouts configurable when forwarding requests
- PR-2414 - DGS-5253 Handle cycles when resolving references
- PR-2344 - DGS-3719, DGS-3720 Added OAuthCredentialProvider and CachedOauthTokenRetriever
- PR-2407 - MINOR: Fix OpenAPI deleteSubjectConfig example
- PR-2405 - DGS-4754 Handle empty record default at field level
Commercial Features¶
Confluent Server¶
- remove extraneous }
- CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
- DP-9030 - Switch to codeartifact repo
- merge from ccs:3.3 to ce:7.3.x
- DGS-5248 Adding SR client OAuth config for Schema Validation.
- KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error messa
- Increase vagrant aws timeout for system test
- Upgrade from Scala 2.13.8 to 2.13.10
- Upgrade to snakeyaml version 1.32
- Upgrade jackson-databind version to 2.13.4.2
- MINOR: TierArchiver - improve logging for cancelled tasks
- KGLOBAL-2143: Return error message containing all racks that dont have enough brokers during topic creation validation
- KGLOBAL-2122: Validate there are enough brokers to satisfy updated topic placement
- Upgrade to protobuf version 3.19.6
- CONFLUENT: Revert protobuf upgrade change
- CIAM-2424 - Name RestClient threads for stackdump identification (#7887)
- CPKAFKA-9173: Disable colocated KRaft upgrade from CP 7.1/7.2
- Bump version to 3.3.1
- KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- Bump 3.3 branch to 3.3.1-SNAPSHOT
Security¶
- DGS-5542: Incorporated code review comments and removed the unused parameter schemaRegistryResourceOperation
- DGS-5542: SchemaId validation for numeric value in authorizer filter
- fixed indentation
- Update AuthorizationFilter.java
- Changed from 5 operations to 1
- updated tests
- added null when schema id does not exist
- changed the logic of authorizeSchemaIdLookup
- changed to Short
- dummy commit for new build
- Adding OAuth config support for Schema validation
- Exclude bcpkix-jdk15on as compile dep and add as test dependency
- Pin bc-nonfips dep as a test dependency
- Pin bouncycastle fips dependency in confluent-security-plugins
- Removed bcfips from common
- excluded bcfips jars
- Removed From Supported Operations to follow RBAC Rules
- Added fix for unit test
- Changed the design of RBAC for Exporters
- Fixed import
- Removed extra line changes
- Added RBAC for Schema linking Phase 1
- AUTHN-1087: Explicit define netty versions in kafka-rest plugin
- AUTHN-1087: [7.0.x only] Enforce Netty versions in kafka-rest plugin
- DGS-3396: introduce LookupFilter for list apis
- AUTHN-1087: Upgrade vertx to 3.9.14
- AUTHN-1325: Update bc-fips version to 1.0.2.3
- Fix bouncycastle issue
- Exclude bouncycastle non-fips jar from connect packaging
- Revert “Exclude bcpkix-jdk15on as compile dep and add as test dependency”
- Revert “Pin bouncycastle fips dependency in confluent-security-plugins”
- Revert “Removed bcfips from common”
- fixed checkstyle error
- Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas
Secret Registry¶
- Exclude Non-FIPS dep and use bc-fips dependency
Control Center¶
- RCCA-7746: CCloud RBAC user unable to view messages from UI
- MMA-9023: memory bounding for streams applications [5.4.x, master]
- [MMA-12762][MMA-12624] Add testng dependency
- Fix Failing Tests due to CC-18163
- Revert “MMA-12206 upgrade io.netty:netty-codec-http”
MQTT Proxy¶
- Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
- Upgrading netty
Metadata Service¶
- Update Launchdarkly
- APIF-3122: Unpin protobuf version.
- MMA-12624 Fix testng dependencies
- CIAM-2392 Bump PSQL Version
Replicator¶
- KGLOBAL-2436: Don’t seek to beginning on pause for schema translator
- REPL-2055-: Log WakeupException at debug level
Version 7.3.0¶
Released October 2022
Community Features¶
Common¶
- aeca780b - Update Jolokia version in pom.xml
- PR-472 - Upgrade Netty to 4.1.79.Final
- PR-471 - fix checkstyle
- PR-468 - APPSEC-1393: Fix dependency in disk-usage-agent [6.2 and 7.0]
- PR-464 - APPSEC-1393:Migrate from confluent-log4j to reload4j [5.4.x – 7.0.x]
- PR-465 - APPSEC-1393: Confluent log4j to reload4j [7.1.x Only]
- PR-467 - APPSEC-1412: Upgrade JUNIT in 5.4.x only
- PR-442 - Introduce Pull Request Reviewers
Kafka¶
- eefe8671 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- PR-12642 - KAFKA-14207; KRaft Operations documentation (#12642)
- PR-12681 - KAFKA-14259: BrokerRegistration#toString throws an exception, terminating metadata replay (#12681)
- 9d1f9f77 - Bump version to 3.3.0
- PR-12628 - KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
- PR-12664 - KAFKA-14243: Temporarily disable unsafe downgrade (#12664)
- PR-12653 - KAFKA-14240; Validate KRaft snapshot state on startup (#12653)
- PR-12658 - KAFKA-14233: Disable testReloadUpdatedFilesWithoutConfigChange first to fix the build (#12658)
- PR-12655 - KAFKA-14238; KRaft metadata log should not delete segment past the latest snapshot (#12655)
- PR-12570 - KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
- PR-12625 - KAFKA-14222; KRaft’s memory pool should always allocate a buffer (#12625)
- PR-12626 - KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626)
- PR-12603 - KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603)
- PR-12624 - KAFKA-14215; Ensure forwarded requests are applied to broker request quota (#12624)
- b2639c8d - Remove the html end tag from upgrade.html
- PR-12597 - KAFKA-14205; Document how to replace the disk for the KRaft Controller (#12597)
- PR-12596 - KAFKA-14203 Disable snapshot generation on broker after metadata errors (#12596)
- PR-12617 - KAFKA-14216: Remove ZK reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc (#12617)
- PR-12618 - KAFKA-14217: app-reset-tool.html should not show –zookeeper flag that no longer exists (#12618)
- PR-12609 - KAFKA-14198; swagger-jaxrs2 dependency should be compileOnly (#12609)
- PR-12584 - KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584)
- PR-12604 - KAFKA-14188; Getting started for Kafka with KRaft (#12604)
- PR-12599 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (#12599)
- PR-12598 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (server side) (#12598)
- PR-12595 - KAFKA-14204: QuorumController must correctly handle overly large batches (#12595)
- PR-11783 - KAFKA-14143: Exactly-once source connector system tests (#11783)
- PR-12586 - KAFKA-14200: kafka-features.sh must exit with non-zero error code on error (#12586)
- PR-12578 - KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case (#12578)
- PR-12533 - KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Controller (#12533)
- PR-12294 - KAFKA-13990: KRaft controller should return right features in ApiVersionResponse (#12294)
- PR-12571 - KAFKA-14187: kafka-features.sh: add support for –metadata (#12571)
- PR-12565 - KAFKA-14183; Cluster metadata bootstrap file should use header/footer (#12565)
- PR-12513 - KAFKA-14177: Correctly support older kraft versions without FeatureLevelRecord (#12513)
- PR-12103 - KAFKA-13850: Show missing record type in MetadataShell (#12103)
- PR-12551 - KAFKA-14178 Don’t record queue time for deferred events (#12551)
- PR-12469 - KAFKA-13914: Add command line tool kafka-metadata-quorum.sh (#12469)
- PR-12508 - KAFKA-13888; Implement LastFetchTimestamp and in LastCaughtUpTimestamp for DescribeQuorumResponse [KIP-836] (#12508)
- PR-12518 - KAFKA-14167; Completion exceptions should not be translated directly to error codes (#12518)
- PR-12517 - KAFKA-13940; Return NOT_LEADER_OR_FOLLOWER if DescribeQuorum sent to non-leader (#12517)
- PR-12491 - KAFKA-14148: Update ResetOffsetsDoc (#12491)
- PR-12514 - KAFKA-14154; KRaft controller should return NOT_CONTROLLER if request epoch is ahead (#12514)
- PR-12274 - KAFKA-13959: Controller should unfence Broker with busy metadata log (#12274)
- PR-12506 - KAFKA-14154; Return NOT_CONTROLLER from AlterPartition if leader is ahead of controller (#12506)
- PR-12498 - KAFKA-13986; Brokers should include node.id in fetches to metadata quorum (#12498)
- PR-12184 - Fix the rate window size calculation for edge cases (#12184)
- PR-12487 - KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode (#12487)
- f3cf6db3 - KAFKA-14114: Add Metadata Error Related Metrics
- PR-12396 - KAFKA-14051: Create metrics reporters in KRaft remote controllers (#12396)
- PR-12403 - KAFKA-13166 Fix missing ControllerApis error handling (#12403)
- PR-12467 - KAFKA-14129: KRaft must check manual assignments for createTopics are contiguous (#12467)
- PR-12447 - KAFKA-14124: improve quorum controller fault handling (#12447)
- c2422f63 - Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
- PR-12489 - KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing partition epoch (#12489)
- PR-12457 - KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
- PR-12440 - KAFKA-14107: Upgrade Jetty version (#12440)
- PR-12483 - KAFKA-14136 Generate ConfigRecord for brokers even if the value is unchanged (#12483)
- ba219265 -: upgrading netty to v4.1.79.Final #754
- PR-12429 - KAFKA-14089: Only check for committed seqnos after disabling exactly-once support in Connect integration test (#12429)
- PR-12415 - KAFKA-14079 - Ack failed records in WorkerSourceTask when error tolerance is ALL (#12415)
- PR-12374 - KAFKA-14039 Fix AlterConfigPolicy usage in KRaft (#12374)
- PR-12411 - KAFKA-14078; Do leader/epoch validation in Fetch before checking for valid replica (#12411)
- PR-12433 - KAFKA-14093: Use single-worker Connect cluster when testing fenced leader recovery (#12433)
- PR-12347 - KAFKA-13919: expose log recovery metrics (#12347)
- PR-12408 - KAFKA-14076: Fix issues with KafkaStreams.CloseOptions (#12408)
- PR-12365 - KAFKA-14020: Performance regression in Producer (#12365)
- PR-12349 - KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance (#12349)
- PR-12421 - Revert “KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)” (#12421)
- PR-12420 - KAFKA-13769 Fix version check in SubscriptionJoinForeignProcessorSupplier (#12420)
- PR-12405 - KAFKA-13572 Fix negative preferred replica imbalanced count metric (#12405)
- PR-10964 - KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
- PR-12265 - KAFKA-13968: Fix 3 major bugs of KRaft snapshot generating (#12265)
- PR-12398 - KAFKA-14062: OAuth client token refresh fails with SASL extensions (#12398)
- PR-12390 - KAFKA-14055; Txn markers should not be removed by matching records in the offset map (#12390)
- PR-12381 - KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection (#12381)
- PR-12296 - KAFKA-13996: log.cleaner.io.max.bytes.per.second can be changed dynamically (#12296)
- PR-12359 - KAFKA-13983: Fail the creation with “/” in resource name in zk ACL (#12359)
- PR-12091 - KAFKA-12943: update aggregating documentation (#12091)
- PR-12297 - KAFKA-13846: Follow up PR to address review comments (#12297)
- PR-12337 - KAFKA-10199: Remove main consumer from store changelog reader (#12337)
- PR-12360 - KAFKA-14032; Dequeue time for forwarded requests is unset (#12360)
- PR-12379 - KAFKA-10199: Remove call to Task#completeRestoration from state updater (#12379)
- PR-12224 - KAFKA-13943; Make LocalLogManager implementation consistent with the RaftClient contract (#12224)
- PR-11782 - KAFKA-10000: Integration tests (#11782)
- PR-11784 - KAFKA-13228; Ensure ApiVersionRequest is properly handled KRaft co-resident mode (#11784)
- PR-11894 - KAFKA-13613: Remove hard dependency on HmacSHA256 algorithm for Connect (#11894)
- PR-12376 - Upgrade Netty and Jackson versions [KAFKA-14044] (#12376)
- PR-12372 - KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
- PR-12371 - KAFKA-14035; Fix NPE in SnapshottableHashTable::mergeFrom() (#12371)
- PR-12204 - [9/N][Emit final] Emit final for session window aggregations (#12204)
- PR-12329 - KAFKA-14010: AlterPartition request won’t retry when receiving retriable error (#12329)
- PR-12139 - KAFKA-13821: Update Kafka Streams WordCount demo to new Processor API (#12139)
- PR-12293 - KAFKA-13963: Clarified TopologyDescription JavaDoc for Processors API forward() calls (#12293)
- PR-12312 - KAFKA-10199: Expose tasks in state updater (#12312)
- PR-12279 - KAFKA-10199: Commit the restoration progress within StateUpdater (#12279)
- PR-12269 - KAFKA-13966 Prepend bootstrap metadata to controller queue (#12269)
- PR-12291 - KAFKA-13987: Isolate REST request timeout changes in Connect integration tests (#12291)
- PR-12209 - KAFKA-13930: Add 3.2.0 Streams upgrade system tests (#12209)
- PR-11781 - KAFKA-10000: Per-connector offsets topics (#11781)
- PR-10738 - KAFKA-6945: KIP-373, allow users to create delegation token for others (#10738)
- PR-12298 - KAFKA-13998: JoinGroupRequestData ‘reason’ can be too large (#12298)
- PR-12304 - KAFKA-13880: Remove DefaultPartitioner from StreamPartitioner (#12304)
- PR-12226 - KAFKA-13890: Improve documentation of ssl.keystore.type and ssl.truststore.type (#12226)
- PR-12263 - KAFKA-13939: Only track dirty keys if logging is enabled. (#12263)
- PR-12161 - KAFKA-13873 Add ability to Pause / Resume KafkaStreams Topologies (#12161)
- PR-12206 - KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
- PR-12287 - KAFKA-13846: Use the new addMetricsIfAbsent API (#12287)
- PR-12248 - KAFKA-13958: Expose logdirs total/usable space via Kafka API (KIP-827) (#12248)
- PR-12181 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2) (#12181)
- PR-12250 - KAFKA-13935 Fix static usages of IBP in KRaft mode (#12250)
- PR-12121 - KAFKA-13846: Adding overloaded metricOrElseCreate method (#12121)
- PR-11473 - KAFKA-13436: Omitted BrokerTopicMetrics metrics in the documentation (#11473)
- PR-11780 - KAFKA-10000: Exactly-once source tasks (#11780)
- PR-12140 - KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)
- PR-12264 - KAFKA-13967: Document guarantees for producer callbacks on transaction commit (#12264)
- PR-11779 - KAFKA-10000: Zombie fencing logic (#11779)
- PR-12267 - KAFKA-13947: Use %d formatting for integers rather than %s (#12267)
- PR-12197 - KAFKA-13929: Replace legacy File.createNewFile() with NIO.2 Files.createFile() (#12197)
- PR-12067 - KAFKA-13780: Generate OpenAPI file for Connect REST API (#12067)
- PR-12180 - KAFKA-13917: Avoid calling lookupCoordinator() in tight loop (#12180)
- PR-12270 - KAFKA-10199: Implement removing active and standby tasks from the state updater (#12270)
- PR-12245 - KAFKA-13410; Add a –release-version flag for storage-tool (#12245)
- PR-12240 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (#12240)
- PR-12235 - KAFKA-13945: add bytes/records consumed and produced metrics (#12235)
- PR-11778 - KAFKA-10000: Use transactional producer for leader-only writes to the config topic (#11778)
- PR-12159 - KAFKA-13933: Fix stuck SSL unit tests in case of authentication failure (#12159)
- PR-12170 - KAFKA-13875 Adjusted the output the topic describe output to include TopicID & se… (#12170)
- PR-12238 - KIP-835: metadata.max.idle.interval.ms shoud be much bigger than broker.heartbeat.interval.ms (#12238)
- PR-12005 - KAFKA-13803: Refactor Leader API Access (#12005)
- PR-12210 - KAFKA-13930: Add 3.2.0 to core upgrade and compatibility system tests (#12210)
- PR-12246 - KAFKA-13718: kafka-topics describe topic with default config will show segment.bytes overridden config (#12246)
- PR-11776 - KAFKA-10000: Add new preflight connector config validation logic (#11776)
- PR-12191 - KAFKA-12657: Increase timeouts in Connect integration tests (#12191)
- PR-12136 - KAFKA-13773: catch kafkaStorageException to avoid broker shutdown directly (#12136)
- PR-12084 - KAFKA-13845: Add support for reading KRaft snapshots in kafka-dump-log (#12084)
- PR-12183 - KAFKA-13883: Implement NoOpRecord and metadata metrics (#12183)
- PR-721 - Update CODEOWNERS
- PR-12225 - KAFKA-13946; Add missing parameter to kraft test kit ControllerNode.setMetadataDirectory() (#12225)
- PR-10830 - KAFKA-12902: Add unit32 type in generator (#10830)
- PR-12187 - KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (#12187)
- PR-12062 - KAFKA-13833: Remove the min_version_level from the finalized version range written to ZooKeeper (#12062)
- PR-12200 - KAFKA-10199: Implement adding standby tasks to the state updater (#12200)
- PR-12035 - KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so (#12035)
- PR-12190 - KAFKA-13923; Generalize authorizer system test for kraft (#12190)
- PR-12160 - KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD (#12160)
- PR-12109 - KAFKA-13863; Prevent null config value when create topic in KRaft mode (#12109)
- PR-12150 - KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
- PR-12050 - KAFKA-13830 MetadataVersion integration for KRaft controller (#12050)
- PR-12165 - KAFKA-13905: Fix failing ServerShutdownTest.testCleanShutdownAfterFailedStartupDueToCorruptLogs (#12165)
- PR-12162 - KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs (#12162)
- PR-11748 - KAFKA-12635: Don’t emit checkpoints for partitions without offset-syncs (#11748)
- PR-12164 - Update note on upgrade from log4j to reload4j (#12164)
- PR-12087 - KAFKA-13851: Add integration tests for DeleteRecords API (#12087)
- PR-11916 - KAFKA-12703; Allow unencrypted private keys when using PEM files (#11916)
- PR-12135 - KAFKA-13785: [7/N][Emit final] emit final for sliding window (#12135)
- PR-11969 - KAFKA-13649: Implement early.start.listeners and fix StandardAuthorizer loading (#11969)
- PR-11775 - KAFKA-10000: Add all public-facing config properties (#11775)
- 040b11d7 - KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- PR-12108 - KAFKA-13862; Support Append/Subtract multiple config values in KRaft mode (#12108)
- PR-12131 - KAFKA-13879: Reconnect exponential backoff is ineffective in some cases (#12131)
- PR-12085 - KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
- PR-12010 - KAFKA-13793: Add validators for configs that lack validators (#12010)
- PR-11983 - KAFKA-13763: Refactor IncrementalCooperativeAssignor for improved unit testing (#11983)
- PR-710 - : Update jackson packages to 2.13.2
- PR-12049 - KAFKA-10888: Sticky partition leads to uneven produce msg (#12049)
- PR-12028 - KAFKA-13804: Output the reason why broker exit unexpectedly during startup (#12028)
- PR-11773 - KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618) (#11773)
- PR-12127 - KAFKA-13785: [8/N][emit final] time-ordered session store (#12127)
- PR-12128 - KAFKA-10199: Implement adding active tasks to the state updater (#12128)
- PR-12029 - KAFKA-13815: Avoid reinitialization for a replica that is being deleted (#12029)
- PR-12106 - KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
- PR-709 - : Update jackson packages to 2.13.2
- PR-12100 - KAFKA-13785: [6/N][Emit final] Copy: Emit final for TimeWindowedKStreamImpl (#12100)
- PR-12072 - KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
- PR-12111 - KAFKA-13865: Fix ResponseSendTimeMs metric in RequestChannel is removed twice (#12111)
- PR-11955 - KAFKA-12380 shutdown Executor in Connect’s Worker when closed (#11955)
- PR-12096 - KAFKA-13794: Fix comparator of inflightBatchesBySequence in TransactionsManager (round 3) (#12096)
- PR-12064 - KAFKA-12841: Remove an additional call of onAcknowledgement (#12064)
- PR-12092 - KAFKA-13834: add test coverage for RecordAccumulatorTest (#12092)
- PR-12090 - KAFKA-13852: Kafka Acl documentation bug for wildcard ‘*’ (#12090)
- PR-12075 - KAFKA-13841: Fix a case where we were unable to place on fenced brokers in KRaft mode (#12075)
- PR-12066 - KAFKA-13834: fix drain batch starving issue (#12066)
- PR-11703 - KAFKA-13588: consolidate changelogFor methods to simplify the generation of internal topic names (#11703)
- PR-12030 - KAFKA-13785: [5/N][emit final] cache for time ordered window store (#12030)
- PR-12052 - KAFKA-13799: Improve documentation for Kafka zero-copy (#12052)
- PR-12004 - KAFKA-10095: Add stricter assertion in LogCleanerManagerTest (#12004)
- PR-12063 - KAFKA-13835: Fix two bugs related to dynamic broker configs in KRaft (#12063)
- PR-11993 - KAFKA-13654: Extend KStream process with new Processor API (#11993)
- PR-11681 - KAFKA-8785: fix request timeout by waiting for metadata cache up-to-date (#11681)
- PR-12033 - KAFKA-13807: Fix incrementalAlterConfig and refactor some things (#12033)
- PR-11945 - KAFKA-13769: Explicitly route FK join results to correct partitions (#11945)
- PR-12055 - [MINOR] Update upgrade documentation for 3.2 (#12055)
- PR-12036 - KAFKA-13823 Feature flag changes from KIP-778 (#12036)
- PR-10472 - KAFKA-12613: Fix inconsistent validation logic between KafkaConfig and LogConfig (#10472)
- 87aa8259 - KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
- PR-12031 - KAFKA-13651; Add audit logging to StandardAuthorizer (#12031)
- PR-12018 - KAFKA-13542: Add rebalance reason in Kafka Streams (#12018)
- PR-12043 - KAFKA-13828; Ensure reasons sent by the consumer are small (#12043)
- PR-11948 - KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
- PR-11939 - KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)
- PR-12006 - KAFKA-13794: Follow up to fix producer batch comparator (#12006)
- PR-11998 - KAFKA-13801: Kafka server does not respect MetricsReporter contract for dynamically configured reporters (#11998)
- PR-11842 - KAFKA-13687: Limiting the amount of bytes to be read in a segment logs (#11842)
- PR-11997 - KAFKA-6204 KAFKA-7402 ProducerInterceptor should implement AutoCloseable (#11997)
- PR-11974 - KAFKA-13763: Improve unit testing coverage and flexibility for IncrementalCooperativeAssignor (#11974)
- PR-11995 - KAFKA-13782; Ensure correct partition added to txn after abort on full batch (#11995)
- PR-11991 - KAFKA-13794; Fix comparator of inflightBatchesBySequence in TransactionManager (#11991)
- PR-11965 - KAFKA-13778: Fetch from follower should never run the preferred read replica selection (#11965)
- PR-11981 - KAFKA-13791: Fix potential race condition in FetchResponse#`fetchData` and forgottenTopics (#11981)
- PR-11941 - KAFKA-13749: CreateTopics in KRaft must return configs (#11941)
- f68f1a97 - Add muckrake mapping for 7.2 release
- baf8976f - Add muckrake mapping for 7.2 release
- PR-11978 - KAFKA-13786: Add a note in`control.plane.listener.name` doc (#11978)
- PR-11950 - KAFKA-12875: Change Log layer segment map mutations to avoid absence of active segment (#11950)
- PR-11829 - KAFKA-13785: add processor metadata to be committed with offset (#11829)
- PR-11928 - fix: make sliding window works without grace period (#kafka-13739) (#11928)
- PR-11953 - KAFKA-13772: Partitions are not correctly re-partitioned when the fetcher thread pool is resized (#11953)
- PR-11971 - KAFKA-13783; Remove reason prefixing in JoinGroupRequest and LeaveGroupRequest (#11971)
- PR-11963 - KAFKA-13777: Fix potential FetchResponse#responseData race condition issue (#11963)
- PR-11908 - KAFKA-13748: Do not include file stream connectors in Connect’s CLASSPATH and plugin.path by default (#11908)
- PR-11743 - KAFKA-13660: Switch log4j12 to reload4j (#11743)
- PR-11962 - KAFKA-13775: - Upgrade jackson-databind to 2.12.6.1 (#11962)
- PR-11967 - Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
- PR-11869 - KAFKA-13719: Fix connector restart cause duplicate tasks (#11869)
- PR-11966 - KAFKA-13418: Support key updates with TLS 1.3 (#11966)
- PR-11923 - KAFKA-6718: Add documentation for KIP-708 (#11923)
- PR-11942 - KAFKA-13767; Fetch from consumers should return immediately when preferred read replica is defined by the leader (#11942)
- PR-11760 - KAFKA-13600: Kafka Streams - Fall back to most caught up client if no caught up clients exist (#11760)
- PR-11949 - KAFKA-4801: don’t verify assignment during broker up and down in testConsumptionWithBrokerFailures (#11949)
- PR-11946 - KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (#11946)
- PR-11805 - KAFKA-13692: include metadata wait time in total blocked time (#11805)
- PR-11940 - KAFKA-13689: optimize the log output of logUnused method (#11940)
- PR-11920 - KAFKA-13672: Race condition in DynamicBrokerConfig (#11920)
- PR-11926 - KAFKA-13714: Fix cache flush position (#11926)
- PR-11933 - KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (#11933)
- PR-11892 - [Emit final][4/N] add time ordered store factory (#11892)
- PR-11932 - Revert “KAFKA-7077: Use default producer settings in Connect Worker (#11475)” (#11932)
- PR-11912 - KAFKA-13752: Uuid compare using equals in java (#11912)
- PR-11796 - KAFKA-13152: Replace “buffered.records.per.partition” with “input.buffer.max.bytes” (#11796)
ksqlDB¶
- PR-9468 - remove cc-docker-ksql from downstream builds
- PR-9567 - fix: use resolved configs in precondition checker
- PR-9394 - fix: fix regex used to extract queryId from threadId metrics tag
- PR-9392 - fix: move udf loading to run before the precondition checker
- PR-9391 - fix: make sure to close clients from precondition checker
- PR-9388 - fix: compare topics not sources
- PR-9393 - fix: use internal topic config for transient queries too
- PR-9389 - refactor: remove AVRO_SCHEMA_ID & SCHEMA_ID from QTT historical plans
- PR-9378 - feat: Add support for four and five column arguments to UDAFs
- PR-9361 - feat: UDAFs with multiple/variadic args
- PR-9366 - feat: Add log, power, and cbrt UDFs
- PR-9351 - refactor: combine yatt input and output nodes into one topic node
- PR-9341 - fix: make api client recognize ddl warnings better
- PR-9360 - refactor: Materialized to MaterializedFactory
- ef65f924 - Addressed Jim’s comments
- PR-9336 - fix: Map invalid casts to null.
- 3d2a56f1 - Updated documentation for detailed processing log in KsqlDB.io project
- PR-9337 - bugfix: remove log4j from the classpath (#9334)
- PR-9321 - fix: allow YATT to insert into and check contents of DDL sources
- e986f668 - fix: Create a KsqlSerializationException class
- 56dddbb1 - fix: classify KsqlSerializationException as USER error based on topic(KSE-1045)
- PR-9327 - Bump changelog version heading to 0.27.1
- PR-9130 - fix: use JsonSchemaConverter to support JSON anyOf types
- PR-9314 - fix: Allows functions which return maps to be dereferenced again.
- 6f656c0a - fix spotbugs
- PR-9283 - feat: Adding ksqlDB Query Status metric.
- a700c7ec - refactor: rename getAuthToken to getAuthHeader
- PR-9300 - feat: refresh service context and topic client in precondition checker
- PR-9272 - refactor: Migrate legacy UDAFs to use current annotations
- PR-9203 - feat: Support pausing/resuming persistent queries
- dcfe7941 - fix: Return proper status code for QPS ratelimit.
- PR-9277 - fix: DESCRIBE FUNCTION failing for annotated UDAFs with initial args
- PR-9255 - fix: change auth token provider to accept token strings instead of principals
- PR-9260 - fix: Excludes Guava from Guava-retrying in order to manage Guava depe
- PR-9246 - MINOR: improve error message for missing key
- PR-9248 - fix: Removing reverted configuration org.apache.kafka.streams.Streams
- PR-9239 - fix: add getAuthToken method to AuthenticationPlugin interface
- PR-9141 - feat: enable new emit-final implementation
- PR-9225 - fix: change consumer_group_member_id tag to just member to match Druid label name
- PR-9213 - feat: Added numerous trigonometric UDFs
- PR-9209 - fix:CAST function works with ISO-8601 timestamps with a trailing ‘Z’
- PR-9215 - fix: convert topic tag name and add consumer group member id tag to ThroughputTotalMetrics
- PR-9211 - fix: change group name and extend CumulativeSum in ThroughputMetricsReporter
- PR-9168 - feat: introduce ATTR aggregation function
- PR-9205 - fix: reset collector before reconfiguring
- PR-9180 - fix: add BYTES support for KAFKA format
- PR-9186 - fix: Allows results from CAST to compared.
- PR-9167 - fix: ambiguous reference to close issue
- PR-9134 - feat: cull the list of API consumable/editable properties
- PR-9144 - fix: move misplaced query-level configs to the correct list
- PR-9145 - fix: revert default /query-stream Content-Type to application/vnd.ksqlapi.delimited.v1 from application/vnd.ksql.v1+protobuf
- PR-9127 - fix: Fixes a few null handling bugs
- PR-9103 - feat: add ProtoBuf as a content type for pull queries over /query-stream endpoint
- PR-9045 - feat: add metric for query restarts
- PR-9120 - feat: Support all wildcard (*) on struct reference syntax
- PR-9105 - feat: clean up processing log metric
- PR-9107 - feat: add support for assert statements to migration tool
- PR-9099 - feat: add assert methods to java client
- PR-9035 - feat: add metric that’s emitted when processing log emits an error
- PR-9096 - feat: automatically build confluent cloud image on every master merge
- PR-9036 - fix: re fetch streams for each materializationProviderBuilder
- PR-9091 - feat: add ASSERT SCHEMA statement
- PR-9078 - Add PROTOBUF_NOSR
- 5423da9d - refactor: Fix checkstyle & make naming consistent
- PR-9086 - feat: assert not exists topic
- be09c0a6 - refactor: Set supportedArgs with string & bytes for max/min agg functions
- PR-9072 - fix: classify SR missing subject and access rights query errors as USER errors
- PR-9066 - feat: add ASSERT TOPIC command
- 030f2147 - feat: enable max/min udaf for string & bytes data types
- f2877e8d - fix: classify KsqlFunctionException as USER error
- 9e9d10e7 - fix: throw KsqlFunctionException while aggregating in sum udaf #9052
- 1bb24c31 - feat: migrate java client to use application/vnd.ksql.v1+json format
- PR-9047 - fix: INSERT/VALUES on a stream with SCHEMA_ID/SCHEMA_FULL_NAME fails
- PR-9026 - feat: support checking preconditions before starting core app
- PR-9040 - fix: use the engine’s KsqlConfig to build queries
- PR-9038 - fix: INSERT fails when serializing Proto/Avro nested Structs
- PR-9041 - build: exclude reload4j
- PR-9032 - fix: register state listener after restarting runtime
- PR-8986 - feat: allow aggregations without group bys
- PR-9028 - fix: remove double quotes from json_records function
- PR-8933 - fix: Create stream fails when multiple Protobuf schema definitions exist
- PR-9023 - fix: include header columns when injecting schemas
- PR-8918 - fix: Guard null struct dereferencing inside function calls
- PR-8984 - fix: INSERT VALUES fail when SR schema has a non-default name
- PR-9014 - fix: fail validation on create connector if connector already exists
- PR-8923 - fix: shared runtimes calculate cache size for validation properly
- PR-8999 - fix: move create connector validation to validate phase
- PR-8998 - fix: remove ErrorEntity and throw on connector error instead
- PR-8983 - Revert “feat: Allow to plug-in custom error handling for Connect serv
- PR-8977 - Improved/fixed aggregate function error messages.
- PR-8949 - feat: allow STREAMS with no key
- PR-8926 - fix: Repartition RHS of a FK join if it uses SR schema
- PR-8973 - fix: wait longer while waiting for expected spq
- PR-8947 - revert: consistency APIs
REST Proxy¶
- PR-1049 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-1044 - APIF-2768: Workaround for “Failed to bind to 0.0.0.0/0.0.0.0:9998” error.
- 87452021 - Run mvn spotless:apply
- PR-1039 - KREST-2655 Simple version of disconnect
- PR-1006 - KREST-4932: Add produce record rate-limited metrics.
- PR-1023 - KREST-5637 Use a Meter as recommended by observability and the right Metrics Object
- PR-1024 - Fix Checkstyle empty catch block error.
- PR-1022 - KREST-4067 Introduce import control to Kafka REST.
- PR-978 - KREST-4591 add topic recreate logic to cluster test harness
- PR-985 - KREST-4687 avro consume still fails
- PR-1009 - KREST-5732 Refactor to allow us to override producer metrics in ce-kafka-rest
- PR-1014 - KREST-5830: Create a lazy wrapper around MappingIterator for Produce Action.
- PR-1012 - KREST-5637 Move to cumulative sum for billing metrics
- PR-1008 - KREST-5637 Add count based byte metrics and move tracking earlier
- PR-1002 - KREST-5385: Add error_code to produce responses.
- PR-1003 - Remove ProducerPool.
- PR-1001 - Remove KafkaRestContext#getProducerPool.
- PR-350 - Cherry-pick https://github.com/confluentinc/rest-utils/pull/349 to 5.4.x
- 3d1250af - APIF-2739: Upgrade Jetty to 9.4.48.v20220622.
- d93674fb - Do not create a new sensor for error count
- 88556559 - Set sensors to be expired in 1 hr
- ff3595a0 - DGS-4220: fix request tag based metrics
- PR-343 - Update CODEOWNERS for APIF team
- PR-320 - APIF-2714: Switch from confluent-log4j to reload4j (5.4.x)
- PR-329 - APIF-2705: Update jersey version to 2.36.
- PR-316 - MMA-12033 Fix the connections limits test
- PR-315 - KREST-4977 Allow limiting the number of active connections.
- PR-310 - KREST-4450 500 error when topic not present
Schema Registry¶
NOTE: DGS-4389 added support for Protobuf custom options, which may change the behavior of schema lookups. To retain the old behavior in the Protobuf serializer, set schema.format=ignore_extensions.
- PR-2399 - DGS-5084 Ignore compat check in IMPORT mode
- PR-2395 - DGS-4971 Handle map types with enhanced.protobuf.schema.support
- PR-2389 - MINOR: Update ErrorMessage description
- PR-2388 - MINOR: Add ErrorMessage OpenAPI descriptions
- PR-2387 - MINOR: Add OpenAPI description for deleteGlobalConfig
- PR-2385 - Add OpenAPI operation tags
- PR-2378 - Migrate PowerMock to Mockito in RestServiceTest (#2372)
- PR-2381 - Adding timer in onJoinPrepare
- 0a9fb7d8 - Renamed DocumentedName to RootResource.java
- PR-2376 - DGS-4768 Fix reserved ranges for Protobuf enums
- 646cf301 - Added DocumentedName to RootResource.java
- PR-2373 - DGS-4724 Qualify names and merge maps when normalizing custom options
- 2fd5e6e9 - Remove deprecated methods in SchemaRegistryMetric
- PR-2364 - Adding timer in onJoinPrepare
- PR-2359 - DGS-4395 Fix message indexes of normalized Protobuf with map
- PR-2357 - DGS-4389 Add support for Protobuf v2 extensions
- PR-2353 - [DGS-4361] Added “/schemas/ids/{id}/schema” endpoint
- PR-2351 - DGS-4358 Fix NPE in Protobuf converter for null map value
- PR-2346 - Optimize sync call
- PR-2345 - Add leader change listeners
- PR-2342 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-2339 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-2336 - DGS-4249 Fix name resolution during Protobuf normalization
- PR-2331 - DGS-4172 Bound size of Avro datumReader/Writer caches
- PR-2329 - DGS-4163 Pass topic to schema formatters
- PR-2328 - DGS-4162 Handle refs at different levels in Protobuf converter
- PR-2327 - DGS-1648 Allow Protobuf msg fullname to be passed for console producer
- PR-2326 - DGS-4134 Add config to ignore default for nullable fields
- PR-2313 - DGS-3862 Upgrade to spotbugs 4.7.0
- PR-2305 - Change everit-json-schema coordinates to reflect artifact published on Central
- PR-2289 - Added Custom Annotation
- PR-2262 - Remove static reference over java.util.Random
- PR-2198 - Set-compatibility Goal
- PR-2197 - Adding folder support for Test Local Compatibility maven plugin
Commercial Features¶
Confluent Server¶
- METRICS-4649 Refactor Remote Configuration activeFilters Semantics for 7.3.x
- KMETA-451; Allow broker registration with older confluent.metadata.version
- KMETA-448; Fix auto leader balancing of linked partitions
- Exclude non-fips dependency(bcprov-ext-jdk15on) from trogdor project (#7673)
- Exclude fips and non-fips bouncycastle dependency jar from connect packages (#7668) to 7.3.x
- Back port fix for clm test to 7.3.x
- Cherry-pick excluded bc jdk15 (non fips) from :ce-broker-plugins as it uses non-fips bc jars during compile time to 7.3.x
- KMETA-436; Fix compatibility break with default principal schema
- KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
- Remote Config bug fixes (#7495)
- KGLOBAL-2203: Fix MRC assignment logic so rack positioning in placeme
- Add log message when skipping deletion
- Cherry-pick KGLOBAL-1812:cb3b61c3c7e2d60b17feb2deb6580dfd2fec4973 to 7.3.x
- Cherry-pick KGLOBAL-1777:4d6485ea189094f7c1935de786c607034f91538b to 7.3.x
- Bump version to 3.3.0
- CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
- RCCA-8564: log a warning if LDAP login fails becuase of network issues
- CIAM-2290: Upgrade bc fips drivers
- KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
- KGLOBAL-1812: Fix thread leak in cluster linking test. (#7188)
- Increase timeout, correct error message returned for addBroker test
- KGLOBAL-1797: Remove rack mixing feature flag (#7421)
- Cherry-pick KGLOBAL-2127:72723ca0606d24a3894b58d4fad3eace9b0a07a0 to 7.3.x
- CONFLUENT: implement ControllerLoadTime metric for KRaft
- Cherry-pick Subset Partitioner Fix to 7.3.x
- KENGINE-229; Partition.isReplicaIsrEligible is misused in Partition.maybeIncrementLeaderHW
- KENGINE-212: batch transaction requests.
- KENGINE-211: Add a metric to record the avg latency for a transaction to transit from PreCommit to Commit.
- Remove the html end tag from upgrade.html
- KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case
- KMETA-290 Metadata shell supports snapshot and log
- KGLOBAL-1803 reset lastCaughtupTime when mirror leader appends to its
- Don’t include Server info as an HTTP response header in the Connect REST API (#7264)
- Add Javadoc to undocumented public APIs in SBC
- Update log level in RestClient
- CP 7.3 - Telemetry Reporter Remote Configuration [METRICS-4195][METRICS-4186][METRICS-4189]
- KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Control
- RCCA-7224: Revert CIAM-1419 for 7.3.x
- Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
- KGLOBAL-1952: Attempt to shutdown both clusters in CL tests even if one fails to avoid thread leak impacting other tests
- KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing part
- KMETA-329; Workaround for AlterPartition regression on stale controllers
- cherrypick KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode
- Fix typo in offset writer thread config name
- Fix 7.3.x compilation
- KAFKA-14114: Add Metadata Error Related Metrics
- KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
- [METRICS-4507] Add cluster linking metrics to Telemetry Reporter
- KAFKA-14107: Upgrade Jetty version.
- KAFKA-14114: Add Metadata Error Related Metrics
- Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
- KMETA-321: Disable integration tests with co-located KRaft mode
- Avoid sending partial updates during LDAP group manager startup
- KSTORAGE-2280: Added a compaction CPU Utilzation Metric
- KDATA-484: Adding stateBeforeDeletion field
- [AUTHN-1097] Disable subject claim requirement in OAuth tokens
- KENGINE-219: Fix AlterPartition idempotency
- DGS-4302: ZKTopicMetadataCollector should avoid throwing exceptions
- [minor] Update README with PR docker image push
- Revert “CNKAF-1132: Record & unrecord total-throttle-time metric (#25
- KAFKALESS-1247: Fix CPU metric for non-Linux system and ConfluentTelemetryReporterSamplerIntegrationTest timeout issue
- KAFKALESS-737: Add system test for triggerEvenClusterLoad command
- SBC: Add Resource Optimization Detector framework
- Change CruiseControlMetricsProcessor log levels
- Build and push dirty images to GAR nonprod repo
- KAFKA-6945: KIP-373, allow users to create delegation token for others.
- KGLOBAL-1576 Refactor SaslServerAuthenticator auth to relay NetworkRegionId to SaslServer
- KAFKALESS-1261: Use newly added follower fetch rate metric
- Upgrading netty to v4.1.79.Final #754
- AUDIT-1139: Reducing the verbosity of NBKE
- Add metrics for monitoring of Kafka Management events
- KGLOBAL-1732: Update SBC to handle sync replicas and observers on the same rack
- Address review comments
- Address Yash Mayya’s comments around duplicate tests in AbstractWorkerSourceTaskTest and WorkerSourceTaskTest
- Upgrading netty version to 4.1.79.Final
- Upgrading aws-java-sdk-s3 to v1.12.268
- KAFKALESS-1261: Add fetch count metric at topic level
- Hotset Size Based Retention Breach Deletion of Compacted Segments
- Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 - remove Operation annotation
- Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 KAFKA-10000: Integration tests
- Cherrypick 7098f04c3d5a30a6c16291dd78aa98694ce56e0b - fix WorkerSourceTaskTest
- Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 - Fix missing tracer invocations
- Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 and fix tests and refactor builder related changes in Worker
- Cherrypick AK commit 6853d63e4de03d679978add576aa0977cecc053a - Fix test failures related to producerConfigs and adminConfigs
- Remove updateConnectorConfig method
- Fix MethodLength related to DistributedHerder
- Cherrypick AK commit 603502bf5fb78983434a1a44ccc15a49ef6942b0
- Cherrypick AK commit a110f1fe852ae8c958a8c64b0736a9bb0617338e - Merge header configs along with producer, consumer and admin configs
- KAFKALESS-738: Even cluster load plan system tests
- Cherrypick KAFKA-13803
- Fix compilation issue in AbstractHerder, ReassignPartitionsCommand and use latest sink/source config from AbstractHerder’s ce-kafka/master
- Revert “KAFKA-10000: Add new preflight connector config validation logic (#11776)”
- Modified NonKafkaLogicalClusterMetadata parseLCM to be in sync with r
- KAFKA-13649: Implement early.start.listeners and fix StandardAuthoriz
- Cherry pick KAFKA-13474 on master to 6.1.x
- CONFLUENT: add code for deleting the Acls from the pod by passing BOOTSTRAP_SERVERS_CONFIG, lkcID and principals
- KREST-6986: Only add stats to sensor if metrics are not already registered
- KGLOBAL-1724: Wrong source topic name in DescribeMirrorsResponse.
- KDATA-454: add raft test annotation for clm test
- KGLOBAL-1786: Trodgor task for consumer group operations and listing offsets
- Sync this version of proto file
- CIAM-2169: Add KsqlCluster: Describe to EnvMV & CCMV
- Exponential backoff for automatic alter leadership calls
- KMETA-295: RuntimeException in TelemetryReporter init on KRaft controllers
- KAFKALESS-1216: Add ReplicaEntity and ReplicaMetricSample classes
- KAFKALESS-1167: Configuration of incremental balancing
- [METRICS-4509] Add Consumer Lag Offsets Metric to Telemetry-Reporter Whitelist
- CIAM-1503: Ability to de-code message headers of auth-topic
- AUTHN-1074: Support hierarchy for provider/pool
- KCFUN-199: Convert dynamic quotas tests to KRaft
- KAFKALESS-1217: Move TopicPartition to PartitionInfo
- KGLOBAL-1727 : add NetworkRegionId tag to CL SaslAuthenticate request
- AUTHN-974: Separate identity provider and pool resource types
- KCFUN-506: Improve the quota allocation algorithm by capping at broker limit
- KAFKA-14020: Performance regression in Producer (#12365)
- KAFAKALESS-752: CLI for ComputeEvenClusterLoadPlan
- KAFKALESS-1258: Use ConfigurationsImage during KRaft SBC startup and introduce in-memory BalancerEnabledConfig and consolidate/abstract SBC enablement there
- KAFKALESS-1268: Add metrics for Databalancer engine
- Added resourceId flag in aclCommand to list the ACLs in new format using Kafka CLI
- KAFKALESS-1270: Pause for a bit while Executor reservation acquirement aborts another executor run
- Update tenant transformations for kafka management audit logs
- SD-628: Added resource type and roles for Stream Designer pipelines
- CPKAFKA-8929 Disabling KRAFT failing test
- [METRICS-4508] Add Kafka Controller Preferred Replica Imbalance Count Metric
- KSTORAGE-2284, KSTORAGE-2285, KSTORAGE-2286: measure log append rate / data size / latency
- KCFUN-495: Make Dynamic Quota reconfigurible
- [METRICS-4503] Ignore Topology Change if Preferred Partition Leader Doesn’t Change for RandomBrokerPartitionSubsetPartitioner
- KGLOBAL-1730: Add support for replica placement file with sync replicas and observers on the same rack
- ReplicaManager should use brokerState instead of isShuttingDown to fence partitions followed by shutting down broker
- Fix connect_rest_test.py after introduction of new source configs
- DGS-4151:Update Rolebindings for some SR/DG related roles/operations
- Update CODEOWNERS
- CONFLUENT: Add metadata team to .github/CODEOWNERS for the old controller
- CONFLUENT: Avoid materializing collection in AbstractFetcherManager to compute sum
- AUTHN-1036: Change poolId principal prefix to always show User:
- KSTORAGE-1696: Non contextual or confusing tiering logs seen frequently in Confluent Platform
- KGLOBAL-1584: Add time to stop mirror topic metric
- KAFKALESS-1247 Temporarily disable ConfluentTelemetryReporterSamplerIntegrationTest.testSampler
- CIAM-2156: Add UI viewing permissions to SRResourceOwner, SRDeveloperX roles
- KGLOBAL-1658: Add source topic id to kafka-mirrors –describe output
- KC-2195: Implementing an HTTP API on the KRaft Controllers to check the quorum health
- AUTHN-881: Added support for poolId in authz audit event.
- KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
- KAFKALESS-734: Backend implementation for ComputeEvenClusterLoadPlan
- KDATA-476 Minor Remove unneeded wrap/unwrap in Option
- KAFKALESS-1254: Disable BrokerFailureDetectorTest.testLoadFailedBrokers
- CIAM-1518: KSQL modeled as a “cluster” for RBAC cloud
- Disable failed test CLIENTS-2345
- Disabled failed test CPKAFKA-6522
- CIAM-2178 Split Role Def File for SDS into KSQL SDS and SR SDS
- (JIRA ID : SEC-3593) CP LDAP - Enable Configurable Case Sensitivity for Authorisation
- AUTHN-974: Add providerId to pool events
- KGLOBAL-1769: logging node identifier in the error message and making
- KSTORAGE-2267: lingering storage metrics on deleted logs
- KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
- KAFKALESS-1248: Temporarily disable testSelfHealingWithIgnoredBrokersPresentWithReplicaPlacements
- KAFKALESS-1222: Enable BrokerFailureDetectorTest.testPartialClusterFa
- DGS-3944: Add metrics for ZKMetadataCollector
- KAFKALESS-1230: Enable ReplicaPlacementSelfHealingTest
- KDATA-480 Abstract out Retry Policy from ObjectStoreUtils class
- [skip secret scan] KGLOBAL-1366: Enable CL system tests in KRaft mode
- Collect garbage collection metrics in TelemetryReporter [METRICS-4470]
- INIT-599 - Allow OrgAdmins to delete the whole Organization in cloud_rbac_roles
- [KPERF-454] Batch optimization for committing consumer group offsets.
- KMETA-83 Support for StandardAuthorizer benchmark
- DP-8085 - Migrate to Semaphore self-hosted agent
- KMETA-185: Explicitly start metric reporters in remote KRaft controllers
- Update CODEOWNERS for ce-metrics
- KMETA-249; Ensure linux metrics collected on remote controllers
- Add Cloud resource type of CLUSTER_LINK
- KDATA-392: restore system test to support kraft
- KAFKALESS-1227: Add NPE handling and consider all detection goals ski
- CIAM-2083: Move SDS rbac roles into separate json
- KSTORAGE-2279: Txn markers should not be removed by matching records in the offset map
- Properly gather partition information when detected topics with incon
- KGLOBAL-1351: Fix Incorrect prefixed-destination-link-count
- Bug fix for system tests.
- KGLOBAL-1085: Use default timeout for stop mirror topic.
- KMETA-239 Fix missing ControllerApis error handling
- Extract and Introduce libs for SDS engine to make authnz decisions outside of Kafka server
- DGS-3331, DGS-3332 Get topic config change and snapshot in ZK
- KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
- KAFKALESS-754: Allow altering SBC goals configs (confluent.balancer.rebalancing.goals and confluent.balancer.triggering.goals) dynamically
- Add detailed audit log integration test
- Fix for KafkaAuthStoreTest.testCacheFailureStatus
- MINOR: ignore consecutive handleMigration and trackInitLeader calls in TierDeletedPartitionsCoordinator
- KMETA-186 Fix AlterConfigPolicy usage in KRaft
- KAFKALESS-1221: Ensure SBC does not compute plans when reassignments exist
- KGLOBAL-1649: Compatibility is broken for createClusterLink requests in KRaft mode.
- KSTORAGE-2258: implement bucket storage probe metrics
- KSTORAGE-2137: enable FTPS cleanup in ce-kafka system tests
- [AUTHN-954] Add temp fields to IdentityPoolValue
- KAFKALESS-1218: Use Linux system cpu utilization
- KAFKALESS-733: Add ComputeEvenClusterLoadPlan Kafka admin API
- KAFKALESS-839 exclusion-aware ReplicaPlacementGoal
- fixed import order
- KCFUN-506: Set a minimum value for reported quota consumption
- KENGINE-194: Topic IDs not added to in sync fetcher pool
- KMETA-213: Fix NPE caused by missing null check in SnapshottableHashTable::mergeFrom()
- resolved failing tests
- KAFKALESS-1227: Disable test_topic_rebalance for ZK
- Adding tenant partition availability metric
- KCFUN-386, KCFUN-392, KCFUN-253: Setting a hard limit on number of partitions and topics per cluster. Partial update # of topics and partitions for in-flight requests
- Fix for test testWriterReelectionBeforeProduceComplete
- Fix for test testWriterReelectionBeforeProduceComplete.
- KAFKALESS-1222: Disable BrokerFailureDetectorTest#testPartialClusterFailure in ZK mode
- KAFKALESS-1221: Ignore reassignemnts cancel plan computation test
- KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs
- KSTORAGE-2232: respect endOffset parameter when building offset map
- KAFKALESS-1207 Fix numBrokers created in onPrem case at ClusterModelPBTUtils
- KMETA-131 Cluster Linking metadata.version support
- AUTHN-908: Disable trust policy cache
- KSTORAGE-2249: update MergedLog.read to throw NotLeaderOrFollowerException on spurious OffsetOutOfRangeException
- reverted the LocalLog info logging that came from AK as its already logged as part of MergedLog
- Change DP schema request field config key default behavior
- CNKAF-1195: Don’t compute plan while reassignments are present
- KAFKALESS-1189: Fix SbcUpdateMetadataEvent to not override old metadata or get stuck in a loop
- Add support for slow logs in request logging
- reverted manually applied change from Unified as the logic to increment log start offset is different in MergedLog so this change is not needed
- CONFLUENT: Move license validator after startup completion
- KGLOBAL-1613: Persistent connection is not available (#6679)
- KGLOBAL-1614: Transform ACL binding filter in ClusterLinkSyncAcls when in multi-tenant env and add ACL migration semantics integ test to MultiTenantClusterLinkTest
- CONFLUENT: Log at debug level when pid unavailable
- Integrate Kafka management events with audit log provider
- KGLOBAL-1559: Do not fetch the metadata information under MetadataManager’s lock since the call is blocking.
- KGLOBAL-771: Source Initiated Links for KRaft.
- KAFKALESS-879: Get rid of capacityFor method
- KAFKALESS-1202: Log error from EvenClusterLoadStateManager whenever registering an event with exception
- KGLOBAL-1507: Filter out _schemas during cluster linking auto-mirroring
- KGLOBAL-1613: Persistent connection is not available
- KAFKALESS-1133: Add log to capture rack aware failure
- Modify DP schema request field to config key
- Refactor CLM tests
- KDATA-432: fix tier state fence restore test for Azure
- KPLATFORM-543: Move startup completion to end of startup sequence
- KGLOBAL-1611: Handle ClusterLinkDisabledException in ClusterLinkAutoMirroring when determining if mirror topics need to be filtered
- KGLOBAL-1473: converted ClusterLinkDestConnectionManagerTest and ClusterLinkAutoMirroringTest from easyMock to mockito
- KSTORAGE-1965: Not all bytes were read from the S3ObjectInputStream
- KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2)
- KSTORAGE-2221: persist bounded base offset info of segments into FTPS
- KMETA-203 Add “confluent.metadata.version”
- KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
- DGS-3504:Role changes for DG Catalog RBAC
- KDATA-180: Add internal durability audit metrics
- Kdata 448 refine restore rto
- KAFKALESS-732: Introduce ComputeEvenLoadPlan Protocol
- CONFLUENT: EasyMock -> Mockito conversion for most ce-kafka core tests
- Maintain mapping of userId to resourceId after deletion of API Key
- CIAM-1621: Make RBAC binding scopes support a tree structure
- Remove events from ce-audit as code owner
- RCCA-5913: provide exceptional use utility for mutation of topic ids
- MINOR: Fix broker load tests in MultiTenantKafkaIntegrationTest
- KGLOBAL-1561: Close reverse connections when deleting source side link.
- Fix CLM liveness metric
- KAFKALESS-751: Introduce CLI for trigger even cluster load
- Cherry-pick KAFKA-13935
- KDATA-455: address TopicIdPartition is None when rolling segment
- KGLOBAL-1050: Remove cache in ClusterLinkSyncTopicsConfigs task
- KAFKALESS-1186: Fix race condition in initializing BalancerStatusTracker
- MINOR: Convert oauth tests
- Add capability to enable trace record schema through DataPreview request
- Add principal to authentication failure audit log
- KCFUN-485: Memory leak in ThrottledChannelReaper
- KGLOBAL-1559: Resolve the deadlock between cluster link manager and cluster link metadata manager interactions.
- KAFKALESS-1177: Detect inconsistent replication factor by logging and emitting a metric
- Support dynamic config for resourceId in TenantAclProvider
- KDATA-388: RPO metrics for restore
- KGLOBAL-1451:Remove retry-time-based mirror failure when source topicids are known
- KC-2238: Reduce max.block.ms for telemetry producers in system tests to reduce broker shutdown time
- KSTORAGE-1699: Auto disable segment deletion throttler during low free disk space
- [AUDIT-1015] Fix producer emit operation future value to complete with true on success.
- Readd metrics plumbing for request handler avg idle percent metric.
- KGLOBAL-1485: Fail fast for persistent connection to non-coordinator
- CCLOG-1790 Connector Developer roles should be able to access metrics
- QEC-7888: Ensure that cluster link deletions in progress are completed on broker restart
- KAFKALESS-731: Correct the log for the EvenClusterLoad status value being verified.
- KAFKALESS-1109: Awakened events should execute before those in-queue
- KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft
- [skip secret scan] KAFKALESS-731: Update even cluster load state manager as design rework
- MINOR: Add AWS ECR setup to image building section
- KAFKALESS-1110: Handle rebalance drizzles with fixed-but-not-violated trigger goals.
- Bump telemetry client version to 3.163.0 for minor logging fix
- MINOR: Convert more integration tests
- Authn-526: Kafka AuthN rate and Latency Metrics per saslMechanism
- MINOR: SslCertificateIntegrationTest conversion
- Make resource id support config as dynamic config
- Fix exception handling in RBAC writer coordinator
- Remove reference to cc-deployer.mk
- AUTHN-813: Replace trust policy antlr parser with cel parser
- KCFUN-463: dynamic config to allow client supplied acks setting to be overridden [skip secret scan]
- KCFUN-481: Remove SocketServer state for disconnected channels when IP throttling expires
- KAFKALESS-525: Test both flavors of shouldShutdown in AddBroker tests.
- KAFKALESS-837: Rename Broker#State to Broker#Strategy
- Revert “CIAM-1419: Restructure DefaultAuthCache to lookup access rules for principal (#4758)”
- Revert “CONFLUENT: Rename DataPlaneAuthCache class name to CloudAuthCache (#4885)”
- Update source and serviceName CRN format for authentication failure events
- Revert “KDATA-404: New fields, and structure to be added to Durability Database”
- KDATA-450 CLM support for Azure object store
- KCFUN-128: Decouple recordThreadIdleRatio from request path
- KDATA-421: Add missing azure backend for tier storage system tests
- Test failure ‘KafkaService’ has no attribute ‘ACL_AUTHORIZER’
- Artifactory Migration
- MINOR: fix prefer.tier.fetch config for compacted topic in log_compaction_test system test
- KC-2144: Revert “AUTHN-280: Java client: implement async auth (#4659)”
- KC-2202 Disable kraft cluster linking system tests 7 2
- KENGINE-181: add recoverable partitions to GroupMetadataManager metrics
- KAFKALESS-1063: Allow more flexible leadership exclusion reasons.
- KGLOBAL-1547: Bump ClusterLinkRecord version since we have added a non-nullable link mode
- KGLOBAL-1415: Increase cluster link reconnect backoff max value
- MINOR: Add metadata team as code owners
- MINOR: Log PROXY protocol address when auth fails
- KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (KMETA-108)
- CIAM-2043 Separate permission for OwnKafkaClusterApiKey
- KGLOBAL-1394: Broker fails to start due to deleted cluster link (#6509)
- KSTORAGE-2060: Ignore missing file during log dir deletion
- [MMA-5228] allow operator to describe all topics
- CPKAFKA-8728, CPKAFKA-8729, CPKAFKA-8584 oauth system test fix
- KCFUN-112: Use a separate run method for interbroker network threads
- KGLOBAL-1394: Broker fails to start due to deleted cluster link
- [skip secret scan] KDATA-348: New events from Tier Metadata Snapshot Initiate and Complete
- KGLOBAL-1546: Fix race condition in acls method in StandardAuthorizerData
- KGLOBAL-1486: Deflake ClusterLinkTest.test_offset_migration_early_destination_group_start
- Efficiency Metrics for CLM
- Part7: EasyMock to mockito migration for CL tests
- KGLOBAL-1480: Part6 easyMock to mockito conversion
- KC-2202 Disable CL in KRaft mode for CP 7.2
- KMETA-149; Ensure forwarded requests are sampled for logging
- KGLOBAL-1506: Keep CreateClusterLinkPolicy state in sync with metadata log
- KMETA-160 Add Confluent records to metadata shell
- KC-2223: Disallow enabling SBC with KRaft in CP 7.2 release
- CloudClusterMetricsViewer should be able to view Connector Metrics
- Rename kafka config of user resource id support
- KDATA-353: Add feature flag configuration for FTPS snapshots and dynamic support
- Make the sasl handshake and mechanism max receive sizes configurable
- KC-2202; Disallow TS and KRaft in 7.2.x
- KMETA-172: Ensure partition epoch bumped before ISR expansion
- DGS-3640: Include leader epoch for MetadataImageListener::onLeaderUpdate
- KGLOBAL-1419: Add KRaft support for storing cluster link IDs with ACLs
- graduate MetricsViewer roles to public namespace
- KGLOBAL-1478: Part4 convert from EasyMock to Mockito
- Add view permission on cluster for connector roles
- KGLOBAL-1479: Part5 easyMock to mockito conversion
- [skip secret scan] Log consolidated final states of brokers
- KC-1907; Audit log wiring for the KRaft controller
- retention_stress_test: reliably spread writes across partitions
- MINOR: disable test case testBasicRetention in kraft mode
- KAFKALESS-529: Reset create time on new broker removal operation
- KGLOBAL-1490: Fix bug with mapping of link id to topics
- KGLOBAL-1475: Part3 convert from EasyMock to Mockito
- KGLOBAL-1474: Part2 convert from EasyMock to Mockito
- CONFLUENT: add more validation during KRPC deserialization
- retention_stress_test: follow up formatting fix
- CPKAFKA-5840: tier deletion test doesn’t produce data for the expected duration
- KGLOBAL-1489: Enable AlterMirrorsRequestTest for KRaft mode
- KAFKALESS-1076: Do not process altered exclusion events on inactive databalancer
- Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
- build: kafka system tests should support larger EBS volumes for newer instances (#6371)
- Upgrade Gson
- KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD
- KGLOBAL-1471: Shut down ClusterLinkMetadataThread more gracefully
- KGLOBAL-1357: Enable testSourceTopicRecreateDetectionUsingTopicIds in KRaft mode
- retention_stress_test: long does not exist in python3
- Set hostname verifier to accept everything when identifica
- added support for userResourceID in delete Acls [skip secret scan]
- DGS-3330, DGS-3333 and DGS-3471, run one active TopicMetadataCollector in a cluster
- KGLOBAL-1481: Fix issue where ClusterLinkClearLinkReference does not run on startup in KRaft mode
- KGLOBAL-1483: Source topic id is not populated in create mirror topic request.
- KDATA-352: Cloud API for FTPS Snapshot upload
- CIAM-1793 MetricsApi is available to new DataplaneRoles
- Set hostname verifier to accept everything when identification algorithm is empty
- AUTHN-833: Pass sub and azp as separate validatedExtensions
- KAFKALESS-792: Stop SBC should interrupt SBC startup
- EVENTS-991: Add events team as codeowners of ce-events modules
- Add support for extracting AuditLog Entries from Kafka Request Events
- EVENTS-989: enable use of subset partitioner in events exporter
- build: kafka system tests should support larger EBS volumes for newer instances
- KGLOBAL-1418: Use admin client for creating/deleting ACLs in ClusterLinkClearLinkReference
- Update cloudevent encoding config doc
- CPKAFKA-7262: fully support 7.0 and 7.1 in upgrade, downgrade tests
- KSTORAGE-2180: upgrade flatbuffers to 2.0 to support M1 macs
- KAFKA-13879: Reconnect exponential backoff is ineffective in some cas
- KL-903/934: Enable EvenClusterLoadStatusTest#testBrokerRebalanceWithSelfHealing and SelfHealingAfterEnableTest#testBrokerRebalance
- KSTORAGE-2171: do not adjust baseOffset for compacted segment
- KC-2089: Ensure MZ cluster availability during a network issue (Add leadership priority calls)
- KMETA-16: Add tiered storage topic config change validation to KRaft
- [skip secret scan] KMETA-97 and KMETA-98: Fix compatibility issues with the topic CRUD request versions and bugs in mirror state lookup.
- KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
- KREST-5636 Inject MT secrets store in Kafka HTTP server apps
- KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- EVENTS-874: Create deserializer for LogicalClusterMetadata and headers
- KSTORAGE-2173: skip unnecessary format migration and cleanup during FTPS init
- KSTORAGE-2176: refactor TierTestUtils.deletedSegments
- KCFUN-443: Skip reporting empty quota target[skip secret scan]
- KCFUN-391: Ignore Dynamic Quota if the cluster wide quota is unlimited[skip secret scan]
- KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- KDATA-409: retry when exception of downloading FTPS from kafka pods
- KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
- AUTHN-792: Fix async authn performance regression
- Resolve dependency issue in ce-broker-plugin
- CONFLUENT: Convert ce-kafka-client-plugins to JUnit 5
- KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
- Disable idempotent producer by default in Connect centralized licenses (#6193)
- Add different metric for filtering authorized resources vs authz denied
- KGLOBAL-1355: Fix ClusterLinkFailureTest.testDestinationHighWatermark for KRaft using buffered produce
- KDATA-434: change max wait time for AuditManagerTest
- back port fix for scalabe restore test to 7.2.x
- Update base image to use adoptium java 17.0.3 instead of correto
- Authorizer performance improvements
- KAFKALESS-833 PBT for Self-healing in the presence of ignored brokers [skip secret scan]
- KAFKALESS-1060 Ensure broker#isAlive is used properly
- Disable idempotent producer by default in Connect centralized licenses
- Add Connect team as a codeowner for /connect/
- KAFKALESS-874: Port the last set of broker removal integration tests to run in KRaft
- KGLOBAL-1438: Make confluent.cluster.link.metadata.topic.enable only applicable in ZK mode
- KMETA-127; Get quota integration tests working with KRaft
- KSTORAGE-2153: Handle Azure object store responses during dns failures to ensure they’re retriable
- CONFLUENT: Add error logs to AclAuthorizer
- KMETA-70: Support Confluent’s replica placement plugin with KRaft
- KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
- KAFKALESS-1047: Always acess SBC related Enums by name
- Move partition creation request limit to policy
- METRICS-4064: Subset Partitioner Strategy to Reduce Telemetry Kafka Connections [skip secret scan]
- Add new resourceType and role mapping for RBAC in Health+ API
- KSTORAGE-2149: add compacted.topic.prefer.tier.fetch.ms config
- KSTORAGE-2156: skip dual compaction validation if there are tierable local log segments
- CONFLUENT: Convert rest-authorizer to JUnit 5 and improve its tests
- KC-2116: Migrate tier_unclean_leader_election_test to KRaft
- Update jackson packages to 2.13.2 version
- Fix backward incompatibility issue in license store (#6152)
- KGLOBAL-1349: Enable security for cluster link tests with KRaft
- Update jackson packages to 2.13.2 version
- KSTORAGE-2119: refactor FTPS tools to adapt to cleanup feature
- KSTORAGE-1948: Implement FTPS cleanup logic
- KGLOBAL-1390: Use admin client instead of local authorizer in ClusterLinkSyncAcls for creating/deleting destination ACLs
- KDATA-428: add clusterid parameter in restore system test
- CONFLUENT: Record total usage for cluster link quota even if user quota not set
- KAFKALESS-1089: Increase reason limit for replica exclusions
- KGLOBAL-1297: Revert cluster link sync filter config validation changes
- KAFKA-13660: Switch log4j12 to reload4j (#11743)
- METRICS-514 Kafka event log
- KSTORAGE-2158: exclude internal topics from tier compaction
- KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism (#6235)
- KCFUN-413: Disable user tagging for Client Request Quota
- KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism
- Add kafka-eng as CODEOWNER
- Fix backward incompatibility issue in license store
- KMETA-104: Add internal REST server to KRaft remote controller
- KDATA-313: Segment existence and metadata validation checks
- KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
- Add muckrake mapping for 7.2 release
Security¶
- Removed bcfips from common
- Pin bouncycastle fips dependency in confluent-security-plugins
- Exclude bcpkix-jdk15on as compile dep and add as test dependency
- made SchemaRegistryResourceActionKey private again
- Made nonGlobal request a separate method to have different implementation in cloud
- Backport “Fix a NoClassDefFoundError caused by a missing dependency (#518)” to 7.3.x
- making the SchemaRegistryResourceActionKey and schemaRegistryResourceActionMap protected so that it is visible to the subclasses in cloud plugins repo.
- AUTHN-1087: Upgrade vertx to 4.3.2
- AUTHN-1085: Fix NPE in debug log
- APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- fix: add getAuthHeader method to ksql authorization plugin
- trigger test
- fix tests
- add javadoc
- Trigger Build
- remove unused import
- fix: add getAuthToken method to ksql plugin
- fix: Exclude guava dependencies in ksqldb-rest-app
- fix: get JwtToken from string instead of JwtPrincipal
- Remove redundant config validation in ksql security extension
- checkstyle
- overrides
- rebase to master
- add auth token provider to ksql security extensions
- KSE-1081: Add conf to enable/disable KSQL-SR permission checks
- DGS-3862 Upgrade spotbugs to 4.7.0
- SEC-3245: Migrate from confluent-log4j to reload4j
- Refactor KSQL authorizer and security extension to allow Cloud authZ implementations
- Delete the JAR fetched by MVN wrapper
- Revert “Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision”
- Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision
- KSQL: Wrap RestAuthorizer on a new AuthorizationDecisionMaker class
- Addendum
- Addressing Sergios comments
- Update KsqlSecurityExtensionConfig.java
- Remove Kafka REST ProducerPool.
- KSE-947: Add ksql.service.name to KsqlSecurityExtensionConfig
- KSE-859: Add KSQL config to enable or disable security user impersonation
- update codeowners
- Prepare for removal of Kafka REST ProducerPool.
- feat: add maven wrapper
- Added exclusion since MDC class is failing because of reload4j
- Minor: Fix KafkaYammerMetrics import
Secret Registry¶
- Exclude Non-FIPS dep and use bc-fips dependency
- Handle illegal reflection access under java 9+
- Implement onJoinPrepare with latest AK change in KAFKA-14024
- Migrate from log4j to reload4j for connect-plugin
- CC-17641,CC-17644 Bump jetty version
- APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- ignore new findbugs 4.7.0 detectors causing secret-registry build to fail
- Migrate log4j
- Introduce GitHub CODEOWNERS (Pull Request Reviewers)
Control Center¶
- [MMA-12484][MMA-12483] Introduce Maven Profile for passing different configuration for JDK8 and JDK17
- [MMA-12432] Support Control Center on JDK 17
- MMA-12201 remove slf4j from transitive deps
- MMA-12212: Upgrade jetty-http in blueway
- MMA-12206 upgrade transitive dependency of netty
- MMA-11617: allow for trailing slash in path
- [MMA-12212][MMA-12211] Bump jetty version in blueway
- MMA-12206 upgrade io.netty:netty-codec-http
- MMA-12208 Upgrade okhttp
- MMA-12201 Exclude slf4j in transitive dependency
- MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
- MMA-12201: Use reload4j version properties from common
- MMA-12201 Add logredactor [7.1.x]
- MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
- RCCA-7746: CCloud RBAC user unable to view messages from UI
- MMA-12201 Replace log4j with reload-4j
- MMA-12208 Upgrade okhttp
- MMA-12106 Add new api’s added in kafka streams
- MMA-12010 Suppress errors due to Spotbugs upgrade to 4.7.0
- CIAM-1533 make the default replication factor for Kafka API topic creation configurable
- KEXP-349 experiences code owners
- Remove Kafka REST ProducerPool.
- Prepare for removal of Kafka REST ProducerPool.
MQTT Proxy¶
- Fix 7.3.x test dep for MockFaultHandler
- Update pom.xml
- CC-17620:Remove dependency on slf4j-log4j12 and confluent-log4j
Metadata Service¶
- Fix FIPS deps
- Remove spaces
- Add back newline
- Implement writeTransactionMarkers in DelegateConfluentAdmin
- CIAM-2304: Add unit tests for SecurityAdmin
- Remove unnecessary dep on noop logger
- MMA-12388: Upgrade LD SDK to 5.6.4
- Revert “Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)”””
- Promoted v0.451.0 Extractor to PROD
- CIAM-2201 Close HTTP Application before closing authorizer
- Merge 7.2.x to master
- fix pass
- [AUTHN-1000] stag/prod migrations for provider uniqueness
- RCCA-8245: Fix for invalid consumer group for sync listener
- [AUTHN-1018] Ignore deleted entries on provider/pool lookup
- trust-service: Use v0.47.0 of auditlog library which fixes packaging issue
- [AUTHN-1000] Make identity provider index based on jwks/issuer uniqueness
- CIAM-2219 Re-open DP- 8085 migrate semaphore agent
- Deploy version v0.441.0 of both cc-rbac and Extractor to prod
- CIAM-2237 - Test and build stability prophylactic measures
- [AUTHN-1081] Fix resourcespec parsing
- [AUTHN-1075] Handle server errors gracefully
- AUTHN-541 trust-service: Use recommended methods while testing audit logs
- AUTHN-541 trust-service: Set default=”” for auditlog bootstrap server
- [AUTHN-1070] Enable trust-service feature flags by default
- CIAM-2139 Refactor Extractor code to rename CloudCacheHierarchy to CloudHierarchyCache
- AUTHN-974: Authorize with separate provider/pool resource types
- CIAM-2219: MINOR: Add vault secrets for Docker rate limit raise
- Improve Build Stability
- chore(deps): bump cc-base from v18.6.0-jdk-17 to v18.9.0-jdk-17
- AUTHN-541 trust-service: Add some logging while initializing AuditLogger object
- AUTHN-541 trust-service: Do not auditlog if not configured via configs
- AUTHN-541 trust-service: Use X-B3-Traceid header as requestId
- CIAM-1176 - Remove debugging print statement
- Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
- Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
- Update the oauth validation API specs
- CIAM-1176: Increase Role Bindings Quota
- CIAM-2049 Publish LC RoleBinding with K8 ID for New LC Creation
- [AUTHN-968] Add fixed seed to hashcodes
- [AUTHN-1041] Add identity pool filter limit to update calls
- AUTHN-1074: Update tests to exercise hierarchy for providers/pools
- Revert “DP-8085 - Migrate to Sempahore self-hosted agent (#1187)”
- DP-8085 - Migrate to Semaphore self-hosted agent
- AUTHN-974: Tests to exercise trust service provider/pool authorize
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- chore(deps): bump docker/prod/confluentinc/cc-base from v16.4.0-jdk-16 to v18.6.0-jdk-17
- [JIRA-ID: SEC-3597] - Integration Tests For Configurable Case Sensitivity for Authorisation in CP LDAP
- AUTHN-541 trust-service: Accept X-Request-Id as header param for audit logging
- AUTHN-541 trust-service: Address review comments
- CIAM-2211 Configure the PKC and LKC ID for SDS in config file
- Add utc time zone to metadata timestamps
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- AUTHN-882: Replace principal with pool id
- [AUTHN-954] Modify request field names
- fixes 6.0.x build from 5.5.x merge
- [AUTHN-820] Create AWS IAM Role for Trust service
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- DGS-4151: Fixed mds test for Org/EnvOperator Describe access on Subje
- [AUTHN-948] Rename spec field for identity providers
- CIAM-2197 Fix the type of SR AND KSQL LD Flag
- CIAM-2183 - MDS code fails validation for a topic name >= 80 characters
- CIAM-2183 - MDS code fails validation for a topic name >= 80
- AUTHN-989: Fix multithreading race condition where we only retry refreshing once
- [AUTHN-997][AUTHN-948][AUTHN-946] Update Trust-Service Metadata
- CIAM-1964 Publish RBAC Crud Changes for KSQL/SR with Fixed PKC ID
- [AUTHN-949] Ensure that identity pool operations are using the right provider
- [AUTHN-997] Rename identity pool’s spec to status
- CIAM-2173 Route Kafka and SDS role bindings use RoleDef Files
- CIAM-1518 Generalize Cloud Scope to take SR & KSQL
- AUTHN-974: Add providerId to pool events
- CIAM-2184 Extractor Publishing empty PKC ID if L*C not found in CHC
- AUTHN-541 trust-service: Add a unit test to verify AuditLogUtils
- AUTHN-541 trust-service: Ignore auditlog for IdentityPrincipal & HealthCheck
- AUTHN-541 trust-service: Audit log CRUD APIs for IdentityPool resource
- AUTHN-541: trust-service Audit log CRUD APIs for IdentityProvider resource
- AUTHN-541 trust-service: Update AsyncReponses to accept audit objects
- AUTHN-541 trust-service: Add a util to help build the AuditLogger object
- AUTHN-541 trust-service: Register and bind AuditLogger object for tests
- AUTHN-541 trust-service: Register and bind AuditLogger object
- AUTHN-541 trust-service: Add auditlog config
- AUTHN-541 rbac-extractor: Update events-schema lib from v0.73.0 to v0.82.0
- AUTHN-541 trust-service: Add events-schema & auditlog-emitter-java dependency
- [AUTHN-954] rename data fields for identity pools
- CIAM-2130 - Deactivate environment scoped role-bindings for deactivated accounts in Prod
- INIT-599 - Allow OrgAdmins to delete the whole Organization
- CDMUM-2091 Add DECISION_ENGINE as accepted audience
- Promoting cc-rbac to version 0.400.0
- CIAM-2171 CHC API for LC -> PKC to use describeLogicalCluster
- [PROD][AUTHN-822] Add db user with rds_iam role
- [AUTHN-821] enabling RDS IAM Auth on rbac database
- [AUTHN-954] publish duplicate fields for IdentityPoolValue
- CIAM-2147 Integration With CHC failing with incompatible vertex version
- AUTHN-918[PROD]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- [CIAM-2077]: Add API to Integrate with CHC Batch Processing
- Revert “CIAM-2147 Integration With CHC failing with incompatible vertex version (#1174)”
- CIAM-2147 Integration With CHC failing with incompatible vertex version
- Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1173)”
- CIAM-2147 Update CHC version to fix incompatible vertex version
- Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1172)”
- CIAM-2147 Update CHC version to fix incompatible vertes version
- [AUTHN-973] Modify rbac operation request based on status update #1171
- RCCA-7223: Scale down cc-rbac cpd replicaCount from 2 to 1
- AUTHN-918[DEVEL]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- CIAM-2147 Update CHC version to fix incompatible vertes version
- Promoting cc-rbac to version 0.385.0
- AUTHN-968: Add logging around 500 error in createIdentityPool
- AUTHN-918[STAG]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- [CIAM-1973] Add API to fetch K8 ID from CHC
- Promoted Extractor to v0.381.0
- [CIAM-2131]: Deactivate environment scoped role-bindings for deactivated accounts in Devel
- [CIAM-2127]: Deactivate environment scoped role-bindings for deactivated accounts in Stag
- CIAM-2146: Promoted Extractor to v0.374.0
- [AUTHN-960] Fix create identity providers bug
- revert import optimizations
- re-add fallback for missing gateway host
- remove unnecessary property
- revert endpoint prop and set gateway host always
- add rds endpoint property
- fix bracket
- fix style
- adding aws dependencies
- add semicolons
- add back poassword, cleanup
- [AUTHN-824] instrument service to connect to DB with RDS IAM Auth
- [CIAM-2138]: Updated ConfigMap Value
- AUTHN-740: Add better logs for retrying on refresh failure and fix default refresh interval when cache control is not present
- Add service.yaml file for rbac extractor
- Added getParameterType to error message. Removed getParameterName
- CIAM-2078 Add Exception Handling in CHC API
- RCCA-6743: Test that we can allow rest client to disable hostname verification
- Promoted Extractor version
- trust-service: Add a test to validate CRN format
- AUTHN-794 trust-service: Fix CRN format
- CIAM-2096: Decreased total hc topics from 60 to 20
- [CIAM-2138]: Update CHC API to accept host and port instead of API
- AUTHN-943: Update quota limit error message for idp and pool
- CIAM-2044: Fixing db transaction deadlock caused by delete.
- RCCA-6760 Delete invalid rolebindings
- Rebase with upstream and fix test
- address review comments
- AUTHN-918: update rbac extractor to handle the new sub claim prefix
- AUTHN-918: enforce subject claim to start with ‘claim.’ prefix in IdentityPool create and update
- [AUTHN-894][AUTHN-749][AUTHN-816][AUTHN-817] Update creation/deletion logic
- RCCA-7563: Delete crufty Rolebindings for large customer
- CIAM-347: Prevent OrgAdmin from self-deleting OrgAdmin role for V2 API
- Added getParameterType to error message. Removed getParameterName
- CIAM-2096: Decreased total RBAC HC topics to 20 so decreased total RBAC HC role
- Added getParameterType and getParameterName to error message
- Adding two test cases for pool filters
- Added unit tests for different ParamException types.
- trust-service: Add a test to validate CRN format
- AUTHN-794 trust-service: Fix CRN format
- Catch ParamException 400 errors. Renamed file.
- Catch ParamException 400 errors.
- CIAM-2021 MDS changes to support Tree Binding Scopes
- Remove duplicate dependency
- Remove cpd docker pull secrets
- CIAM-2094: Decrease total HC topics to 20.
- Address review comments
- Implement LDClient for provider and pool limits in trust service
- Skip regular builds for changes in .deployed-versions
- [STAG][AUTHN-822] add db user with rds_iam role]
- CIAM-2093, RCCA-7359: Set cc-rbac prod version to 110, rbac-extractor prod version to 295 (0.350.0)
- CIAM-2093: Update the num of Extractor pods to 0 in cpd
- AUTHN-813: Replace trust policy antlr parser with cel parser
- Address review comments
- Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)””
- CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
- Integrate JWKS limit with LD
- CIAM-2071: Delete invalid role bindings from v2 api (for internal orgs) [prod]
- [AUTHN-822] add db user with rds_iam role
- Promoted rba-extractor to version 0.340.0 on PROD, DEVEL and CPD
- CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
- CIAM-2075 Disabling failure check of PKC Header failure for Integration Testing
- Implement jwks limit
- CIAM-1970: Implement displayName resolution for V2 API
- Set cc-rbac prod version to 93 (0.333.0)
- Address review comment
- address review comment - check filter limit before pool limit
- Added tests
- CIAM-1863: Fix deployed versions name
- CIAM-1929 Added validation on role binding scope.
- Enforce resource limit for IDP, pool and filter in trust service
- CIAM-1828 Publish PKC Header For RBAC Role binding Changes
- Fix Fuzz Test by specifying most specific scope
- [CIAM-2040] Add Log Statement to find out if different ResourceTypes for a single role binding are used by customer
- CIAM-1863: Update icc-rbac and extractor pipelines to stag->prod->devel->cpd
- CIAM-1880 - Runbookize our DBMigrate approach to deleting rolebindings
- CIAM-2030: Updated pattern_type to LITERAL on stag and prod
- CIAM-2025: Fixed error on access RBAC role_bindings sequences.
- CIAM-1523: Start routing traffic to icc-rbac
- CIAM-2028 Remove Dataplane LD Flags from extractor
- [CIAM-1840] KSQL/SR Extractor Record Header
- CIAM-2025: Grant usage on sequence rbac.role_binding_last_change_id_seq to cc_rbac_extractor_0 and cc_rbac_extractor_1
- CIAM-2030 - Updated pattern_type to LITERAL for org 0 role bindings
- CIAM-2024: Added logic cluster type healthcheck .
- CIAM-2027 Update CHC Client Version to support JDK 8
- CIAM-1823 Interface with Cloud Hierarchy Client
- CIAM-1786 Add ResourcePattern to correctly resolve display name
- AUTHN-835, AUTHN-845: Trim whitespaces on user entry
- [CIAM-1839] Enable identification of KSQL/Schema Role bindings
- CIAM-1902: Inserted 60 role bindings into rbac db for prod
- CIAM-1897: Remove client_address field from icc-rbac audit logs
- Remove extra semicolon
- CIAM-1919: Address issues from icc-rbac ops review
- RCCA-6909 - Customer needs role bindings deleted for deleted lkcs again
- CIAM-1939 producer close should timeout
- Fix dependencies
- [AUTHN-747] remove policy version/id
- CIAM-1902: Added role bindings to SA for stag.
- Fix JWTDebugLoggingTest failure by excluding log4j jars
- CIAM-1954 MINOR: Healthcheck to check RUNNING immediately
- CIAM-1951: Updated deployment notification.
- CIAM-1432: Make icc-rbac audit logs match kafka mds
- CIAM-1432: Add icc-rbac devel auditlog config to stag and prod
- AUTHN-737, AUTHN-752, AUTHN-720: Fix self links, add rbac.addr
- MINOR Add log for manual republish
- CIAM-1752: Enable icc-rbac db metrics
- [AUTHN-750] [RCCA-6673] Grant permissions for cts schemas to rbac-extractor user
- Update CODEOWNERS
- AUTHN-582: Add rbac checks for trust service provider/pool CRUD apis
- [AUTHN-567] Add validation checks to getJWKS()
- CIAM-1912 AuthN to be Codeowners of trust-service
- CIAM-1776 Update cc-base to v16.4.0
- CIAM-1432: Fix default cloudevent.codec
- Create/run migrations for trust-service in stag/prod
- CIAM-1544: Remove ce-kafka-version suffix from mds image version
- RCCA-6555 Remove role bindings associated with deleted clusters
- CIAM-1523: Add audiences config to icc-rbac
- [AUTHN-711] Modify prefix for identity pools
- Upgrade cc-base image
- CIAM-1432: Enable Audit Logging for icc-rbac
- AUTHN-619: Add getIdentityPrincipal call
Replicator¶
- RCCA-7678: Reverse proxy header check added
- KGLOBAL-2126 seek to begining only for non empty partitions list
- Add log redactor.
- Migrate confluent-log4j to reload4j.
- fix upstream build