Confluent Platform Component Changelogs¶
This topic provides changelogs for the individual Confluent Platform components.
Version 7.3.5¶
Released September 2023
Community Features¶
Common¶
Kafka¶
- 8544dba7 - KAFKA-15375: fix broken clean shutdown detection logic in LogManager
- PR-14278 - KAFKA-15393: Improve shutdown behavior in MM2 integration tests (#14278)
- PR-932 - KSECURITY-1859: updating netty to 4.1.96 (3.3)
- PR-14082 - KAFKA-15102: Add replication.policy.internal.topic.separator.enabled property to MirrorMaker 2 (KIP-949) (#14082)
- PR-14156 - KAFKA-15202: Fix MM2 offset translation when syncs are variably spaced (#14156)
- PR-14162 - KAFKA-15312; Force channel before atomic file move (#14162)
- PR-14044 - KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers (#14044)
- PR-13948 - KAFKA-15091: Fix misleading Javadoc for SourceTask::commit (#13948)
- PR-14221 - KAFKA-15338: The metric group documentation for metrics added in KAFKA-13945 is incorrect (#14221)
ksqlDB¶
Schema Registry¶
- PR-2734 - DGS-8112 Allow properties to be omitted during serialization
- 3f0e1198 - DGS-7927: update io.grpc_grpc-protobuf client/src/main/java/io/confluent/kafka/schemaregistry/client/security/bearerauth/BearerAuthCredentialProvider.java
- PR-2716 - Upgrade wire to 4.8.0
- PR-2695 - do not reinstantiate objectmapper as it’s expensive (#2686)
Version 7.3.4¶
Released June 2023
Community Features¶
Common¶
- PR-530 - Backport log redactor to CP patches.
- bd393b16 - AUDIT-1600: Updating logredactor version to 1.0.12
- 22a9da57 - APPSEC-2551: Update Prometheus JMX Exporter
- 5ad03d12 - APPSEC-2551: Update Jackson
- PR-514 - Update snakeyaml
- b4334b94 - pin jose4j version to match ce-kafka
- 95ed5124 - include snappy and jetty in the dependency management to match versions with ce-kafka, cleanup jackson imports
Kafka¶
- PR-13690 - KAFKA-14978 ExactlyOnceWorkerSourceTask should remove parent metrics (#13690)
- 811fcac9 - KAFKA-14974: Restore backward compatibility in KafkaBasedLog - fix unit tests
- PR-13688 - KAFKA-14974: Restore backward compatibility in KafkaBasedLog (#13688)
- PR-13592 - KAFKA-14862: Outer stream-stream join does not output all results with multiple input partitions (#13592)
- PR-13429 - KAFKA-14666: Add MM2 in-memory offset translation index for offsets behind replication (#13429)
- PR-13446 - KAFKA-14837/14842:Avoid the rebalance caused by the addition and deletion of irrelevant groups for MirrorCheckPointConnector (#13446)
- PR-13603 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13526) (#13603)
- PR-13599 - KAFKA-14869: Ignore unknown record types for coordinators (KIP-915, Part-1) (#13599)
- bffab974 - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
- PR-13499 - KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable (#13499)
- PR-13470 - KAFKA-14864: Close iterator in KStream windowed aggregation emit on window close (#13470)
- PR-13445 - KAFKA-14843: Include Connect framework properties when retrieving connector config definitions (#13445)
- PR-13148 - KAFKA-14645: Use plugin classloader when retrieving connector plugin config definitions (#13148)
- PR-13831 - KAFKA-15053: Use case insensitive validator for security.protocol config (#13831)
- PR-914 - KSECURITY-1627 KSECURITY-1556 KSECURITY-1568: update snappy, jetty, jose
- PR-13865 - KAFKA-15096: Update snappy-java to 1.1.10.1 (#13865)
ksqlDB¶
- PR-9810 - Bump minimist and mkdirp
- 40d8003d - fix: Replace regex in CommandParser with a split by space
- 4ce2d5f3 - fix: Replace regex in CommandParser with a split by space
- cc0501d8 - fix: Simplify regex in DdlDmlRequestValidators to avoid catastrophic backtracking
- aefe6338 - Fix log4j-extensions
- 08853cd3 - Fix log4j-extensions
REST Proxy¶
Schema Registry¶
- PR-2635 - DGS-7289 Adding CustomBearerAuthCredentialProvider
- PR-2654 - DGS-7432 Fix ClassCastException when getting params option
- PR-2647 - DGS-7422 Optimization: omit compat check during IMPORT
- PR-2646 - DGS-7412 Fix for registering references in non-default context
- PR-2642 - DGS-7367 Propagate error messages even if not JSON
- PR-2625 - Static token provider: adding optional lsrc-id and pool-id
- PR-2616 - DGS-7005 Convert socket timeouts to RetriableException for converters
- PR-2602 - DGS-6929 Ignore invalid Avro defaults for source connectors
- PR-2599 - DGS-6516 Add normalize query param to compatibility APIs
- PR-2590 - DGS-6701 Fix getSchemaById to match subject if passed
- PR-2589 - DGS-6613 Ignore invalid Avro defaults in Avro Converter
- PR-2588 - DGS-6603 Fix default conversions in JsonSchemaConverter
- PR-2585 - Revert behavior of DGS-6522
- PR-2584 - DGS-6522 Disallow empty subject names during schema registration
- PR-2693 - DGS-7746 Fix perf regression due to DGS-6331
- PR-2692 - DGS-7723 Fix for qualifySubjectWithParent method
- PR-2684 - DGS-7675 Fix NPE when print.schema.ids=true
- 1bf961b7 - Update json-schema
Commercial Features¶
Confluent Server¶
- [OBSTEL-303] Update Active Filters if Filters List is Empty for Telemetry Remote Configuration
- KSECURITY-1568: Upgrading jose4j
- Restore backward compatibility in KafkaBasedLog (#9725)
- KDATA-737: Fix incorrectly exposed ccloud only configs on documentation
- KSECURITY-1556: updated jetty
- KAFKA-14887: FinalizedFeatureChangeListener should not shut down when
- KDATA-853: Upgrading dependencies
- KSECURITY-1460, KSECURITY-1466: Updated snakeyaml, json-smart and nim
- KSECURITY-1459: updated golang.org/x/net mod.
- KSECURITY-1455: updated golang.org/x/text mod.
- KSECURITY-1453: Updated version of golang.org/x/sys
- KSTORAGE-2447: Disable ULE on tier topic, if tier topic already exists
- KGLOBAL-3350 Add more logging for persistent connection creation in Outbound connection manage
- KSECURITY-1627: Updated snappy-java
- KSECURITY-1618: update guava version to 32.0.1-jre.
- CONFLUENT: Add exception details to SecurePassConfigProvider logs
- KAFKA-14974: Restore backward compatibility in KafkaBasedLog - fix unit tests
- KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
Security¶
- CIAM-2998 - Log redactor backport for confluent-security-plugins
- KREST-10128 Create LicenseManager instance in iteration run instead o
Secret Registry¶
- unpin netty, as rest-utils brings updated version
Control Center¶
- [MMA-13078] revisit session caching in basic auth
- [MMA-13089] Handle large message JSON on topic inspection
- [MMA-13013] Add property to specify alias cert per component for SchemaRegistry, KSql & Connect
- [MMA-13042] Fixing log statement for websocket time out
- [MMA-13216] update jose4j
- [MMA-13201] Csrf Enabled Flag not working
MQTT Proxy¶
- CC-19108 | Rename isChannelClosed to isAuthenticated.
- CC-19108 | Use an atomic reference to avoid publishing message when the channel is closed or marked for closure.
Metadata Service¶
- CIAM-2936: update jose4j
- update
- update
- update
- update
- Update pom.xml
- CIAM-2810: Update http response status code if writer is null
- MMA-13145: Update launchdarkly
- CIAM-2986 - Older MDS builds are not running all the tests they should be
- update comment, clean up whitespaces
- another attempt with mina-core
- pin mina core in cli, scope mina-core as test in main pom
- pin mina-core in testingLdap only
- SEC-xyz: Update mina-core
Replicator¶
- disable builds of connect-replicator version 5.5.x
- used spotbugs version variable instead
- use spotbugs annotations instead of findbugs
- resolving compile issue
- KGLOBAL-3224 removed find bugs usage
- remove aws-maven plugin to unblock the downstream validation for common: RCCA-10412, APPSEC-2551
Version 7.3.3¶
Released March 2023
Community Features¶
Common¶
- PR-512 - Add snakeyaml to dependency management
- 7bcaab9e - Fix typo in with method calls
- 3868e27b - Fix typo
- 783c5721 - Return the original “version_range” if it isn’t an actual Maven version range
- e71293e4 - Update logredactor depdendency to 1.0.11
- PR-502 - Fix: override maven version plugin’s default versionrange
- 9350d231 - Update resolver-maven-plugin to 0.6.0
- PR-497 - Upgrade Netty to 4.1.86.Final
- PR-496 - Bump to 7.4.4
Kafka¶
- PR-13367 - KAFKA-14797: Emit offset sync when offset translation lag would exceed max.offset.lag (#13367)
- PR-13386 - KAFKA-14809 Fix logging conditional on WorkerSourceTask (#13386)
- PR-13379 - KAFKA-14799: Ignore source task requests to abort empty transactions (#13379)
- PR-12320 - KAFKA-13702: Connect RestClient overrides response status code on request failure (#12320)
- PR-13193 - KAFKA-14659 source-record-write-[rate|total] metrics should exclude filtered records (#13193)
- PR-13052 - KAFKA-14545: Make MirrorCheckpointTask.checkpoint handle null OffsetAndMetadata gracefully (#13052)
- PR-13181 - KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method (#13181)
- PR-11818 - KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (#11818)
- PR-13273 - KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
- PR-13262 - KAFKA-14727: Enable periodic offset commits for EOS source tasks (#13262)
- PR-13168 - Kafka 14565: On failure, close AutoCloseable objects instantiated and configured by AbstractConfig (#13168)
- PR-13208 - KAFKA-5756: Wait for concurrent source task offset flush to complete before starting next flush (#13208)
- PR-13230 - KAFKA-14704; Follower should truncate before incrementing high watermark (#13230)
- PR-13211 - KAFKA-14676: Include all SASL configs in login cache key to ensure clients in a JVM can use different OAuth configs (#13211)
- PR-12984 - KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic (#12984)
- PR-871 - KC-2332: Upgrade netty to 4.1.86
- PR-12535 - KAFKA-13769 Fix version check in SubscriptionStoreReceiveProcessorSupplier (#12535)
- PR-12437 - KAFKA-13769: Add tests for ForeignJoinSubscriptionProcessorSupplier (#12437)
- PR-13119 - KAFKA-14623: OAuth’s HttpAccessTokenRetriever potentially leaks secrets in logging (#13119)
- PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
- PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
- PR-13106 - KAFKA-13709 (follow-up): Avoid mention of ‘exactly-once delivery’ or ‘delivery guarantees’ in Connect (#13106)
- PR-13058 - KAFKA-14557; Lock metadata log dir (#13058)
- PR-13073 - KAFKA-14571: Include rack info in ZkMetadataCache.getClusterMetadata (#13073)
- b66af662 - Bump version to 3.3.2
- PR-12994 - KAFKA-14457; Controller metrics should only expose committed data (#12994)
- PR-13023 - KAFKA-14532: Correctly handle failed fetch when partitions unassigned (#13023)
- PR-12968 - KAFKA-14417: Address incompatible error code returned by broker from InitProducerId (#12968)
- PR-13000 - KAFKA-14496: Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler (#13000)
- PR-12856 - KAFKA-14392: Fix overly long request timeouts in BrokerToControllerChannelManager (#12856)
- PR-12956 - KAFKA-14379: Consumer should refresh preferred read replica on update metadata (#12956)
- 4154a1ca - KAFKA-14435: Fix allow.everyone.if.no.acl.found config behavior for StandardAuthorizer
- PR-12915 - KAFKA-14417: Producer doesn’t handle REQUEST_TIMED_OUT for InitProducerIdRequest, treats as fatal error (#12915)
- PR-12935 - KAFKA-14432: RocksDBStore relies on finalizers to not leak memory (#12935)
- PR-12885 - KAFKA-14358; Disallow creation of cluster metadata topic (#12885)
- PR-12898 - KAFKA-14430: Specify JMX RMI port system property when not already set (#12898)
- PR-12920 - KAFKA-14339 : Do not perform producerCommit on serializationError when trying offsetWriter flush (#12920)
- PR-12909 - KAFKA-14422; Consumer rebalance stuck after new static member joins a group with members not supporting static members (#12909)
- 16938e5b - KAFKA-14009: Rebalance timeout should be updated when static member rejoins
- PR-12877 - KAFKA-14372: Choose replicas only from ISR for preferred read replica (#12877)
- PR-12651 - KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error message (#12651)
- PR-12859 - KAFKA-14325: Fix NPE on Processor Parameters toString (#12859)
- PR-12869 - KAFKA-14382: wait for current rebalance to complete before triggering followup (#12869)
- PR-12840 - KAFKA-14320: Updated Jackson to version 2.13.4 (#12840)
- PR-12295 - KAFKA-13586: Prevent exception thrown during connector update from crashing distributed herder (#12295)
- PR-12872 - KAFKA-14303 Producer.send without record key and batch.size=0 goes into infinite loop (#12752) (#12872)
- PR-12861 - KAFKA-14388 - Fixes the NPE when using the new Processor API with the DSL (#12861)
- PR-12783 - KAFKA-14334: Complete delayed purgatory after replication (#12783)
ksqlDB¶
REST Proxy¶
- 60edebfe - changed ‘all configs’ to ‘dynamic configs’
- PR-1083 - Fix build breakage caused by KAFKA-14334
- 69ed0cd9 - remove incorrect synchronization
- PR-1108 - Set kafka-schema-registry version in dependency mamanagement
- 7a7f5dcd - Remove unnecessary ${project.version} tag
- PR-1102 - Fix transitive dependency of kafka-schema-registry
- PR-1101 - Fix dependency org.yaml_snakeyaml, upgrade to fixed
- PR-1096 - Make sure SchemaRegistryRestApplication check for leader election finish
- PR-365 - Adapt Test to handle Exception class changing its parent
Schema Registry¶
- PR-2582 - Reset default for leader election delay to false
- PR-2577 - DGS-6373 Ignore leading dot when merging Protobuf custom options
- PR-2575 - DGS-6331 Handle javaType for oneOfs during JSON deserialization
- PR-2570 - [DGS-6267] - Changing logs to debug in SR
- PR-2568 - DGS-6306 Allow multiple oneofs in Protobuf converter
- PR-2560 - Add config whether to delay leader election
- PR-2550 - DGS-6192 Include default ctx when using subjectPrefix w/wildcard ctx
- PR-2546 - Adding createBrokerConfig changes
- PR-2533 - DGS-5624 SR Oauth client config : Make Identity pool and logical cluster id as optional
- PR-2540 - DGS-6075 : Add explicit definition of snakeyaml.
- PR-2539 - DGS-6071 Handle nested messages in Protobuf custom options
- PR-2530 - DGS-6032: Externalize Kafka group configurations
- PR-2523 - DGS-6023 Add Protobuf converter config to not generate index for unions
- PR-2522 - DGS-6022 Add Protobuf converter config to generate struct for nulls
- cf3c6ac9 - Fix merge
- PR-2517 - MINOR cherry-pick Protobuf validate optimization to 7.0.x
- PR-2519 - DGS-6014 cherry pick Maven plugin enhancement to 5.5.x
- PR-2518 - DGS-6014 Make url decoding in Maven plugin more lenient
- PR-2516 - DGS-5908 Qualify extension fields from Protobuf descriptor
- PR-2514 - DGS-5950: Allow optional map key in Avro
- PR-2509 - DGS-5909 When normalizing Avro, ensure defaults are valid
- aaf29183 - Fix merge issue
- PR-2505 - DGS-5908 Qualify extension fields when normalizing Protobuf
- PR-2501 - DGS-5897 Support return Avro schemas with all refs resolved
- PR-2469 - DGS-5796: Initiate leader election after resources have been set up
- PR-2474 - Fix handling of Protobuf repeated options
- PR-2471 - Preserve metadata associated with primitive types during normalization
- PR-2468 - DGS-5567 Handle Protobuf map options in toCanonicalString()
Commercial Features¶
Confluent Server¶
- Backport “Minor: add logging to inLock / inWriteLock to catch leaked locks (#8594)” to 7.3.x
- KMETA-478 Enable zk audit logs by default
- Fix version comparison in kafkatests (#8922)
- KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
- KGLOBAL-2456: Clear partitionsWithNewHighWatermark in ClusterLinkFetcherThread
- CONFLUENT: CL fetcher should complete fetches when new messages are replicated
- cherry pick KAFKA-14334: Complete delayed purgatory after replication (#12783)
- KGLOBAL-2035: Unblock local follower fetch requests when records are appended to mirror leader logs
- KAFKA-14358; Disallow creation of cluster metadata topic (#12885)
- KSECURITY-1005: [7.3.x] Enforce kafka-client-plugins consumers to use the same snakeyaml version
- cherrypick KAFKA-14417: Address incompatible error code returned by broker from`InitProducerId`
- KSECURITY-897: Make expiration timeout for ZK ACL change notification path configurable
- KSECURITY-1001: Close old rest client when creating new one (#8785)
- KENGINE-287: RPCProducerIdManager should not wait on new block
- KC-2328: Update yaml.v2 to 2.2.4
- KSECURITY-965: Update git version 1.13.0
- KSECURITY-953: Update azure-identity to 1.7.3
- KSECURITY-981: Add FeatureZNode zk node path to secure root paths
- KC-2333: Update io.netty libraries to 4.1.86.Final
- CPKAFKA-3855 Don’t allow produce logging to the audit log topic
- Revert “RCCA-9325: Add truncated checksum for LDAP passwords in trace logs”
- RCCA-9325: Add truncated checksum for LDAP passwords in trace logs
- KSTORAGE-82: Tiered storage topic deletion support
- MINOR: disable use of ConfluentLeaderAndIsrRequest in 5.4.x
- [KENGINE-314]: Fix a NPE when FetchSession close.
- Bump version to 3.3.2
- KAFKALESS-817: Set RF = -1 for internal topics if placement constraint is set
- KAFKA-9038: [WIP] Allow creating partitions for topics partitions not in reassignment
- CONFLUENT: Use single audit log provider for MDS and its hosting broker
- KAFKA-14009: Rebalance timeout should be updated when static member rejoins
- MINOR: Fix jackson version inconsistency; update netty, boringssl, avro
Security¶
- Removed From Supported Operations to follow RBAC Rules
- Added fix for unit test
- Fixed import
- Changed the design of RBAC for Exporters
- Removed extra line changes
- Added RBAC for Schema linking Phase 1
- revert createBrokerConfig for 7.3.x
- Remove logging statement
- Removed bcfips from common
- Pin bouncycastle fips dependency in confluent-security-plugins
- Exclude bcpkix-jdk15on as compile dep and add as test dependency
- SEC-5350: Update netty-codec-http to 4.1.86.Final
- adding null check in AuthorizationFilter operation method
- Changed operation to compatibility read.
- Update AuthorizationFilter.java
- Moved comment and updated variable names
- Made suggested change
- Added RBAC for /schemas/ids/1/subjects and versions endpoints
Secret Registry¶
- Revert “Revert “Fixing createBrokerConfig””
- Exclude Non-FIPS dep and use bc-fips dependency
- Remove unused variables
- Update RestService.java
Control Center¶
- RCCA-9557 - fix broker flapping issue
- Use getTokenLifetimeMs from Util
- Update docker image tag for integration-test
- Use docker image tag 7.1.6
- Increase wait start time for control center in test
- Make sure control center stop properly
- Change PORT to avoid conflict with TelemetryReporterIntegrationTest
- KC-2261 timeout websocket connection via jwt token
- Update CODEOWNERS to match latest definition from master
- MMA-13013: Override SSL Stores priority
- Refactoring- made code more functional and improved readability.
- Updated NotFoundException message in scenario where topicName provided returns 0 no consumergroupdata.
- Updated code based on PR feedback.
- Updated consumerGroupId API to also support topicName queryParam.
- Improved test cases with scenarios of single consumergroup covering multiple topics. Updated consumerGroupId to not support topicName queryParam,as it is not required.
- Adding unit test for CachedConsumerOffsetsResource
- [MMA-12987][MMA-12439] Fix ControlCenterSecureIntegrationTest Test
- Adding support for topicName in query param to return only consumer groups for that topic
- [MMA-12439] Fix ControlCenterSecureIntegrationTest Test
- Pin bc-fips version correctly
- [MMA-12439] Update Websocket configuration for Control-Center if BasePath supplied
- Reading password using getPassword instead of getString, this fixes the casting exception
- Adding a config to force armeria health check to use HTTP1
- Fixing failing test case, due to merge issue
- Fixing check style error, removing unused import
- Renaming Health_check to health check
- MMA-12911: Propagate numPartitions and replication factor from CreateTopicsResult
- MMA-12912: Disable offset commits, don’t supply group ID
- adding config based functionality to force http1 for armeria health checks
- MINOR: Update repo to use mvn-wrapper to speed up builds
- [MMA-12804] Re-enable failing tests in SslUtilsTest
- Fix codeowners to make c3 default ownens as well
- Cherry pick 7746 codewoner
MQTT Proxy¶
- remove the pin of netty version to use the (current) version defined in common
Metadata Service¶
- Clean testng
- [Fix build] Drop down to testNG 7.5 - which is the last version to support jdk8
- Adapt 7.1.x to testNg 7
- Clean up surefire
- CIAM-2615 - Bump TestNG to 7.7.0
- Fix notnull in 7.1.x
- Fix NotNull import
- Squashed ‘mk-include/’ content from commit 7df56b0fff
- Revert “Merge remote-tracking branch ‘origin/7.1.x’ into 7.2.x”
- Squashed ‘mk-include/’ content from commit 7df56b0fff
- CIAM-2579: update pgsql
- CIAM-2577: Turned off test RbacExtractorMetricsTest
- Fixed build error relate to FIPS jar for 7.0.x branch
- Increase buffer size for Proxy Servlet
Replicator¶
- Fixed deprecated method usage
- Changed to 5 sec to honor TASK_SHUTDOWN_GRACEFUL_TIMEOUT_MS_CONFIG
- Changed default to 120 sec to match topic config sync
- KGLOBAL-3060 added consumer poll timeout ms property
Version 7.3.1¶
Released December 2022
Community Features¶
Common¶
Kafka¶
- PR-845 - CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
- PR-842 - DP-9030: Use the new withGradleFile closure
- PR-12809 - [KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)
- PR-829 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
- PR-12836 - KAFKA-14282: stop tracking Produced sensors by processor node id (#12836)
- PR-816 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
- PR-12794 - Revert “KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)” (#12794)
- PR-12790 - KAFKA-14337; Correctly remove topicsWithCollisionChars after topic deletion (#12790)
- PR-12765 - KAFKA-14316; Fix feature control iterator metadata version handling (#12765)
- PR-12747 - KAFKA-14300; Generate snapshot after repeated controller resign (#12747)
- PR-12676 - KAFKA-14209 : Integration tests 3/3 (#12676)
- PR-12741 - KAFKA-14296; Partition leaders are not demoted during kraft controlled shutdown (#12741)
- PR-12736 - KAFKA-14292; Fix KRaft controlled shutdown delay (#12736)
- PR-12709 - KAFKA-14275; KRaft Controllers should crash after failing to apply any metadata record (#12709)
- PR-12634 - KAFKA-14225; Fix deadlock caused by lazy val exemptSensor (#12634)
- e23c59d0 - Bump version to 3.3.1
- 1780f266 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- 4b35f247 - Bump 3.3 branch to 3.3.1-SNAPSHOT
ksqlDB¶
REST Proxy¶
Schema Registry¶
- PR-2467 - changed the header name of pool id
- PR-2460 - DGS-5544 Support nulls when using optional fields in proto2
- PR-2459 - DGS-5400 Support subjectPrefix containing wildcard context and subject
- PR-2456 - Derive schema maven plugin 7.3.x
- 87defb47 - Exposed the method to schema registry client to use in security-plugins
- PR-2450 - Pin Snakeyaml version
- PR-2443 - DGS-5423 Check subject when looking up ID across contexts
- PR-2441 - * Minor: change OAuth config string
- PR-2440 - Update the Snakeyaml dependency
- PR-2409 - DGS-3721 SaslOAuthCredentialProvider
- PR-2371 - DGS-3396: Add deletedOnly=true to SR list APIs
- 8e5ce168 - Unpin Protobuf version.
- PR-2421 - DGS-5220 Handle nested extend decls in Protobuf
- PR-2415 - DGS-5254 Make timeouts configurable when forwarding requests
- PR-2414 - DGS-5253 Handle cycles when resolving references
- PR-2344 - DGS-3719, DGS-3720 Added OAuthCredentialProvider and CachedOauthTokenRetriever
- PR-2407 - MINOR: Fix OpenAPI deleteSubjectConfig example
- PR-2405 - DGS-4754 Handle empty record default at field level
Commercial Features¶
Confluent Server¶
- remove extraneous }
- CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
- DP-9030 - Switch to codeartifact repo
- merge from ccs:3.3 to ce:7.3.x
- DGS-5248 Adding SR client OAuth config for Schema Validation.
- KAFKA-14212: Enhance HttpAccessTokenRetriever to retrieve error messa
- Increase vagrant aws timeout for system test
- Upgrade from Scala 2.13.8 to 2.13.10
- Upgrade to snakeyaml version 1.32
- Upgrade jackson-databind version to 2.13.4.2
- MINOR: TierArchiver - improve logging for cancelled tasks
- KGLOBAL-2143: Return error message containing all racks that dont have enough brokers during topic creation validation
- KGLOBAL-2122: Validate there are enough brokers to satisfy updated topic placement
- Upgrade to protobuf version 3.19.6
- CONFLUENT: Revert protobuf upgrade change
- CIAM-2424 - Name RestClient threads for stackdump identification (#7887)
- CPKAFKA-9173: Disable colocated KRaft upgrade from CP 7.1/7.2
- Bump version to 3.3.1
- KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- Bump 3.3 branch to 3.3.1-SNAPSHOT
Security¶
- DGS-5542: Incorporated code review comments and removed the unused parameter schemaRegistryResourceOperation
- DGS-5542: SchemaId validation for numeric value in authorizer filter
- fixed indentation
- Update AuthorizationFilter.java
- Changed from 5 operations to 1
- updated tests
- added null when schema id does not exist
- changed the logic of authorizeSchemaIdLookup
- changed to Short
- dummy commit for new build
- Adding OAuth config support for Schema validation
- Exclude bcpkix-jdk15on as compile dep and add as test dependency
- Pin bc-nonfips dep as a test dependency
- Pin bouncycastle fips dependency in confluent-security-plugins
- Removed bcfips from common
- excluded bcfips jars
- Removed From Supported Operations to follow RBAC Rules
- Added fix for unit test
- Changed the design of RBAC for Exporters
- Fixed import
- Removed extra line changes
- Added RBAC for Schema linking Phase 1
- AUTHN-1087: Explicit define netty versions in kafka-rest plugin
- AUTHN-1087: [7.0.x only] Enforce Netty versions in kafka-rest plugin
- DGS-3396: introduce LookupFilter for list apis
- AUTHN-1087: Upgrade vertx to 3.9.14
- AUTHN-1325: Update bc-fips version to 1.0.2.3
- Fix bouncycastle issue
- Exclude bouncycastle non-fips jar from connect packaging
- Revert “Exclude bcpkix-jdk15on as compile dep and add as test dependency”
- Revert “Pin bouncycastle fips dependency in confluent-security-plugins”
- Revert “Removed bcfips from common”
- fixed checkstyle error
- Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas
Secret Registry¶
- Exclude Non-FIPS dep and use bc-fips dependency
Control Center¶
- RCCA-7746: CCloud RBAC user unable to view messages from UI
- MMA-9023: memory bounding for streams applications [5.4.x, master]
- [MMA-12762][MMA-12624] Add testng dependency
- Fix Failing Tests due to CC-18163
- Revert “MMA-12206 upgrade io.netty:netty-codec-http”
MQTT Proxy¶
- Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
- Upgrading netty
Metadata Service¶
- Update Launchdarkly
- APIF-3122: Unpin protobuf version.
- MMA-12624 Fix testng dependencies
- CIAM-2392 Bump PSQL Version
Replicator¶
- KGLOBAL-2436: Don’t seek to beginning on pause for schema translator
- REPL-2055-: Log WakeupException at debug level
Version 7.3.0¶
Released October 2022
Community Features¶
Common¶
- aeca780b - Update Jolokia version in pom.xml
- PR-472 - Upgrade Netty to 4.1.79.Final
- PR-471 - fix checkstyle
- PR-468 - APPSEC-1393: Fix dependency in disk-usage-agent [6.2 and 7.0]
- PR-464 - APPSEC-1393:Migrate from confluent-log4j to reload4j [5.4.x – 7.0.x]
- PR-465 - APPSEC-1393: Confluent log4j to reload4j [7.1.x Only]
- PR-467 - APPSEC-1412: Upgrade JUNIT in 5.4.x only
- PR-442 - Introduce Pull Request Reviewers
Kafka¶
- eefe8671 - KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- PR-12642 - KAFKA-14207; KRaft Operations documentation (#12642)
- PR-12681 - KAFKA-14259: BrokerRegistration#toString throws an exception, terminating metadata replay (#12681)
- 9d1f9f77 - Bump version to 3.3.0
- PR-12628 - KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
- PR-12664 - KAFKA-14243: Temporarily disable unsafe downgrade (#12664)
- PR-12653 - KAFKA-14240; Validate KRaft snapshot state on startup (#12653)
- PR-12658 - KAFKA-14233: Disable testReloadUpdatedFilesWithoutConfigChange first to fix the build (#12658)
- PR-12655 - KAFKA-14238; KRaft metadata log should not delete segment past the latest snapshot (#12655)
- PR-12570 - KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
- PR-12625 - KAFKA-14222; KRaft’s memory pool should always allocate a buffer (#12625)
- PR-12626 - KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits (#12626)
- PR-12603 - KAFKA-14196; Do not continue fetching partitions awaiting auto-commit prior to revocation (#12603)
- PR-12624 - KAFKA-14215; Ensure forwarded requests are applied to broker request quota (#12624)
- b2639c8d - Remove the html end tag from upgrade.html
- PR-12597 - KAFKA-14205; Document how to replace the disk for the KRaft Controller (#12597)
- PR-12596 - KAFKA-14203 Disable snapshot generation on broker after metadata errors (#12596)
- PR-12617 - KAFKA-14216: Remove ZK reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc (#12617)
- PR-12618 - KAFKA-14217: app-reset-tool.html should not show –zookeeper flag that no longer exists (#12618)
- PR-12609 - KAFKA-14198; swagger-jaxrs2 dependency should be compileOnly (#12609)
- PR-12584 - KAFKA-14194: Fix NPE in Cluster.nodeIfOnline (#12584)
- PR-12604 - KAFKA-14188; Getting started for Kafka with KRaft (#12604)
- PR-12599 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (#12599)
- PR-12598 - KAFKA-14201; Consumer should not send group instance ID if committing with empty member ID (server side) (#12598)
- PR-12595 - KAFKA-14204: QuorumController must correctly handle overly large batches (#12595)
- PR-11783 - KAFKA-14143: Exactly-once source connector system tests (#11783)
- PR-12586 - KAFKA-14200: kafka-features.sh must exit with non-zero error code on error (#12586)
- PR-12578 - KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case (#12578)
- PR-12533 - KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Controller (#12533)
- PR-12294 - KAFKA-13990: KRaft controller should return right features in ApiVersionResponse (#12294)
- PR-12571 - KAFKA-14187: kafka-features.sh: add support for –metadata (#12571)
- PR-12565 - KAFKA-14183; Cluster metadata bootstrap file should use header/footer (#12565)
- PR-12513 - KAFKA-14177: Correctly support older kraft versions without FeatureLevelRecord (#12513)
- PR-12103 - KAFKA-13850: Show missing record type in MetadataShell (#12103)
- PR-12551 - KAFKA-14178 Don’t record queue time for deferred events (#12551)
- PR-12469 - KAFKA-13914: Add command line tool kafka-metadata-quorum.sh (#12469)
- PR-12508 - KAFKA-13888; Implement LastFetchTimestamp and in LastCaughtUpTimestamp for DescribeQuorumResponse [KIP-836] (#12508)
- PR-12518 - KAFKA-14167; Completion exceptions should not be translated directly to error codes (#12518)
- PR-12517 - KAFKA-13940; Return NOT_LEADER_OR_FOLLOWER if DescribeQuorum sent to non-leader (#12517)
- PR-12491 - KAFKA-14148: Update ResetOffsetsDoc (#12491)
- PR-12514 - KAFKA-14154; KRaft controller should return NOT_CONTROLLER if request epoch is ahead (#12514)
- PR-12274 - KAFKA-13959: Controller should unfence Broker with busy metadata log (#12274)
- PR-12506 - KAFKA-14154; Return NOT_CONTROLLER from AlterPartition if leader is ahead of controller (#12506)
- PR-12498 - KAFKA-13986; Brokers should include node.id in fetches to metadata quorum (#12498)
- PR-12184 - Fix the rate window size calculation for edge cases (#12184)
- PR-12487 - KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode (#12487)
- f3cf6db3 - KAFKA-14114: Add Metadata Error Related Metrics
- PR-12396 - KAFKA-14051: Create metrics reporters in KRaft remote controllers (#12396)
- PR-12403 - KAFKA-13166 Fix missing ControllerApis error handling (#12403)
- PR-12467 - KAFKA-14129: KRaft must check manual assignments for createTopics are contiguous (#12467)
- PR-12447 - KAFKA-14124: improve quorum controller fault handling (#12447)
- c2422f63 - Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
- PR-12489 - KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing partition epoch (#12489)
- PR-12457 - KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
- PR-12440 - KAFKA-14107: Upgrade Jetty version (#12440)
- PR-12483 - KAFKA-14136 Generate ConfigRecord for brokers even if the value is unchanged (#12483)
- ba219265 -: upgrading netty to v4.1.79.Final #754
- PR-12429 - KAFKA-14089: Only check for committed seqnos after disabling exactly-once support in Connect integration test (#12429)
- PR-12415 - KAFKA-14079 - Ack failed records in WorkerSourceTask when error tolerance is ALL (#12415)
- PR-12374 - KAFKA-14039 Fix AlterConfigPolicy usage in KRaft (#12374)
- PR-12411 - KAFKA-14078; Do leader/epoch validation in Fetch before checking for valid replica (#12411)
- PR-12433 - KAFKA-14093: Use single-worker Connect cluster when testing fenced leader recovery (#12433)
- PR-12347 - KAFKA-13919: expose log recovery metrics (#12347)
- PR-12408 - KAFKA-14076: Fix issues with KafkaStreams.CloseOptions (#12408)
- PR-12365 - KAFKA-14020: Performance regression in Producer (#12365)
- PR-12349 - KAFKA-14024: Consumer keeps Commit offset in onJoinPrepare in Cooperative rebalance (#12349)
- PR-12421 - Revert “KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)” (#12421)
- PR-12420 - KAFKA-13769 Fix version check in SubscriptionJoinForeignProcessorSupplier (#12420)
- PR-12405 - KAFKA-13572 Fix negative preferred replica imbalanced count metric (#12405)
- PR-10964 - KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
- PR-12265 - KAFKA-13968: Fix 3 major bugs of KRaft snapshot generating (#12265)
- PR-12398 - KAFKA-14062: OAuth client token refresh fails with SASL extensions (#12398)
- PR-12390 - KAFKA-14055; Txn markers should not be removed by matching records in the offset map (#12390)
- PR-12381 - KAFKA-13474: Allow reconfiguration of SSL certs for broker to controller connection (#12381)
- PR-12296 - KAFKA-13996: log.cleaner.io.max.bytes.per.second can be changed dynamically (#12296)
- PR-12359 - KAFKA-13983: Fail the creation with “/” in resource name in zk ACL (#12359)
- PR-12091 - KAFKA-12943: update aggregating documentation (#12091)
- PR-12297 - KAFKA-13846: Follow up PR to address review comments (#12297)
- PR-12337 - KAFKA-10199: Remove main consumer from store changelog reader (#12337)
- PR-12360 - KAFKA-14032; Dequeue time for forwarded requests is unset (#12360)
- PR-12379 - KAFKA-10199: Remove call to Task#completeRestoration from state updater (#12379)
- PR-12224 - KAFKA-13943; Make LocalLogManager implementation consistent with the RaftClient contract (#12224)
- PR-11782 - KAFKA-10000: Integration tests (#11782)
- PR-11784 - KAFKA-13228; Ensure ApiVersionRequest is properly handled KRaft co-resident mode (#11784)
- PR-11894 - KAFKA-13613: Remove hard dependency on HmacSHA256 algorithm for Connect (#11894)
- PR-12376 - Upgrade Netty and Jackson versions [KAFKA-14044] (#12376)
- PR-12372 - KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
- PR-12371 - KAFKA-14035; Fix NPE in SnapshottableHashTable::mergeFrom() (#12371)
- PR-12204 - [9/N][Emit final] Emit final for session window aggregations (#12204)
- PR-12329 - KAFKA-14010: AlterPartition request won’t retry when receiving retriable error (#12329)
- PR-12139 - KAFKA-13821: Update Kafka Streams WordCount demo to new Processor API (#12139)
- PR-12293 - KAFKA-13963: Clarified TopologyDescription JavaDoc for Processors API forward() calls (#12293)
- PR-12312 - KAFKA-10199: Expose tasks in state updater (#12312)
- PR-12279 - KAFKA-10199: Commit the restoration progress within StateUpdater (#12279)
- PR-12269 - KAFKA-13966 Prepend bootstrap metadata to controller queue (#12269)
- PR-12291 - KAFKA-13987: Isolate REST request timeout changes in Connect integration tests (#12291)
- PR-12209 - KAFKA-13930: Add 3.2.0 Streams upgrade system tests (#12209)
- PR-11781 - KAFKA-10000: Per-connector offsets topics (#11781)
- PR-10738 - KAFKA-6945: KIP-373, allow users to create delegation token for others (#10738)
- PR-12298 - KAFKA-13998: JoinGroupRequestData ‘reason’ can be too large (#12298)
- PR-12304 - KAFKA-13880: Remove DefaultPartitioner from StreamPartitioner (#12304)
- PR-12226 - KAFKA-13890: Improve documentation of ssl.keystore.type and ssl.truststore.type (#12226)
- PR-12263 - KAFKA-13939: Only track dirty keys if logging is enabled. (#12263)
- PR-12161 - KAFKA-13873 Add ability to Pause / Resume KafkaStreams Topologies (#12161)
- PR-12206 - KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
- PR-12287 - KAFKA-13846: Use the new addMetricsIfAbsent API (#12287)
- PR-12248 - KAFKA-13958: Expose logdirs total/usable space via Kafka API (KIP-827) (#12248)
- PR-12181 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2) (#12181)
- PR-12250 - KAFKA-13935 Fix static usages of IBP in KRaft mode (#12250)
- PR-12121 - KAFKA-13846: Adding overloaded metricOrElseCreate method (#12121)
- PR-11473 - KAFKA-13436: Omitted BrokerTopicMetrics metrics in the documentation (#11473)
- PR-11780 - KAFKA-10000: Exactly-once source tasks (#11780)
- PR-12140 - KAFKA-13891: reset generation when syncgroup failed with REBALANCE_IN_PROGRESS (#12140)
- PR-12264 - KAFKA-13967: Document guarantees for producer callbacks on transaction commit (#12264)
- PR-11779 - KAFKA-10000: Zombie fencing logic (#11779)
- PR-12267 - KAFKA-13947: Use %d formatting for integers rather than %s (#12267)
- PR-12197 - KAFKA-13929: Replace legacy File.createNewFile() with NIO.2 Files.createFile() (#12197)
- PR-12067 - KAFKA-13780: Generate OpenAPI file for Connect REST API (#12067)
- PR-12180 - KAFKA-13917: Avoid calling lookupCoordinator() in tight loop (#12180)
- PR-12270 - KAFKA-10199: Implement removing active and standby tasks from the state updater (#12270)
- PR-12245 - KAFKA-13410; Add a –release-version flag for storage-tool (#12245)
- PR-12240 - KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (#12240)
- PR-12235 - KAFKA-13945: add bytes/records consumed and produced metrics (#12235)
- PR-11778 - KAFKA-10000: Use transactional producer for leader-only writes to the config topic (#11778)
- PR-12159 - KAFKA-13933: Fix stuck SSL unit tests in case of authentication failure (#12159)
- PR-12170 - KAFKA-13875 Adjusted the output the topic describe output to include TopicID & se… (#12170)
- PR-12238 - KIP-835: metadata.max.idle.interval.ms shoud be much bigger than broker.heartbeat.interval.ms (#12238)
- PR-12005 - KAFKA-13803: Refactor Leader API Access (#12005)
- PR-12210 - KAFKA-13930: Add 3.2.0 to core upgrade and compatibility system tests (#12210)
- PR-12246 - KAFKA-13718: kafka-topics describe topic with default config will show segment.bytes overridden config (#12246)
- PR-11776 - KAFKA-10000: Add new preflight connector config validation logic (#11776)
- PR-12191 - KAFKA-12657: Increase timeouts in Connect integration tests (#12191)
- PR-12136 - KAFKA-13773: catch kafkaStorageException to avoid broker shutdown directly (#12136)
- PR-12084 - KAFKA-13845: Add support for reading KRaft snapshots in kafka-dump-log (#12084)
- PR-12183 - KAFKA-13883: Implement NoOpRecord and metadata metrics (#12183)
- PR-721 - Update CODEOWNERS
- PR-12225 - KAFKA-13946; Add missing parameter to kraft test kit ControllerNode.setMetadataDirectory() (#12225)
- PR-10830 - KAFKA-12902: Add unit32 type in generator (#10830)
- PR-12187 - KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (#12187)
- PR-12062 - KAFKA-13833: Remove the min_version_level from the finalized version range written to ZooKeeper (#12062)
- PR-12200 - KAFKA-10199: Implement adding standby tasks to the state updater (#12200)
- PR-12035 - KAFKA-13217: Reconsider skipping the LeaveGroup on close() or add an overload that does so (#12035)
- PR-12190 - KAFKA-13923; Generalize authorizer system test for kraft (#12190)
- PR-12160 - KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD (#12160)
- PR-12109 - KAFKA-13863; Prevent null config value when create topic in KRaft mode (#12109)
- PR-12150 - KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
- PR-12050 - KAFKA-13830 MetadataVersion integration for KRaft controller (#12050)
- PR-12165 - KAFKA-13905: Fix failing ServerShutdownTest.testCleanShutdownAfterFailedStartupDueToCorruptLogs (#12165)
- PR-12162 - KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs (#12162)
- PR-11748 - KAFKA-12635: Don’t emit checkpoints for partitions without offset-syncs (#11748)
- PR-12164 - Update note on upgrade from log4j to reload4j (#12164)
- PR-12087 - KAFKA-13851: Add integration tests for DeleteRecords API (#12087)
- PR-11916 - KAFKA-12703; Allow unencrypted private keys when using PEM files (#11916)
- PR-12135 - KAFKA-13785: [7/N][Emit final] emit final for sliding window (#12135)
- PR-11969 - KAFKA-13649: Implement early.start.listeners and fix StandardAuthorizer loading (#11969)
- PR-11775 - KAFKA-10000: Add all public-facing config properties (#11775)
- 040b11d7 - KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- PR-12108 - KAFKA-13862; Support Append/Subtract multiple config values in KRaft mode (#12108)
- PR-12131 - KAFKA-13879: Reconnect exponential backoff is ineffective in some cases (#12131)
- PR-12085 - KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
- PR-12010 - KAFKA-13793: Add validators for configs that lack validators (#12010)
- PR-11983 - KAFKA-13763: Refactor IncrementalCooperativeAssignor for improved unit testing (#11983)
- PR-710 - : Update jackson packages to 2.13.2
- PR-12049 - KAFKA-10888: Sticky partition leads to uneven produce msg (#12049)
- PR-12028 - KAFKA-13804: Output the reason why broker exit unexpectedly during startup (#12028)
- PR-11773 - KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618) (#11773)
- PR-12127 - KAFKA-13785: [8/N][emit final] time-ordered session store (#12127)
- PR-12128 - KAFKA-10199: Implement adding active tasks to the state updater (#12128)
- PR-12029 - KAFKA-13815: Avoid reinitialization for a replica that is being deleted (#12029)
- PR-12106 - KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
- PR-709 - : Update jackson packages to 2.13.2
- PR-12100 - KAFKA-13785: [6/N][Emit final] Copy: Emit final for TimeWindowedKStreamImpl (#12100)
- PR-12072 - KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
- PR-12111 - KAFKA-13865: Fix ResponseSendTimeMs metric in RequestChannel is removed twice (#12111)
- PR-11955 - KAFKA-12380 shutdown Executor in Connect’s Worker when closed (#11955)
- PR-12096 - KAFKA-13794: Fix comparator of inflightBatchesBySequence in TransactionsManager (round 3) (#12096)
- PR-12064 - KAFKA-12841: Remove an additional call of onAcknowledgement (#12064)
- PR-12092 - KAFKA-13834: add test coverage for RecordAccumulatorTest (#12092)
- PR-12090 - KAFKA-13852: Kafka Acl documentation bug for wildcard ‘*’ (#12090)
- PR-12075 - KAFKA-13841: Fix a case where we were unable to place on fenced brokers in KRaft mode (#12075)
- PR-12066 - KAFKA-13834: fix drain batch starving issue (#12066)
- PR-11703 - KAFKA-13588: consolidate changelogFor methods to simplify the generation of internal topic names (#11703)
- PR-12030 - KAFKA-13785: [5/N][emit final] cache for time ordered window store (#12030)
- PR-12052 - KAFKA-13799: Improve documentation for Kafka zero-copy (#12052)
- PR-12004 - KAFKA-10095: Add stricter assertion in LogCleanerManagerTest (#12004)
- PR-12063 - KAFKA-13835: Fix two bugs related to dynamic broker configs in KRaft (#12063)
- PR-11993 - KAFKA-13654: Extend KStream process with new Processor API (#11993)
- PR-11681 - KAFKA-8785: fix request timeout by waiting for metadata cache up-to-date (#11681)
- PR-12033 - KAFKA-13807: Fix incrementalAlterConfig and refactor some things (#12033)
- PR-11945 - KAFKA-13769: Explicitly route FK join results to correct partitions (#11945)
- PR-12055 - [MINOR] Update upgrade documentation for 3.2 (#12055)
- PR-12036 - KAFKA-13823 Feature flag changes from KIP-778 (#12036)
- PR-10472 - KAFKA-12613: Fix inconsistent validation logic between KafkaConfig and LogConfig (#10472)
- 87aa8259 - KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
- PR-12031 - KAFKA-13651; Add audit logging to StandardAuthorizer (#12031)
- PR-12018 - KAFKA-13542: Add rebalance reason in Kafka Streams (#12018)
- PR-12043 - KAFKA-13828; Ensure reasons sent by the consumer are small (#12043)
- PR-11948 - KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
- PR-11939 - KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)
- PR-12006 - KAFKA-13794: Follow up to fix producer batch comparator (#12006)
- PR-11998 - KAFKA-13801: Kafka server does not respect MetricsReporter contract for dynamically configured reporters (#11998)
- PR-11842 - KAFKA-13687: Limiting the amount of bytes to be read in a segment logs (#11842)
- PR-11997 - KAFKA-6204 KAFKA-7402 ProducerInterceptor should implement AutoCloseable (#11997)
- PR-11974 - KAFKA-13763: Improve unit testing coverage and flexibility for IncrementalCooperativeAssignor (#11974)
- PR-11995 - KAFKA-13782; Ensure correct partition added to txn after abort on full batch (#11995)
- PR-11991 - KAFKA-13794; Fix comparator of inflightBatchesBySequence in TransactionManager (#11991)
- PR-11965 - KAFKA-13778: Fetch from follower should never run the preferred read replica selection (#11965)
- PR-11981 - KAFKA-13791: Fix potential race condition in FetchResponse#`fetchData` and forgottenTopics (#11981)
- PR-11941 - KAFKA-13749: CreateTopics in KRaft must return configs (#11941)
- f68f1a97 - Add muckrake mapping for 7.2 release
- baf8976f - Add muckrake mapping for 7.2 release
- PR-11978 - KAFKA-13786: Add a note in`control.plane.listener.name` doc (#11978)
- PR-11950 - KAFKA-12875: Change Log layer segment map mutations to avoid absence of active segment (#11950)
- PR-11829 - KAFKA-13785: add processor metadata to be committed with offset (#11829)
- PR-11928 - fix: make sliding window works without grace period (#kafka-13739) (#11928)
- PR-11953 - KAFKA-13772: Partitions are not correctly re-partitioned when the fetcher thread pool is resized (#11953)
- PR-11971 - KAFKA-13783; Remove reason prefixing in JoinGroupRequest and LeaveGroupRequest (#11971)
- PR-11963 - KAFKA-13777: Fix potential FetchResponse#responseData race condition issue (#11963)
- PR-11908 - KAFKA-13748: Do not include file stream connectors in Connect’s CLASSPATH and plugin.path by default (#11908)
- PR-11743 - KAFKA-13660: Switch log4j12 to reload4j (#11743)
- PR-11962 - KAFKA-13775: - Upgrade jackson-databind to 2.12.6.1 (#11962)
- PR-11967 - Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
- PR-11869 - KAFKA-13719: Fix connector restart cause duplicate tasks (#11869)
- PR-11966 - KAFKA-13418: Support key updates with TLS 1.3 (#11966)
- PR-11923 - KAFKA-6718: Add documentation for KIP-708 (#11923)
- PR-11942 - KAFKA-13767; Fetch from consumers should return immediately when preferred read replica is defined by the leader (#11942)
- PR-11760 - KAFKA-13600: Kafka Streams - Fall back to most caught up client if no caught up clients exist (#11760)
- PR-11949 - KAFKA-4801: don’t verify assignment during broker up and down in testConsumptionWithBrokerFailures (#11949)
- PR-11946 - KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (#11946)
- PR-11805 - KAFKA-13692: include metadata wait time in total blocked time (#11805)
- PR-11940 - KAFKA-13689: optimize the log output of logUnused method (#11940)
- PR-11920 - KAFKA-13672: Race condition in DynamicBrokerConfig (#11920)
- PR-11926 - KAFKA-13714: Fix cache flush position (#11926)
- PR-11933 - KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (#11933)
- PR-11892 - [Emit final][4/N] add time ordered store factory (#11892)
- PR-11932 - Revert “KAFKA-7077: Use default producer settings in Connect Worker (#11475)” (#11932)
- PR-11912 - KAFKA-13752: Uuid compare using equals in java (#11912)
- PR-11796 - KAFKA-13152: Replace “buffered.records.per.partition” with “input.buffer.max.bytes” (#11796)
ksqlDB¶
- PR-9468 - remove cc-docker-ksql from downstream builds
- PR-9567 - fix: use resolved configs in precondition checker
- PR-9394 - fix: fix regex used to extract queryId from threadId metrics tag
- PR-9392 - fix: move udf loading to run before the precondition checker
- PR-9391 - fix: make sure to close clients from precondition checker
- PR-9388 - fix: compare topics not sources
- PR-9393 - fix: use internal topic config for transient queries too
- PR-9389 - refactor: remove AVRO_SCHEMA_ID & SCHEMA_ID from QTT historical plans
- PR-9378 - feat: Add support for four and five column arguments to UDAFs
- PR-9361 - feat: UDAFs with multiple/variadic args
- PR-9366 - feat: Add log, power, and cbrt UDFs
- PR-9351 - refactor: combine yatt input and output nodes into one topic node
- PR-9341 - fix: make api client recognize ddl warnings better
- PR-9360 - refactor: Materialized to MaterializedFactory
- ef65f924 - Addressed Jim’s comments
- PR-9336 - fix: Map invalid casts to null.
- 3d2a56f1 - Updated documentation for detailed processing log in KsqlDB.io project
- PR-9337 - bugfix: remove log4j from the classpath (#9334)
- PR-9321 - fix: allow YATT to insert into and check contents of DDL sources
- e986f668 - fix: Create a KsqlSerializationException class
- 56dddbb1 - fix: classify KsqlSerializationException as USER error based on topic(KSE-1045)
- PR-9327 - Bump changelog version heading to 0.27.1
- PR-9130 - fix: use JsonSchemaConverter to support JSON anyOf types
- PR-9314 - fix: Allows functions which return maps to be dereferenced again.
- 6f656c0a - fix spotbugs
- PR-9283 - feat: Adding ksqlDB Query Status metric.
- a700c7ec - refactor: rename getAuthToken to getAuthHeader
- PR-9300 - feat: refresh service context and topic client in precondition checker
- PR-9272 - refactor: Migrate legacy UDAFs to use current annotations
- PR-9203 - feat: Support pausing/resuming persistent queries
- dcfe7941 - fix: Return proper status code for QPS ratelimit.
- PR-9277 - fix: DESCRIBE FUNCTION failing for annotated UDAFs with initial args
- PR-9255 - fix: change auth token provider to accept token strings instead of principals
- PR-9260 - fix: Excludes Guava from Guava-retrying in order to manage Guava depe
- PR-9246 - MINOR: improve error message for missing key
- PR-9248 - fix: Removing reverted configuration org.apache.kafka.streams.Streams
- PR-9239 - fix: add getAuthToken method to AuthenticationPlugin interface
- PR-9141 - feat: enable new emit-final implementation
- PR-9225 - fix: change consumer_group_member_id tag to just member to match Druid label name
- PR-9213 - feat: Added numerous trigonometric UDFs
- PR-9209 - fix:CAST function works with ISO-8601 timestamps with a trailing ‘Z’
- PR-9215 - fix: convert topic tag name and add consumer group member id tag to ThroughputTotalMetrics
- PR-9211 - fix: change group name and extend CumulativeSum in ThroughputMetricsReporter
- PR-9168 - feat: introduce ATTR aggregation function
- PR-9205 - fix: reset collector before reconfiguring
- PR-9180 - fix: add BYTES support for KAFKA format
- PR-9186 - fix: Allows results from CAST to compared.
- PR-9167 - fix: ambiguous reference to close issue
- PR-9134 - feat: cull the list of API consumable/editable properties
- PR-9144 - fix: move misplaced query-level configs to the correct list
- PR-9145 - fix: revert default /query-stream Content-Type to application/vnd.ksqlapi.delimited.v1 from application/vnd.ksql.v1+protobuf
- PR-9127 - fix: Fixes a few null handling bugs
- PR-9103 - feat: add ProtoBuf as a content type for pull queries over /query-stream endpoint
- PR-9045 - feat: add metric for query restarts
- PR-9120 - feat: Support all wildcard (*) on struct reference syntax
- PR-9105 - feat: clean up processing log metric
- PR-9107 - feat: add support for assert statements to migration tool
- PR-9099 - feat: add assert methods to java client
- PR-9035 - feat: add metric that’s emitted when processing log emits an error
- PR-9096 - feat: automatically build confluent cloud image on every master merge
- PR-9036 - fix: re fetch streams for each materializationProviderBuilder
- PR-9091 - feat: add ASSERT SCHEMA statement
- PR-9078 - Add PROTOBUF_NOSR
- 5423da9d - refactor: Fix checkstyle & make naming consistent
- PR-9086 - feat: assert not exists topic
- be09c0a6 - refactor: Set supportedArgs with string & bytes for max/min agg functions
- PR-9072 - fix: classify SR missing subject and access rights query errors as USER errors
- PR-9066 - feat: add ASSERT TOPIC command
- 030f2147 - feat: enable max/min udaf for string & bytes data types
- f2877e8d - fix: classify KsqlFunctionException as USER error
- 9e9d10e7 - fix: throw KsqlFunctionException while aggregating in sum udaf #9052
- 1bb24c31 - feat: migrate java client to use application/vnd.ksql.v1+json format
- PR-9047 - fix: INSERT/VALUES on a stream with SCHEMA_ID/SCHEMA_FULL_NAME fails
- PR-9026 - feat: support checking preconditions before starting core app
- PR-9040 - fix: use the engine’s KsqlConfig to build queries
- PR-9038 - fix: INSERT fails when serializing Proto/Avro nested Structs
- PR-9041 - build: exclude reload4j
- PR-9032 - fix: register state listener after restarting runtime
- PR-8986 - feat: allow aggregations without group bys
- PR-9028 - fix: remove double quotes from json_records function
- PR-8933 - fix: Create stream fails when multiple Protobuf schema definitions exist
- PR-9023 - fix: include header columns when injecting schemas
- PR-8918 - fix: Guard null struct dereferencing inside function calls
- PR-8984 - fix: INSERT VALUES fail when SR schema has a non-default name
- PR-9014 - fix: fail validation on create connector if connector already exists
- PR-8923 - fix: shared runtimes calculate cache size for validation properly
- PR-8999 - fix: move create connector validation to validate phase
- PR-8998 - fix: remove ErrorEntity and throw on connector error instead
- PR-8983 - Revert “feat: Allow to plug-in custom error handling for Connect serv
- PR-8977 - Improved/fixed aggregate function error messages.
- PR-8949 - feat: allow STREAMS with no key
- PR-8926 - fix: Repartition RHS of a FK join if it uses SR schema
- PR-8973 - fix: wait longer while waiting for expected spq
- PR-8947 - revert: consistency APIs
REST Proxy¶
- PR-1049 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-1044 - APIF-2768: Workaround for “Failed to bind to 0.0.0.0/0.0.0.0:9998” error.
- 87452021 - Run mvn spotless:apply
- PR-1039 - KREST-2655 Simple version of disconnect
- PR-1006 - KREST-4932: Add produce record rate-limited metrics.
- PR-1023 - KREST-5637 Use a Meter as recommended by observability and the right Metrics Object
- PR-1024 - Fix Checkstyle empty catch block error.
- PR-1022 - KREST-4067 Introduce import control to Kafka REST.
- PR-978 - KREST-4591 add topic recreate logic to cluster test harness
- PR-985 - KREST-4687 avro consume still fails
- PR-1009 - KREST-5732 Refactor to allow us to override producer metrics in ce-kafka-rest
- PR-1014 - KREST-5830: Create a lazy wrapper around MappingIterator for Produce Action.
- PR-1012 - KREST-5637 Move to cumulative sum for billing metrics
- PR-1008 - KREST-5637 Add count based byte metrics and move tracking earlier
- PR-1002 - KREST-5385: Add error_code to produce responses.
- PR-1003 - Remove ProducerPool.
- PR-1001 - Remove KafkaRestContext#getProducerPool.
- PR-350 - Cherry-pick https://github.com/confluentinc/rest-utils/pull/349 to 5.4.x
- 3d1250af - APIF-2739: Upgrade Jetty to 9.4.48.v20220622.
- d93674fb - Do not create a new sensor for error count
- 88556559 - Set sensors to be expired in 1 hr
- ff3595a0 - DGS-4220: fix request tag based metrics
- PR-343 - Update CODEOWNERS for APIF team
- PR-320 - APIF-2714: Switch from confluent-log4j to reload4j (5.4.x)
- PR-329 - APIF-2705: Update jersey version to 2.36.
- PR-316 - MMA-12033 Fix the connections limits test
- PR-315 - KREST-4977 Allow limiting the number of active connections.
- PR-310 - KREST-4450 500 error when topic not present
Schema Registry¶
NOTE: DGS-4389 added support for Protobuf custom options, which may change the behavior of schema lookups. To retain the old behavior in the Protobuf serializer, set schema.format=ignore_extensions.
- PR-2399 - DGS-5084 Ignore compat check in IMPORT mode
- PR-2395 - DGS-4971 Handle map types with enhanced.protobuf.schema.support
- PR-2389 - MINOR: Update ErrorMessage description
- PR-2388 - MINOR: Add ErrorMessage OpenAPI descriptions
- PR-2387 - MINOR: Add OpenAPI description for deleteGlobalConfig
- PR-2385 - Add OpenAPI operation tags
- PR-2378 - Migrate PowerMock to Mockito in RestServiceTest (#2372)
- PR-2381 - Adding timer in onJoinPrepare
- 0a9fb7d8 - Renamed DocumentedName to RootResource.java
- PR-2376 - DGS-4768 Fix reserved ranges for Protobuf enums
- 646cf301 - Added DocumentedName to RootResource.java
- PR-2373 - DGS-4724 Qualify names and merge maps when normalizing custom options
- 2fd5e6e9 - Remove deprecated methods in SchemaRegistryMetric
- PR-2364 - Adding timer in onJoinPrepare
- PR-2359 - DGS-4395 Fix message indexes of normalized Protobuf with map
- PR-2357 - DGS-4389 Add support for Protobuf v2 extensions
- PR-2353 - [DGS-4361] Added “/schemas/ids/{id}/schema” endpoint
- PR-2351 - DGS-4358 Fix NPE in Protobuf converter for null map value
- PR-2346 - Optimize sync call
- PR-2345 - Add leader change listeners
- PR-2342 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-2339 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- PR-2336 - DGS-4249 Fix name resolution during Protobuf normalization
- PR-2331 - DGS-4172 Bound size of Avro datumReader/Writer caches
- PR-2329 - DGS-4163 Pass topic to schema formatters
- PR-2328 - DGS-4162 Handle refs at different levels in Protobuf converter
- PR-2327 - DGS-1648 Allow Protobuf msg fullname to be passed for console producer
- PR-2326 - DGS-4134 Add config to ignore default for nullable fields
- PR-2313 - DGS-3862 Upgrade to spotbugs 4.7.0
- PR-2305 - Change everit-json-schema coordinates to reflect artifact published on Central
- PR-2289 - Added Custom Annotation
- PR-2262 - Remove static reference over java.util.Random
- PR-2198 - Set-compatibility Goal
- PR-2197 - Adding folder support for Test Local Compatibility maven plugin
Commercial Features¶
Confluent Server¶
- METRICS-4649 Refactor Remote Configuration activeFilters Semantics for 7.3.x
- KMETA-451; Allow broker registration with older confluent.metadata.version
- KMETA-448; Fix auto leader balancing of linked partitions
- Exclude non-fips dependency(bcprov-ext-jdk15on) from trogdor project (#7673)
- Exclude fips and non-fips bouncycastle dependency jar from connect packages (#7668) to 7.3.x
- Back port fix for clm test to 7.3.x
- Cherry-pick excluded bc jdk15 (non fips) from :ce-broker-plugins as it uses non-fips bc jars during compile time to 7.3.x
- KMETA-436; Fix compatibility break with default principal schema
- KAFKA-14265: Prefix ACLs may shadow other prefix ACLs
- KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. (#12628)
- Remote Config bug fixes (#7495)
- KGLOBAL-2203: Fix MRC assignment logic so rack positioning in placeme
- Add log message when skipping deletion
- Cherry-pick KGLOBAL-1812:cb3b61c3c7e2d60b17feb2deb6580dfd2fec4973 to 7.3.x
- Cherry-pick KGLOBAL-1777:4d6485ea189094f7c1935de786c607034f91538b to 7.3.x
- Bump version to 3.3.0
- CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
- RCCA-8564: log a warning if LDAP login fails becuase of network issues
- CIAM-2290: Upgrade bc fips drivers
- KAFKA-14156: Built-in partitioner may create suboptimal batches (#12570)
- KGLOBAL-1812: Fix thread leak in cluster linking test. (#7188)
- Increase timeout, correct error message returned for addBroker test
- KGLOBAL-1797: Remove rack mixing feature flag (#7421)
- Cherry-pick KGLOBAL-2127:72723ca0606d24a3894b58d4fad3eace9b0a07a0 to 7.3.x
- CONFLUENT: implement ControllerLoadTime metric for KRaft
- Cherry-pick Subset Partitioner Fix to 7.3.x
- KENGINE-229; Partition.isReplicaIsrEligible is misused in Partition.maybeIncrementLeaderHW
- KENGINE-212: batch transaction requests.
- KENGINE-211: Add a metric to record the avg latency for a transaction to transit from PreCommit to Commit.
- Remove the html end tag from upgrade.html
- KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case
- KMETA-290 Metadata shell supports snapshot and log
- KGLOBAL-1803 reset lastCaughtupTime when mirror leader appends to its
- Don’t include Server info as an HTTP response header in the Connect REST API (#7264)
- Add Javadoc to undocumented public APIs in SBC
- Update log level in RestClient
- CP 7.3 - Telemetry Reporter Remote Configuration [METRICS-4195][METRICS-4186][METRICS-4189]
- KAFKA-14170: Fix NPE in the deleteTopics() code path of KRaft Control
- RCCA-7224: Revert CIAM-1419 for 7.3.x
- Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
- KGLOBAL-1952: Attempt to shutdown both clusters in CL tests even if one fails to avoid thread leak impacting other tests
- KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing part
- KMETA-329; Workaround for AlterPartition regression on stale controllers
- cherrypick KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode
- Fix typo in offset writer thread config name
- Fix 7.3.x compilation
- KAFKA-14114: Add Metadata Error Related Metrics
- KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (#12457)
- [METRICS-4507] Add cluster linking metrics to Telemetry Reporter
- KAFKA-14107: Upgrade Jetty version.
- KAFKA-14114: Add Metadata Error Related Metrics
- Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.3) #775
- KMETA-321: Disable integration tests with co-located KRaft mode
- Avoid sending partial updates during LDAP group manager startup
- KSTORAGE-2280: Added a compaction CPU Utilzation Metric
- KDATA-484: Adding stateBeforeDeletion field
- [AUTHN-1097] Disable subject claim requirement in OAuth tokens
- KENGINE-219: Fix AlterPartition idempotency
- DGS-4302: ZKTopicMetadataCollector should avoid throwing exceptions
- [minor] Update README with PR docker image push
- Revert “CNKAF-1132: Record & unrecord total-throttle-time metric (#25
- KAFKALESS-1247: Fix CPU metric for non-Linux system and ConfluentTelemetryReporterSamplerIntegrationTest timeout issue
- KAFKALESS-737: Add system test for triggerEvenClusterLoad command
- SBC: Add Resource Optimization Detector framework
- Change CruiseControlMetricsProcessor log levels
- Build and push dirty images to GAR nonprod repo
- KAFKA-6945: KIP-373, allow users to create delegation token for others.
- KGLOBAL-1576 Refactor SaslServerAuthenticator auth to relay NetworkRegionId to SaslServer
- KAFKALESS-1261: Use newly added follower fetch rate metric
- Upgrading netty to v4.1.79.Final #754
- AUDIT-1139: Reducing the verbosity of NBKE
- Add metrics for monitoring of Kafka Management events
- KGLOBAL-1732: Update SBC to handle sync replicas and observers on the same rack
- Address review comments
- Address Yash Mayya’s comments around duplicate tests in AbstractWorkerSourceTaskTest and WorkerSourceTaskTest
- Upgrading netty version to 4.1.79.Final
- Upgrading aws-java-sdk-s3 to v1.12.268
- KAFKALESS-1261: Add fetch count metric at topic level
- Hotset Size Based Retention Breach Deletion of Compacted Segments
- Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 - remove Operation annotation
- Cherrypick 3ae1afa43838066e44ea78918050c6780c208042 KAFKA-10000: Integration tests
- Cherrypick 7098f04c3d5a30a6c16291dd78aa98694ce56e0b - fix WorkerSourceTaskTest
- Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 - Fix missing tracer invocations
- Cherrypick AK commit 9e8ef8bb317599c184ce8201d494edf109d9c528 and fix tests and refactor builder related changes in Worker
- Cherrypick AK commit 6853d63e4de03d679978add576aa0977cecc053a - Fix test failures related to producerConfigs and adminConfigs
- Remove updateConnectorConfig method
- Fix MethodLength related to DistributedHerder
- Cherrypick AK commit 603502bf5fb78983434a1a44ccc15a49ef6942b0
- Cherrypick AK commit a110f1fe852ae8c958a8c64b0736a9bb0617338e - Merge header configs along with producer, consumer and admin configs
- KAFKALESS-738: Even cluster load plan system tests
- Cherrypick KAFKA-13803
- Fix compilation issue in AbstractHerder, ReassignPartitionsCommand and use latest sink/source config from AbstractHerder’s ce-kafka/master
- Revert “KAFKA-10000: Add new preflight connector config validation logic (#11776)”
- Modified NonKafkaLogicalClusterMetadata parseLCM to be in sync with r
- KAFKA-13649: Implement early.start.listeners and fix StandardAuthoriz
- Cherry pick KAFKA-13474 on master to 6.1.x
- CONFLUENT: add code for deleting the Acls from the pod by passing BOOTSTRAP_SERVERS_CONFIG, lkcID and principals
- KREST-6986: Only add stats to sensor if metrics are not already registered
- KGLOBAL-1724: Wrong source topic name in DescribeMirrorsResponse.
- KDATA-454: add raft test annotation for clm test
- KGLOBAL-1786: Trodgor task for consumer group operations and listing offsets
- Sync this version of proto file
- CIAM-2169: Add KsqlCluster: Describe to EnvMV & CCMV
- Exponential backoff for automatic alter leadership calls
- KMETA-295: RuntimeException in TelemetryReporter init on KRaft controllers
- KAFKALESS-1216: Add ReplicaEntity and ReplicaMetricSample classes
- KAFKALESS-1167: Configuration of incremental balancing
- [METRICS-4509] Add Consumer Lag Offsets Metric to Telemetry-Reporter Whitelist
- CIAM-1503: Ability to de-code message headers of auth-topic
- AUTHN-1074: Support hierarchy for provider/pool
- KCFUN-199: Convert dynamic quotas tests to KRaft
- KAFKALESS-1217: Move TopicPartition to PartitionInfo
- KGLOBAL-1727 : add NetworkRegionId tag to CL SaslAuthenticate request
- AUTHN-974: Separate identity provider and pool resource types
- KCFUN-506: Improve the quota allocation algorithm by capping at broker limit
- KAFKA-14020: Performance regression in Producer (#12365)
- KAFAKALESS-752: CLI for ComputeEvenClusterLoadPlan
- KAFKALESS-1258: Use ConfigurationsImage during KRaft SBC startup and introduce in-memory BalancerEnabledConfig and consolidate/abstract SBC enablement there
- KAFKALESS-1268: Add metrics for Databalancer engine
- Added resourceId flag in aclCommand to list the ACLs in new format using Kafka CLI
- KAFKALESS-1270: Pause for a bit while Executor reservation acquirement aborts another executor run
- Update tenant transformations for kafka management audit logs
- SD-628: Added resource type and roles for Stream Designer pipelines
- CPKAFKA-8929 Disabling KRAFT failing test
- [METRICS-4508] Add Kafka Controller Preferred Replica Imbalance Count Metric
- KSTORAGE-2284, KSTORAGE-2285, KSTORAGE-2286: measure log append rate / data size / latency
- KCFUN-495: Make Dynamic Quota reconfigurible
- [METRICS-4503] Ignore Topology Change if Preferred Partition Leader Doesn’t Change for RandomBrokerPartitionSubsetPartitioner
- KGLOBAL-1730: Add support for replica placement file with sync replicas and observers on the same rack
- ReplicaManager should use brokerState instead of isShuttingDown to fence partitions followed by shutting down broker
- Fix connect_rest_test.py after introduction of new source configs
- DGS-4151:Update Rolebindings for some SR/DG related roles/operations
- Update CODEOWNERS
- CONFLUENT: Add metadata team to .github/CODEOWNERS for the old controller
- CONFLUENT: Avoid materializing collection in AbstractFetcherManager to compute sum
- AUTHN-1036: Change poolId principal prefix to always show User:
- KSTORAGE-1696: Non contextual or confusing tiering logs seen frequently in Confluent Platform
- KGLOBAL-1584: Add time to stop mirror topic metric
- KAFKALESS-1247 Temporarily disable ConfluentTelemetryReporterSamplerIntegrationTest.testSampler
- CIAM-2156: Add UI viewing permissions to SRResourceOwner, SRDeveloperX roles
- KGLOBAL-1658: Add source topic id to kafka-mirrors –describe output
- KC-2195: Implementing an HTTP API on the KRaft Controllers to check the quorum health
- AUTHN-881: Added support for poolId in authz audit event.
- KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
- KAFKALESS-734: Backend implementation for ComputeEvenClusterLoadPlan
- KDATA-476 Minor Remove unneeded wrap/unwrap in Option
- KAFKALESS-1254: Disable BrokerFailureDetectorTest.testLoadFailedBrokers
- CIAM-1518: KSQL modeled as a “cluster” for RBAC cloud
- Disable failed test CLIENTS-2345
- Disabled failed test CPKAFKA-6522
- CIAM-2178 Split Role Def File for SDS into KSQL SDS and SR SDS
- (JIRA ID : SEC-3593) CP LDAP - Enable Configurable Case Sensitivity for Authorisation
- AUTHN-974: Add providerId to pool events
- KGLOBAL-1769: logging node identifier in the error message and making
- KSTORAGE-2267: lingering storage metrics on deleted logs
- KAFKA-13837; Return an error from Fetch if follower is not a valid replica (#12150)
- KAFKALESS-1248: Temporarily disable testSelfHealingWithIgnoredBrokersPresentWithReplicaPlacements
- KAFKALESS-1222: Enable BrokerFailureDetectorTest.testPartialClusterFa
- DGS-3944: Add metrics for ZKMetadataCollector
- KAFKALESS-1230: Enable ReplicaPlacementSelfHealingTest
- KDATA-480 Abstract out Retry Policy from ObjectStoreUtils class
- [skip secret scan] KGLOBAL-1366: Enable CL system tests in KRaft mode
- Collect garbage collection metrics in TelemetryReporter [METRICS-4470]
- INIT-599 - Allow OrgAdmins to delete the whole Organization in cloud_rbac_roles
- [KPERF-454] Batch optimization for committing consumer group offsets.
- KMETA-83 Support for StandardAuthorizer benchmark
- DP-8085 - Migrate to Semaphore self-hosted agent
- KMETA-185: Explicitly start metric reporters in remote KRaft controllers
- Update CODEOWNERS for ce-metrics
- KMETA-249; Ensure linux metrics collected on remote controllers
- Add Cloud resource type of CLUSTER_LINK
- KDATA-392: restore system test to support kraft
- KAFKALESS-1227: Add NPE handling and consider all detection goals ski
- CIAM-2083: Move SDS rbac roles into separate json
- KSTORAGE-2279: Txn markers should not be removed by matching records in the offset map
- Properly gather partition information when detected topics with incon
- KGLOBAL-1351: Fix Incorrect prefixed-destination-link-count
- Bug fix for system tests.
- KGLOBAL-1085: Use default timeout for stop mirror topic.
- KMETA-239 Fix missing ControllerApis error handling
- Extract and Introduce libs for SDS engine to make authnz decisions outside of Kafka server
- DGS-3331, DGS-3332 Get topic config change and snapshot in ZK
- KAFKA-14036; Set local time in ControllerApis when handle returns (#12372)
- KAFKALESS-754: Allow altering SBC goals configs (confluent.balancer.rebalancing.goals and confluent.balancer.triggering.goals) dynamically
- Add detailed audit log integration test
- Fix for KafkaAuthStoreTest.testCacheFailureStatus
- MINOR: ignore consecutive handleMigration and trackInitLeader calls in TierDeletedPartitionsCoordinator
- KMETA-186 Fix AlterConfigPolicy usage in KRaft
- KAFKALESS-1221: Ensure SBC does not compute plans when reassignments exist
- KGLOBAL-1649: Compatibility is broken for createClusterLink requests in KRaft mode.
- KSTORAGE-2258: implement bucket storage probe metrics
- KSTORAGE-2137: enable FTPS cleanup in ce-kafka system tests
- [AUTHN-954] Add temp fields to IdentityPoolValue
- KAFKALESS-1218: Use Linux system cpu utilization
- KAFKALESS-733: Add ComputeEvenClusterLoadPlan Kafka admin API
- KAFKALESS-839 exclusion-aware ReplicaPlacementGoal
- fixed import order
- KCFUN-506: Set a minimum value for reported quota consumption
- KENGINE-194: Topic IDs not added to in sync fetcher pool
- KMETA-213: Fix NPE caused by missing null check in SnapshottableHashTable::mergeFrom()
- resolved failing tests
- KAFKALESS-1227: Disable test_topic_rebalance for ZK
- Adding tenant partition availability metric
- KCFUN-386, KCFUN-392, KCFUN-253: Setting a hard limit on number of partitions and topics per cluster. Partial update # of topics and partitions for in-flight requests
- Fix for test testWriterReelectionBeforeProduceComplete
- Fix for test testWriterReelectionBeforeProduceComplete.
- KAFKALESS-1222: Disable BrokerFailureDetectorTest#testPartialClusterFailure in ZK mode
- KAFKALESS-1221: Ignore reassignemnts cancel plan computation test
- KAFKA-13899: Use INVALID_CONFIG error code consistently in AlterConfig APIs
- KSTORAGE-2232: respect endOffset parameter when building offset map
- KAFKALESS-1207 Fix numBrokers created in onPrem case at ClusterModelPBTUtils
- KMETA-131 Cluster Linking metadata.version support
- AUTHN-908: Disable trust policy cache
- KSTORAGE-2249: update MergedLog.read to throw NotLeaderOrFollowerException on spurious OffsetOutOfRangeException
- reverted the LocalLog info logging that came from AK as its already logged as part of MergedLog
- Change DP schema request field config key default behavior
- CNKAF-1195: Don’t compute plan while reassignments are present
- KAFKALESS-1189: Fix SbcUpdateMetadataEvent to not override old metadata or get stuck in a loop
- Add support for slow logs in request logging
- reverted manually applied change from Unified as the logic to increment log start offset is different in MergedLog so this change is not needed
- CONFLUENT: Move license validator after startup completion
- KGLOBAL-1613: Persistent connection is not available (#6679)
- KGLOBAL-1614: Transform ACL binding filter in ClusterLinkSyncAcls when in multi-tenant env and add ACL migration semantics integ test to MultiTenantClusterLinkTest
- CONFLUENT: Log at debug level when pid unavailable
- Integrate Kafka management events with audit log provider
- KGLOBAL-1559: Do not fetch the metadata information under MetadataManager’s lock since the call is blocking.
- KGLOBAL-771: Source Initiated Links for KRaft.
- KAFKALESS-879: Get rid of capacityFor method
- KAFKALESS-1202: Log error from EvenClusterLoadStateManager whenever registering an event with exception
- KGLOBAL-1507: Filter out _schemas during cluster linking auto-mirroring
- KGLOBAL-1613: Persistent connection is not available
- KAFKALESS-1133: Add log to capture rack aware failure
- Modify DP schema request field to config key
- Refactor CLM tests
- KDATA-432: fix tier state fence restore test for Azure
- KPLATFORM-543: Move startup completion to end of startup sequence
- KGLOBAL-1611: Handle ClusterLinkDisabledException in ClusterLinkAutoMirroring when determining if mirror topics need to be filtered
- KGLOBAL-1473: converted ClusterLinkDestConnectionManagerTest and ClusterLinkAutoMirroringTest from easyMock to mockito
- KSTORAGE-1965: Not all bytes were read from the S3ObjectInputStream
- KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft (KIP-841, Part 2)
- KSTORAGE-2221: persist bounded base offset info of segments into FTPS
- KMETA-203 Add “confluent.metadata.version”
- KAFKA-13888; Addition of Information in DescribeQuorumResponse about Voter Lag (#12206)
- DGS-3504:Role changes for DG Catalog RBAC
- KDATA-180: Add internal durability audit metrics
- Kdata 448 refine restore rto
- KAFKALESS-732: Introduce ComputeEvenLoadPlan Protocol
- CONFLUENT: EasyMock -> Mockito conversion for most ce-kafka core tests
- Maintain mapping of userId to resourceId after deletion of API Key
- CIAM-1621: Make RBAC binding scopes support a tree structure
- Remove events from ce-audit as code owner
- RCCA-5913: provide exceptional use utility for mutation of topic ids
- MINOR: Fix broker load tests in MultiTenantKafkaIntegrationTest
- KGLOBAL-1561: Close reverse connections when deleting source side link.
- Fix CLM liveness metric
- KAFKALESS-751: Introduce CLI for trigger even cluster load
- Cherry-pick KAFKA-13935
- KDATA-455: address TopicIdPartition is None when rolling segment
- KGLOBAL-1050: Remove cache in ClusterLinkSyncTopicsConfigs task
- KAFKALESS-1186: Fix race condition in initializing BalancerStatusTracker
- MINOR: Convert oauth tests
- Add capability to enable trace record schema through DataPreview request
- Add principal to authentication failure audit log
- KCFUN-485: Memory leak in ThrottledChannelReaper
- KGLOBAL-1559: Resolve the deadlock between cluster link manager and cluster link metadata manager interactions.
- KAFKALESS-1177: Detect inconsistent replication factor by logging and emitting a metric
- Support dynamic config for resourceId in TenantAclProvider
- KDATA-388: RPO metrics for restore
- KGLOBAL-1451:Remove retry-time-based mirror failure when source topicids are known
- KC-2238: Reduce max.block.ms for telemetry producers in system tests to reduce broker shutdown time
- KSTORAGE-1699: Auto disable segment deletion throttler during low free disk space
- [AUDIT-1015] Fix producer emit operation future value to complete with true on success.
- Readd metrics plumbing for request handler avg idle percent metric.
- KGLOBAL-1485: Fail fast for persistent connection to non-coordinator
- CCLOG-1790 Connector Developer roles should be able to access metrics
- QEC-7888: Ensure that cluster link deletions in progress are completed on broker restart
- KAFKALESS-731: Correct the log for the EvenClusterLoad status value being verified.
- KAFKALESS-1109: Awakened events should execute before those in-queue
- KAFKA-13916; Fenced replicas should not be allowed to join the ISR in KRaft
- [skip secret scan] KAFKALESS-731: Update even cluster load state manager as design rework
- MINOR: Add AWS ECR setup to image building section
- KAFKALESS-1110: Handle rebalance drizzles with fixed-but-not-violated trigger goals.
- Bump telemetry client version to 3.163.0 for minor logging fix
- MINOR: Convert more integration tests
- Authn-526: Kafka AuthN rate and Latency Metrics per saslMechanism
- MINOR: SslCertificateIntegrationTest conversion
- Make resource id support config as dynamic config
- Fix exception handling in RBAC writer coordinator
- Remove reference to cc-deployer.mk
- AUTHN-813: Replace trust policy antlr parser with cel parser
- KCFUN-463: dynamic config to allow client supplied acks setting to be overridden [skip secret scan]
- KCFUN-481: Remove SocketServer state for disconnected channels when IP throttling expires
- KAFKALESS-525: Test both flavors of shouldShutdown in AddBroker tests.
- KAFKALESS-837: Rename Broker#State to Broker#Strategy
- Revert “CIAM-1419: Restructure DefaultAuthCache to lookup access rules for principal (#4758)”
- Revert “CONFLUENT: Rename DataPlaneAuthCache class name to CloudAuthCache (#4885)”
- Update source and serviceName CRN format for authentication failure events
- Revert “KDATA-404: New fields, and structure to be added to Durability Database”
- KDATA-450 CLM support for Azure object store
- KCFUN-128: Decouple recordThreadIdleRatio from request path
- KDATA-421: Add missing azure backend for tier storage system tests
- Test failure ‘KafkaService’ has no attribute ‘ACL_AUTHORIZER’
- Artifactory Migration
- MINOR: fix prefer.tier.fetch config for compacted topic in log_compaction_test system test
- KC-2144: Revert “AUTHN-280: Java client: implement async auth (#4659)”
- KC-2202 Disable kraft cluster linking system tests 7 2
- KENGINE-181: add recoverable partitions to GroupMetadataManager metrics
- KAFKALESS-1063: Allow more flexible leadership exclusion reasons.
- KGLOBAL-1547: Bump ClusterLinkRecord version since we have added a non-nullable link mode
- KGLOBAL-1415: Increase cluster link reconnect backoff max value
- MINOR: Add metadata team as code owners
- MINOR: Log PROXY protocol address when auth fails
- KAFKA-13858; Kraft should not shutdown metadata listener until controller shutdown is finished (KMETA-108)
- CIAM-2043 Separate permission for OwnKafkaClusterApiKey
- KGLOBAL-1394: Broker fails to start due to deleted cluster link (#6509)
- KSTORAGE-2060: Ignore missing file during log dir deletion
- [MMA-5228] allow operator to describe all topics
- CPKAFKA-8728, CPKAFKA-8729, CPKAFKA-8584 oauth system test fix
- KCFUN-112: Use a separate run method for interbroker network threads
- KGLOBAL-1394: Broker fails to start due to deleted cluster link
- [skip secret scan] KDATA-348: New events from Tier Metadata Snapshot Initiate and Complete
- KGLOBAL-1546: Fix race condition in acls method in StandardAuthorizerData
- KGLOBAL-1486: Deflake ClusterLinkTest.test_offset_migration_early_destination_group_start
- Efficiency Metrics for CLM
- Part7: EasyMock to mockito migration for CL tests
- KGLOBAL-1480: Part6 easyMock to mockito conversion
- KC-2202 Disable CL in KRaft mode for CP 7.2
- KMETA-149; Ensure forwarded requests are sampled for logging
- KGLOBAL-1506: Keep CreateClusterLinkPolicy state in sync with metadata log
- KMETA-160 Add Confluent records to metadata shell
- KC-2223: Disallow enabling SBC with KRaft in CP 7.2 release
- CloudClusterMetricsViewer should be able to view Connector Metrics
- Rename kafka config of user resource id support
- KDATA-353: Add feature flag configuration for FTPS snapshots and dynamic support
- Make the sasl handshake and mechanism max receive sizes configurable
- KC-2202; Disallow TS and KRaft in 7.2.x
- KMETA-172: Ensure partition epoch bumped before ISR expansion
- DGS-3640: Include leader epoch for MetadataImageListener::onLeaderUpdate
- KGLOBAL-1419: Add KRaft support for storing cluster link IDs with ACLs
- graduate MetricsViewer roles to public namespace
- KGLOBAL-1478: Part4 convert from EasyMock to Mockito
- Add view permission on cluster for connector roles
- KGLOBAL-1479: Part5 easyMock to mockito conversion
- [skip secret scan] Log consolidated final states of brokers
- KC-1907; Audit log wiring for the KRaft controller
- retention_stress_test: reliably spread writes across partitions
- MINOR: disable test case testBasicRetention in kraft mode
- KAFKALESS-529: Reset create time on new broker removal operation
- KGLOBAL-1490: Fix bug with mapping of link id to topics
- KGLOBAL-1475: Part3 convert from EasyMock to Mockito
- KGLOBAL-1474: Part2 convert from EasyMock to Mockito
- CONFLUENT: add more validation during KRPC deserialization
- retention_stress_test: follow up formatting fix
- CPKAFKA-5840: tier deletion test doesn’t produce data for the expected duration
- KGLOBAL-1489: Enable AlterMirrorsRequestTest for KRaft mode
- KAFKALESS-1076: Do not process altered exclusion events on inactive databalancer
- Upgrade RocksDB from 6.27.3 to 6.29.4.1 (#11967)
- build: kafka system tests should support larger EBS volumes for newer instances (#6371)
- Upgrade Gson
- KAFKA-13889: Fix AclsDelta handling of REMOVE_ACCESS_CONTROL_ENTRY_RECORD
- KGLOBAL-1471: Shut down ClusterLinkMetadataThread more gracefully
- KGLOBAL-1357: Enable testSourceTopicRecreateDetectionUsingTopicIds in KRaft mode
- retention_stress_test: long does not exist in python3
- Set hostname verifier to accept everything when identifica
- added support for userResourceID in delete Acls [skip secret scan]
- DGS-3330, DGS-3333 and DGS-3471, run one active TopicMetadataCollector in a cluster
- KGLOBAL-1481: Fix issue where ClusterLinkClearLinkReference does not run on startup in KRaft mode
- KGLOBAL-1483: Source topic id is not populated in create mirror topic request.
- KDATA-352: Cloud API for FTPS Snapshot upload
- CIAM-1793 MetricsApi is available to new DataplaneRoles
- Set hostname verifier to accept everything when identification algorithm is empty
- AUTHN-833: Pass sub and azp as separate validatedExtensions
- KAFKALESS-792: Stop SBC should interrupt SBC startup
- EVENTS-991: Add events team as codeowners of ce-events modules
- Add support for extracting AuditLog Entries from Kafka Request Events
- EVENTS-989: enable use of subset partitioner in events exporter
- build: kafka system tests should support larger EBS volumes for newer instances
- KGLOBAL-1418: Use admin client for creating/deleting ACLs in ClusterLinkClearLinkReference
- Update cloudevent encoding config doc
- CPKAFKA-7262: fully support 7.0 and 7.1 in upgrade, downgrade tests
- KSTORAGE-2180: upgrade flatbuffers to 2.0 to support M1 macs
- KAFKA-13879: Reconnect exponential backoff is ineffective in some cas
- KL-903/934: Enable EvenClusterLoadStatusTest#testBrokerRebalanceWithSelfHealing and SelfHealingAfterEnableTest#testBrokerRebalance
- KSTORAGE-2171: do not adjust baseOffset for compacted segment
- KC-2089: Ensure MZ cluster availability during a network issue (Add leadership priority calls)
- KMETA-16: Add tiered storage topic config change validation to KRaft
- [skip secret scan] KMETA-97 and KMETA-98: Fix compatibility issues with the topic CRUD request versions and bugs in mirror state lookup.
- KAFKA-13790; ReplicaManager should be robust to all partition updates from kraft metadata log (#12085)
- KREST-5636 Inject MT secrets store in Kafka HTTP server apps
- KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- EVENTS-874: Create deserializer for LogicalClusterMetadata and headers
- KSTORAGE-2173: skip unnecessary format migration and cleanup during FTPS init
- KSTORAGE-2176: refactor TierTestUtils.deletedSegments
- KCFUN-443: Skip reporting empty quota target[skip secret scan]
- KCFUN-391: Ignore Dynamic Quota if the cluster wide quota is unlimited[skip secret scan]
- KAFKA-13892: Fix bug where multiple remove records are generated for one ACL
- KDATA-409: retry when exception of downloading FTPS from kafka pods
- KAFKA-13854 Refactor ApiVersion to MetadataVersion (#12072)
- AUTHN-792: Fix async authn performance regression
- Resolve dependency issue in ce-broker-plugin
- CONFLUENT: Convert ce-kafka-client-plugins to JUnit 5
- KAFKA-10405: Set purge interval explicitly in PurgeRepartitionTopicIntegrationTest (#11948)
- Disable idempotent producer by default in Connect centralized licenses (#6193)
- Add different metric for filtering authorized resources vs authz denied
- KGLOBAL-1355: Fix ClusterLinkFailureTest.testDestinationHighWatermark for KRaft using buffered produce
- KDATA-434: change max wait time for AuditManagerTest
- back port fix for scalabe restore test to 7.2.x
- Update base image to use adoptium java 17.0.3 instead of correto
- Authorizer performance improvements
- KAFKALESS-833 PBT for Self-healing in the presence of ignored brokers [skip secret scan]
- KAFKALESS-1060 Ensure broker#isAlive is used properly
- Disable idempotent producer by default in Connect centralized licenses
- Add Connect team as a codeowner for /connect/
- KAFKALESS-874: Port the last set of broker removal integration tests to run in KRaft
- KGLOBAL-1438: Make confluent.cluster.link.metadata.topic.enable only applicable in ZK mode
- KMETA-127; Get quota integration tests working with KRaft
- KSTORAGE-2153: Handle Azure object store responses during dns failures to ensure they’re retriable
- CONFLUENT: Add error logs to AclAuthorizer
- KMETA-70: Support Confluent’s replica placement plugin with KRaft
- KAFKA-13861; Fix the validateOnly behavior for CreatePartitions requests in KRaft mode (#12106)
- KAFKALESS-1047: Always acess SBC related Enums by name
- Move partition creation request limit to policy
- METRICS-4064: Subset Partitioner Strategy to Reduce Telemetry Kafka Connections [skip secret scan]
- Add new resourceType and role mapping for RBAC in Health+ API
- KSTORAGE-2149: add compacted.topic.prefer.tier.fetch.ms config
- KSTORAGE-2156: skip dual compaction validation if there are tierable local log segments
- CONFLUENT: Convert rest-authorizer to JUnit 5 and improve its tests
- KC-2116: Migrate tier_unclean_leader_election_test to KRaft
- Update jackson packages to 2.13.2 version
- Fix backward incompatibility issue in license store (#6152)
- KGLOBAL-1349: Enable security for cluster link tests with KRaft
- Update jackson packages to 2.13.2 version
- KSTORAGE-2119: refactor FTPS tools to adapt to cleanup feature
- KSTORAGE-1948: Implement FTPS cleanup logic
- KGLOBAL-1390: Use admin client instead of local authorizer in ClusterLinkSyncAcls for creating/deleting destination ACLs
- KDATA-428: add clusterid parameter in restore system test
- CONFLUENT: Record total usage for cluster link quota even if user quota not set
- KAFKALESS-1089: Increase reason limit for replica exclusions
- KGLOBAL-1297: Revert cluster link sync filter config validation changes
- KAFKA-13660: Switch log4j12 to reload4j (#11743)
- METRICS-514 Kafka event log
- KSTORAGE-2158: exclude internal topics from tier compaction
- KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism (#6235)
- KCFUN-413: Disable user tagging for Client Request Quota
- KSTORAGE-2163: regression in confluent.tier.local.hotset.ms dynamism
- Add kafka-eng as CODEOWNER
- Fix backward incompatibility issue in license store
- KMETA-104: Add internal REST server to KRaft remote controller
- KDATA-313: Segment existence and metadata validation checks
- KAFKA-13743: Prevent topics with conflicting metrics names from being created in KRaft mode #11910
- Add muckrake mapping for 7.2 release
Security¶
- Removed bcfips from common
- Pin bouncycastle fips dependency in confluent-security-plugins
- Exclude bcpkix-jdk15on as compile dep and add as test dependency
- made SchemaRegistryResourceActionKey private again
- Made nonGlobal request a separate method to have different implementation in cloud
- Backport “Fix a NoClassDefFoundError caused by a missing dependency (#518)” to 7.3.x
- making the SchemaRegistryResourceActionKey and schemaRegistryResourceActionMap protected so that it is visible to the subclasses in cloud plugins repo.
- AUTHN-1087: Upgrade vertx to 4.3.2
- AUTHN-1085: Fix NPE in debug log
- APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- fix: add getAuthHeader method to ksql authorization plugin
- trigger test
- fix tests
- add javadoc
- Trigger Build
- remove unused import
- fix: add getAuthToken method to ksql plugin
- fix: Exclude guava dependencies in ksqldb-rest-app
- fix: get JwtToken from string instead of JwtPrincipal
- Remove redundant config validation in ksql security extension
- checkstyle
- overrides
- rebase to master
- add auth token provider to ksql security extensions
- KSE-1081: Add conf to enable/disable KSQL-SR permission checks
- DGS-3862 Upgrade spotbugs to 4.7.0
- SEC-3245: Migrate from confluent-log4j to reload4j
- Refactor KSQL authorizer and security extension to allow Cloud authZ implementations
- Delete the JAR fetched by MVN wrapper
- Revert “Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision”
- Add LoggableAuthenticatedPrincipal that wraps a LoggableAuthenticationDecision
- KSQL: Wrap RestAuthorizer on a new AuthorizationDecisionMaker class
- Addendum
- Addressing Sergios comments
- Update KsqlSecurityExtensionConfig.java
- Remove Kafka REST ProducerPool.
- KSE-947: Add ksql.service.name to KsqlSecurityExtensionConfig
- KSE-859: Add KSQL config to enable or disable security user impersonation
- update codeowners
- Prepare for removal of Kafka REST ProducerPool.
- feat: add maven wrapper
- Added exclusion since MDC class is failing because of reload4j
- Minor: Fix KafkaYammerMetrics import
Secret Registry¶
- Exclude Non-FIPS dep and use bc-fips dependency
- Handle illegal reflection access under java 9+
- Implement onJoinPrepare with latest AK change in KAFKA-14024
- Migrate from log4j to reload4j for connect-plugin
- CC-17641,CC-17644 Bump jetty version
- APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
- ignore new findbugs 4.7.0 detectors causing secret-registry build to fail
- Migrate log4j
- Introduce GitHub CODEOWNERS (Pull Request Reviewers)
Control Center¶
- [MMA-12484][MMA-12483] Introduce Maven Profile for passing different configuration for JDK8 and JDK17
- [MMA-12432] Support Control Center on JDK 17
- MMA-12201 remove slf4j from transitive deps
- MMA-12212: Upgrade jetty-http in blueway
- MMA-12206 upgrade transitive dependency of netty
- MMA-11617: allow for trailing slash in path
- [MMA-12212][MMA-12211] Bump jetty version in blueway
- MMA-12206 upgrade io.netty:netty-codec-http
- MMA-12208 Upgrade okhttp
- MMA-12201 Exclude slf4j in transitive dependency
- MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
- MMA-12201: Use reload4j version properties from common
- MMA-12201 Add logredactor [7.1.x]
- MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
- RCCA-7746: CCloud RBAC user unable to view messages from UI
- MMA-12201 Replace log4j with reload-4j
- MMA-12208 Upgrade okhttp
- MMA-12106 Add new api’s added in kafka streams
- MMA-12010 Suppress errors due to Spotbugs upgrade to 4.7.0
- CIAM-1533 make the default replication factor for Kafka API topic creation configurable
- KEXP-349 experiences code owners
- Remove Kafka REST ProducerPool.
- Prepare for removal of Kafka REST ProducerPool.
MQTT Proxy¶
- Fix 7.3.x test dep for MockFaultHandler
- Update pom.xml
- CC-17620:Remove dependency on slf4j-log4j12 and confluent-log4j
Metadata Service¶
- Fix FIPS deps
- Remove spaces
- Add back newline
- Implement writeTransactionMarkers in DelegateConfluentAdmin
- CIAM-2304: Add unit tests for SecurityAdmin
- Remove unnecessary dep on noop logger
- MMA-12388: Upgrade LD SDK to 5.6.4
- Revert “Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)”””
- Promoted v0.451.0 Extractor to PROD
- CIAM-2201 Close HTTP Application before closing authorizer
- Merge 7.2.x to master
- fix pass
- [AUTHN-1000] stag/prod migrations for provider uniqueness
- RCCA-8245: Fix for invalid consumer group for sync listener
- [AUTHN-1018] Ignore deleted entries on provider/pool lookup
- trust-service: Use v0.47.0 of auditlog library which fixes packaging issue
- [AUTHN-1000] Make identity provider index based on jwks/issuer uniqueness
- CIAM-2219 Re-open DP- 8085 migrate semaphore agent
- Deploy version v0.441.0 of both cc-rbac and Extractor to prod
- CIAM-2237 - Test and build stability prophylactic measures
- [AUTHN-1081] Fix resourcespec parsing
- [AUTHN-1075] Handle server errors gracefully
- AUTHN-541 trust-service: Use recommended methods while testing audit logs
- AUTHN-541 trust-service: Set default=”” for auditlog bootstrap server
- [AUTHN-1070] Enable trust-service feature flags by default
- CIAM-2139 Refactor Extractor code to rename CloudCacheHierarchy to CloudHierarchyCache
- AUTHN-974: Authorize with separate provider/pool resource types
- CIAM-2219: MINOR: Add vault secrets for Docker rate limit raise
- Improve Build Stability
- chore(deps): bump cc-base from v18.6.0-jdk-17 to v18.9.0-jdk-17
- AUTHN-541 trust-service: Add some logging while initializing AuditLogger object
- AUTHN-541 trust-service: Do not auditlog if not configured via configs
- AUTHN-541 trust-service: Use X-B3-Traceid header as requestId
- CIAM-1176 - Remove debugging print statement
- Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
- Update trust-service/src/main/resources/WEB-INF/openapi/trust-service-spec-v2.yaml
- Update the oauth validation API specs
- CIAM-1176: Increase Role Bindings Quota
- CIAM-2049 Publish LC RoleBinding with K8 ID for New LC Creation
- [AUTHN-968] Add fixed seed to hashcodes
- [AUTHN-1041] Add identity pool filter limit to update calls
- AUTHN-1074: Update tests to exercise hierarchy for providers/pools
- Revert “DP-8085 - Migrate to Sempahore self-hosted agent (#1187)”
- DP-8085 - Migrate to Semaphore self-hosted agent
- AUTHN-974: Tests to exercise trust service provider/pool authorize
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- chore(deps): bump docker/prod/confluentinc/cc-base from v16.4.0-jdk-16 to v18.6.0-jdk-17
- [JIRA-ID: SEC-3597] - Integration Tests For Configurable Case Sensitivity for Authorisation in CP LDAP
- AUTHN-541 trust-service: Accept X-Request-Id as header param for audit logging
- AUTHN-541 trust-service: Address review comments
- CIAM-2211 Configure the PKC and LKC ID for SDS in config file
- Add utc time zone to metadata timestamps
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- AUTHN-882: Replace principal with pool id
- [AUTHN-954] Modify request field names
- fixes 6.0.x build from 5.5.x merge
- [AUTHN-820] Create AWS IAM Role for Trust service
- CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS
- DGS-4151: Fixed mds test for Org/EnvOperator Describe access on Subje
- [AUTHN-948] Rename spec field for identity providers
- CIAM-2197 Fix the type of SR AND KSQL LD Flag
- CIAM-2183 - MDS code fails validation for a topic name >= 80 characters
- CIAM-2183 - MDS code fails validation for a topic name >= 80
- AUTHN-989: Fix multithreading race condition where we only retry refreshing once
- [AUTHN-997][AUTHN-948][AUTHN-946] Update Trust-Service Metadata
- CIAM-1964 Publish RBAC Crud Changes for KSQL/SR with Fixed PKC ID
- [AUTHN-949] Ensure that identity pool operations are using the right provider
- [AUTHN-997] Rename identity pool’s spec to status
- CIAM-2173 Route Kafka and SDS role bindings use RoleDef Files
- CIAM-1518 Generalize Cloud Scope to take SR & KSQL
- AUTHN-974: Add providerId to pool events
- CIAM-2184 Extractor Publishing empty PKC ID if L*C not found in CHC
- AUTHN-541 trust-service: Add a unit test to verify AuditLogUtils
- AUTHN-541 trust-service: Ignore auditlog for IdentityPrincipal & HealthCheck
- AUTHN-541 trust-service: Audit log CRUD APIs for IdentityPool resource
- AUTHN-541: trust-service Audit log CRUD APIs for IdentityProvider resource
- AUTHN-541 trust-service: Update AsyncReponses to accept audit objects
- AUTHN-541 trust-service: Add a util to help build the AuditLogger object
- AUTHN-541 trust-service: Register and bind AuditLogger object for tests
- AUTHN-541 trust-service: Register and bind AuditLogger object
- AUTHN-541 trust-service: Add auditlog config
- AUTHN-541 rbac-extractor: Update events-schema lib from v0.73.0 to v0.82.0
- AUTHN-541 trust-service: Add events-schema & auditlog-emitter-java dependency
- [AUTHN-954] rename data fields for identity pools
- CIAM-2130 - Deactivate environment scoped role-bindings for deactivated accounts in Prod
- INIT-599 - Allow OrgAdmins to delete the whole Organization
- CDMUM-2091 Add DECISION_ENGINE as accepted audience
- Promoting cc-rbac to version 0.400.0
- CIAM-2171 CHC API for LC -> PKC to use describeLogicalCluster
- [PROD][AUTHN-822] Add db user with rds_iam role
- [AUTHN-821] enabling RDS IAM Auth on rbac database
- [AUTHN-954] publish duplicate fields for IdentityPoolValue
- CIAM-2147 Integration With CHC failing with incompatible vertex version
- AUTHN-918[PROD]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- [CIAM-2077]: Add API to Integrate with CHC Batch Processing
- Revert “CIAM-2147 Integration With CHC failing with incompatible vertex version (#1174)”
- CIAM-2147 Integration With CHC failing with incompatible vertex version
- Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1173)”
- CIAM-2147 Update CHC version to fix incompatible vertex version
- Revert “CIAM-2147 Update CHC version to fix incompatible vertes version (#1172)”
- CIAM-2147 Update CHC version to fix incompatible vertes version
- [AUTHN-973] Modify rbac operation request based on status update #1171
- RCCA-7223: Scale down cc-rbac cpd replicaCount from 2 to 1
- AUTHN-918[DEVEL]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- CIAM-2147 Update CHC version to fix incompatible vertes version
- Promoting cc-rbac to version 0.385.0
- AUTHN-968: Add logging around 500 error in createIdentityPool
- AUTHN-918[STAG]: Add ‘claims.’ prefix to subject_claim to identity pools in database
- [CIAM-1973] Add API to fetch K8 ID from CHC
- Promoted Extractor to v0.381.0
- [CIAM-2131]: Deactivate environment scoped role-bindings for deactivated accounts in Devel
- [CIAM-2127]: Deactivate environment scoped role-bindings for deactivated accounts in Stag
- CIAM-2146: Promoted Extractor to v0.374.0
- [AUTHN-960] Fix create identity providers bug
- revert import optimizations
- re-add fallback for missing gateway host
- remove unnecessary property
- revert endpoint prop and set gateway host always
- add rds endpoint property
- fix bracket
- fix style
- adding aws dependencies
- add semicolons
- add back poassword, cleanup
- [AUTHN-824] instrument service to connect to DB with RDS IAM Auth
- [CIAM-2138]: Updated ConfigMap Value
- AUTHN-740: Add better logs for retrying on refresh failure and fix default refresh interval when cache control is not present
- Add service.yaml file for rbac extractor
- Added getParameterType to error message. Removed getParameterName
- CIAM-2078 Add Exception Handling in CHC API
- RCCA-6743: Test that we can allow rest client to disable hostname verification
- Promoted Extractor version
- trust-service: Add a test to validate CRN format
- AUTHN-794 trust-service: Fix CRN format
- CIAM-2096: Decreased total hc topics from 60 to 20
- [CIAM-2138]: Update CHC API to accept host and port instead of API
- AUTHN-943: Update quota limit error message for idp and pool
- CIAM-2044: Fixing db transaction deadlock caused by delete.
- RCCA-6760 Delete invalid rolebindings
- Rebase with upstream and fix test
- address review comments
- AUTHN-918: update rbac extractor to handle the new sub claim prefix
- AUTHN-918: enforce subject claim to start with ‘claim.’ prefix in IdentityPool create and update
- [AUTHN-894][AUTHN-749][AUTHN-816][AUTHN-817] Update creation/deletion logic
- RCCA-7563: Delete crufty Rolebindings for large customer
- CIAM-347: Prevent OrgAdmin from self-deleting OrgAdmin role for V2 API
- Added getParameterType to error message. Removed getParameterName
- CIAM-2096: Decreased total RBAC HC topics to 20 so decreased total RBAC HC role
- Added getParameterType and getParameterName to error message
- Adding two test cases for pool filters
- Added unit tests for different ParamException types.
- trust-service: Add a test to validate CRN format
- AUTHN-794 trust-service: Fix CRN format
- Catch ParamException 400 errors. Renamed file.
- Catch ParamException 400 errors.
- CIAM-2021 MDS changes to support Tree Binding Scopes
- Remove duplicate dependency
- Remove cpd docker pull secrets
- CIAM-2094: Decrease total HC topics to 20.
- Address review comments
- Implement LDClient for provider and pool limits in trust service
- Skip regular builds for changes in .deployed-versions
- [STAG][AUTHN-822] add db user with rds_iam role]
- CIAM-2093, RCCA-7359: Set cc-rbac prod version to 110, rbac-extractor prod version to 295 (0.350.0)
- CIAM-2093: Update the num of Extractor pods to 0 in cpd
- AUTHN-813: Replace trust policy antlr parser with cel parser
- Address review comments
- Revert “Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)””
- CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
- Integrate JWKS limit with LD
- CIAM-2071: Delete invalid role bindings from v2 api (for internal orgs) [prod]
- [AUTHN-822] add db user with rds_iam role
- Promoted rba-extractor to version 0.340.0 on PROD, DEVEL and CPD
- CIAM-1959 Extractor LKC processor should resume at last committed, not EndOffset
- CIAM-2075 Disabling failure check of PKC Header failure for Integration Testing
- Implement jwks limit
- CIAM-1970: Implement displayName resolution for V2 API
- Set cc-rbac prod version to 93 (0.333.0)
- Address review comment
- address review comment - check filter limit before pool limit
- Added tests
- CIAM-1863: Fix deployed versions name
- CIAM-1929 Added validation on role binding scope.
- Enforce resource limit for IDP, pool and filter in trust service
- CIAM-1828 Publish PKC Header For RBAC Role binding Changes
- Fix Fuzz Test by specifying most specific scope
- [CIAM-2040] Add Log Statement to find out if different ResourceTypes for a single role binding are used by customer
- CIAM-1863: Update icc-rbac and extractor pipelines to stag->prod->devel->cpd
- CIAM-1880 - Runbookize our DBMigrate approach to deleting rolebindings
- CIAM-2030: Updated pattern_type to LITERAL on stag and prod
- CIAM-2025: Fixed error on access RBAC role_bindings sequences.
- CIAM-1523: Start routing traffic to icc-rbac
- CIAM-2028 Remove Dataplane LD Flags from extractor
- [CIAM-1840] KSQL/SR Extractor Record Header
- CIAM-2025: Grant usage on sequence rbac.role_binding_last_change_id_seq to cc_rbac_extractor_0 and cc_rbac_extractor_1
- CIAM-2030 - Updated pattern_type to LITERAL for org 0 role bindings
- CIAM-2024: Added logic cluster type healthcheck .
- CIAM-2027 Update CHC Client Version to support JDK 8
- CIAM-1823 Interface with Cloud Hierarchy Client
- CIAM-1786 Add ResourcePattern to correctly resolve display name
- AUTHN-835, AUTHN-845: Trim whitespaces on user entry
- [CIAM-1839] Enable identification of KSQL/Schema Role bindings
- CIAM-1902: Inserted 60 role bindings into rbac db for prod
- CIAM-1897: Remove client_address field from icc-rbac audit logs
- Remove extra semicolon
- CIAM-1919: Address issues from icc-rbac ops review
- RCCA-6909 - Customer needs role bindings deleted for deleted lkcs again
- CIAM-1939 producer close should timeout
- Fix dependencies
- [AUTHN-747] remove policy version/id
- CIAM-1902: Added role bindings to SA for stag.
- Fix JWTDebugLoggingTest failure by excluding log4j jars
- CIAM-1954 MINOR: Healthcheck to check RUNNING immediately
- CIAM-1951: Updated deployment notification.
- CIAM-1432: Make icc-rbac audit logs match kafka mds
- CIAM-1432: Add icc-rbac devel auditlog config to stag and prod
- AUTHN-737, AUTHN-752, AUTHN-720: Fix self links, add rbac.addr
- MINOR Add log for manual republish
- CIAM-1752: Enable icc-rbac db metrics
- [AUTHN-750] [RCCA-6673] Grant permissions for cts schemas to rbac-extractor user
- Update CODEOWNERS
- AUTHN-582: Add rbac checks for trust service provider/pool CRUD apis
- [AUTHN-567] Add validation checks to getJWKS()
- CIAM-1912 AuthN to be Codeowners of trust-service
- CIAM-1776 Update cc-base to v16.4.0
- CIAM-1432: Fix default cloudevent.codec
- Create/run migrations for trust-service in stag/prod
- CIAM-1544: Remove ce-kafka-version suffix from mds image version
- RCCA-6555 Remove role bindings associated with deleted clusters
- CIAM-1523: Add audiences config to icc-rbac
- [AUTHN-711] Modify prefix for identity pools
- Upgrade cc-base image
- CIAM-1432: Enable Audit Logging for icc-rbac
- AUTHN-619: Add getIdentityPrincipal call
Replicator¶
- RCCA-7678: Reverse proxy header check added
- KGLOBAL-2126 seek to begining only for non empty partitions list
- Add log redactor.
- Migrate confluent-log4j to reload4j.
- fix upstream build