Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Version 7.1.11

Released December 2023

Community Features

Common

  • e99d8b1a - exclude logback-core and logback-classic in line with the changes in kafka-rest
  • eb200670 - update zookeeper component to match version in ce-kafka
  • 0c034e86 - re-add dependencymanagement sections for old bouncycastle to satisfy downstream validation
  • 33734c5f - add additional version definition to ease the update of the downstream repos
  • 8e71caf2 - update bouncycastle

Kafka

  • c3f8c710 - CONFLUENT: Bump ZK version to 3.8.3 for 7.1.x
  • PR-14496 - HOTIFX: fix Kafka versions for system tests (#14496)
  • PR-14082 - KAFKA-15102: Add replication.policy.internal.topic.separator.enabled property to MirrorMaker 2 (KIP-949) (#14082)

REST Proxy

  • PR-1230 - KREST-12372 Upgrade dependency to JDK 1.8 with version 1.77
  • PR-1221 - KREST-12630 Exclude logback from zookeeper dependency in tests
  • PR-1219 - KREST-12630 Add missing dependency due to zookeeper upgrade
  • PR-1212 - KREST-12277 semaphore migration to release branches
  • PR-440 - KREST-11812 upgrade jaxb-api dependency
  • ee8dd479 - KREST-12364: update bcpix
  • PR-427 - KRES-12288 migration semaphore release branches

Schema Registry

  • 98a1a4de - update swagger core
  • PR-2888 - Apply service bot changes to version branches
  • bafbad8c - DGS-9343 Allow removing enum in JSON Schema to be backward compatible
  • PR-2874 - DGS-9048: Semaphore Migration of Version Branches
  • 899b38a2 - migrate to semaphore
  • 7f7a7ab1 - add metrics-core to fix the build
  • d3b1912d - upgrade com.squareup.okio:okio to 3.4.0
  • c8ab3fd0 - Add log if encoding fails
  • d81d5a53 - Encode subject names for Maven download plugin
  • PR-2820 - Cache ProtobufSchema.toDynamicSchema calls (#2818)
  • PR-2816 - DGS-8960 Fix handling of diamond structure in Protobuf converter

Version 7.1.10

Released October 2023

Community Features

Common

  • 35a1415d - APPSEC-3039: Update jetty and netty
  • 81295738 - update snappy to 1.10.5
  • aea31af8 - APPSEC-2796: add guava to dependencyManagement

Kafka

  • PR-972 - KSECURITY-2013: Upgraded Jetty to 9.4.53.v20231009 and Netty to 4.1.100.Final
  • PR-960 - KAFKA-15498: bump snappy-java version to 1.1.10.5 (#14434)

Schema Registry

  • PR-2790 - DGS-8088 Get the mode in scope rather that just for the subject

Version 7.1.9

Released September 2023

Community Features

Common

  • 59440c88 - APPSEC-2855: add generation of SBOMs to maven based Java projects
  • 27a3c1ca - APPSEC-2796: update guava

Kafka

  • PR-941 - KSECURITY-1789: upgrading requests to v2.31.0 (3.1)
  • PR-934 - KSECURIYT-1859: updating netty to 4.1.96 (3.1)

ksqlDB

  • d3f0e149 - fix: fix compile error introduced by merge conflict resolution

REST Proxy

Schema Registry

  • PR-2734 - DGS-8112 Allow properties to be omitted during serialization
  • 3f0e1198 - DGS-7927: update io.grpc_grpc-protobuf
  • PR-2695 - do not reinstantiate objectmapper as it’s expensive (#2686)

Version 7.1.8

Released June 2023

Community Features

Common

  • PR-530 - Backport log redactor to CP patches.
  • bd393b16 - AUDIT-1600: Updating logredactor version to 1.0.12
  • 22a9da57 - APPSEC-2551: Update Prometheus JMX Exporter
  • 5ad03d12 - APPSEC-2551: Update Jackson
  • PR-514 - Update snakeyaml
  • b4334b94 - pin jose4j version to match ce-kafka
  • 95ed5124 - include snappy and jetty in the dependency management to match versions with ce-kafka, cleanup jackson imports

Kafka

  • PR-13601 - KAFKA-14869: Bump coordinator value records to flexible versions (KIP-915, Part-2) (#13526) (#13601)
  • PR-13597 - KAFKA-14869: Ignore unknown record types for coordinators (KIP-915, P… (#13597)
  • c7278d91 - KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
  • PR-13499 - KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable (#13499)
  • PR-886 - CONFLUENT: fix 3.0 dependency loading for streams upgrade test
  • PR-13386 - KAFKA-14809 Fix logging conditional on WorkerSourceTask (#13386)
  • PR-13273 - KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
  • PR-912 - KSECURITY-1627 KSECURITY-1556 KSECURITY-1568: update snappy, jetty, jose

ksqlDB

  • PR-9810 - Bump minimist and mkdirp
  • 4ce2d5f3 - fix: Replace regex in CommandParser with a split by space
  • cc0501d8 - fix: Simplify regex in DdlDmlRequestValidators to avoid catastrophic backtracking
  • aefe6338 - Fix log4j-extensions
  • 08853cd3 - Fix log4j-extensions

REST Proxy

  • ab17aa29 - Fix the overzealous conflict resolution
  • 5be3406e - KREST-9942: Remove SnakeYaml from dependency management
  • a54dde7e - CNC-366: Update Jetty

Schema Registry

  • 0f13c16e - fix checkstyle
  • 20526ee5 - solve conflicts
  • e0626f64 - Update the test to make it clear to customer
  • dc6f2096 - update the doc
  • 33659feb - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
  • 3ffd8fc7 - Update schema-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaSchemaSerDeConfig.java
  • 1ca2a39d - Update the codes with checking string for http connect timeout
  • ffba37fe - * changed the config names
  • 4c383ee4 - Apply suggestions from code review
  • 7155f0fb - * checkstyle error
  • 7e2dec9c - * removing unused import
  • 9ab1b0ec - * adding defaults
  • 01c6d7a7 - * Made SR client http connect and read timeout configurable.
  • 89aa28be - * Made SR client http connect and read timeout configurable.
  • PR-2654 - DGS-7432 Fix ClassCastException when getting params option
  • PR-2647 - DGS-7422 Optimization: omit compat check during IMPORT
  • PR-2642 - DGS-7367 Propagate error messages even if not JSON
  • PR-2616 - DGS-7005 Convert socket timeouts to RetriableException for converters
  • f45fe28e - DGS-6698: Remove SnakeYaml from the dependencyManagement
  • PR-2602 - DGS-6929 Ignore invalid Avro defaults for source connectors
  • PR-2590 - DGS-6701 Fix getSchemaById to match subject if passed
  • ed4ec914 - DGS-6698: Remove SnakeYaml from the dependencyManagement
  • PR-2589 - DGS-6613 Ignore invalid Avro defaults in Avro Converter
  • PR-2588 - DGS-6603 Fix default conversions in JsonSchemaConverter
  • PR-2693 - DGS-7746 Fix perf regression due to DGS-6331
  • PR-2684 - DGS-7675 Fix NPE when print.schema.ids=true
  • 1bf961b7 - Update json-schema

Commercial Features

Confluent Server

  • Cherry-pick KAFKA-5756: Wait for concurrent source task offset flush to complete
  • KSECURITY-1568: Upgrading jose4j
  • KGLOBAL-2035: Unblock local follower fetch requests when records are appended to mirror leader logs
  • KDATA-853: Upgrading dependencies
  • KDATA-737: Fix incorrectly exposed ccloud only configs on documentation
  • KSECURITY-1556: updated jetty version
  • KAFKA-14887: FinalizedFeatureChangeListener should not shut down when
  • KSECURITY-1460, KSECURITY-1466: Updated snakeyaml, json-smart and nim
  • KSECURITY-1459: updated golang.org/x/net mod.
  • KSECURITY-1455: updated golang.org/x/text mod.
  • KSECURITY-1453: Updated version of golang.org/x/sys
  • KSECURITY-1467: Disabled colocated kraft mode test. (#9371)
  • CONFLUENT: fix 3.0 dependency loading for streams upgrade test
  • KGLOBAL-3350 Add more logging for persistent connection creation in Outbound connection manager
  • KSECURITY-1627: Updated snappy-java
  • KSECURITY-1618: update guava version
  • KAFKA-14887: FinalizedFeatureChangeListener should not shut down when ZK session expires
  • CPKAFKA-3853: Use Kafka protocol for centralized ACL update from brokers

Security

  • CIAM-2998 - Log redactor backport for confluent-security-plugins
  • KREST-10128 Create LicenseManager instance in iteration run instead o

Secret Registry

  • unpin netty, as rest-utils brings updated version

Control Center

  • [MMA-13078] revisit session caching in basic auth
  • [MMA-13089] Handle large message JSON on topic inspection
  • [MMA-13013] Add property to specify alias cert per component for SchemaRegistry, KSql & Connect
  • [MMA-13042] Fixing log statement for websocket time out
  • [MMA-13216] update jose4j
  • [MMA-13201] Csrf Enabled Flag not working

MQTT Proxy

  • CC-19108 | Rename isChannelClosed to isAuthenticated.
  • CC-19108 | Use an atomic reference to avoid publishing message when the channel is closed or marked for closure.

Metadata Service

  • CIAM-2936: update jose4j
  • update
  • update
  • Update pom.xml
  • CIAM-2986 - Older MDS builds are not running all the tests they should be
  • update comment, clean up whitespaces
  • another attempt with mina-core
  • pin mina core in cli, scope mina-core as test in main pom
  • pin mina-core in testingLdap only
  • SEC-xyz: Update mina-core

Replicator

  • disable builds of connect-replicator version 5.5.x
  • used spotbugs version variable instead
  • use spotbugs annotations instead of findbugs
  • resolving compile issue
  • KGLOBAL-3224 removed find bugs usage
  • remove aws-maven plugin to unblock the downstream validation for common: RCCA-10412, APPSEC-2551

Version 7.1.7

Released March 2023

Community Features

Common

  • PR-513 - Merge Conflict Resolution (from 7.0.x to 7.1.x)
  • PR-512 - Add snakeyaml to dependency management
  • 7bcaab9e - Fix typo in with method calls
  • 3868e27b - Fix typo
  • 783c5721 - Return the original “version_range” if it isn’t an actual Maven version range
  • e71293e4 - Update logredactor depdendency to 1.0.11
  • PR-502 - Fix: override maven version plugin’s default versionrange
  • 9350d231 - Update resolver-maven-plugin to 0.6.0
  • PR-497 - Upgrade Netty to 4.1.86.Final
  • PR-496 - Bump to 7.4.4

Kafka

  • PR-13245 - KAFKA-14704; Follower should truncate before incrementing high watermark (#13245)
  • PR-873 - KC-2332: Upgrade netty to 4.1.86
  • PR-857 - CONFLUENT: Fix filter for not publishing streams upgrade test artifacts
  • PR-853 - CONFLUENT: Skip publishing for kafka-streams-upgrade-system-tests
  • PR-862 - KAFKA-9648: Add configuration to adjust listen backlog size for Accep
  • PR-13000 - KAFKA-14496: Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler (#13000)
  • PR-12935 - KAFKA-14432: RocksDBStore relies on finalizers to not leak memory (#12935)
  • PR-12869 - KAFKA-14382: wait for current rebalance to complete before triggering followup (#12869)
  • PR-12809 - [KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)
  • PR-11690 - KAFKA-13599: Upgrade RocksDB to 6.27.3 (#11690)

ksqlDB

  • PR-9821 - Fix: migrations tool without basic auth works again
  • PR-9770 - Upgrade netty to 4.1.86 #2
  • PR-9717 - Fix: change password-based configs to Type.PASSWORD

REST Proxy

  • 69ed0cd9 - Remove incorrect synchronization
  • PR-1108 - Set kafka-schema-registry version in dependency mamanagement
  • 7a7f5dcd - Remove unnecessary ${project.version} tag
  • PR-1102 - Fix transitive dependency of kafka-schema-registry
  • PR-1101 - Fix dependency org.yaml_snakeyaml, upgrade to fixed

Schema Registry

  • PR-2577 - DGS-6373 Ignore leading dot when merging Protobuf custom options
  • PR-2575 - DGS-6331 Handle javaType for oneOfs during JSON deserialization
  • PR-2568 - DGS-6306 Allow multiple oneofs in Protobuf converter
  • PR-2550 - DGS-6192 Include default ctx when using subjectPrefix w/wildcard ctx
  • PR-2540 - DGS-6075 Add explicit definition of snakeyaml.
  • PR-2517 - MINOR cherry-pick Protobuf validate optimization to 7.0.x
  • PR-2519 - DGS-6014 cherry pick Maven plugin enhancement to 5.5.x
  • PR-2514 - DGS-5950: Allow optional map key in Avro
  • PR-2474 - Fix handling of Protobuf repeated options
  • PR-2471 - Preserve metadata associated with primitive types during normalization
  • PR-2468 - DGS-5567 Handle Protobuf map options in toCanonicalString()

Commercial Features

Confluent Server

  • Backport “Minor: add logging to inLock / inWriteLock to catch leaked locks (#8594)” to 7.1.x
  • KMETA-478 Enable zk audit logs by default
  • Fix version comparison in kafkatests (#8922)
  • KAFKA-14731: Upgrade ZooKeeper to 3.6.4 (#13273)
  • KSECURITY-1005: [7.1.x] Enforce kafka-client-plugins consumers to use the same snakeyaml version
  • cherrypick KAFKA-14417: Address incompatible error code returned by broker from`InitProducerId`
  • KSECURITY-897: Make expiration timeout for ZK ACL change notification path configurable
  • KENGINE-287: RPCProducerIdManager should not wait on new block
  • KC-2328: Update yaml.v2 to 2.2.4
  • KSECURITY-965: Update git version 1.13.0
  • KSECURITY-953: Update azure-identity to 1.7.3
  • KSECURITY-981: Add FeatureZNode zk node path to secure root paths
  • KC-2333: Update io.netty libraries to 4.1.86.Final
  • Audit log Trogodor performance test
  • KSTORAGE-82: Tiered storage topic deletion support
  • MINOR: disable use of ConfluentLeaderAndIsrRequest in 5.4.x
  • [KENGINE-314]: Fix a NPE when FetchSession close.
  • KSTORAGE-290: retry archiving for general S3 AmazonClientException
  • KAFKA-9038: [WIP] Allow creating partitions for topics partitions not in reassignment
  • CONFLUENT: Use single audit log provider for MDS and its hosting broker
  • MCM-364 Init container support for encoding host IP into advertised.listeners
  • MINOR: print topicIdPartition consistently

Security

  • Removed From Supported Operations to follow RBAC Rules
  • Added fix for unit test
  • Fixed import
  • Changed the design of RBAC for Exporters
  • Removed extra line changes
  • Added RBAC for Schema linking Phase 1
  • Remove logging statement
  • Update netty-codec-http to 4.1.86.Final

Secret Registry

  • Remove unused variables
  • Update RestService.java

Control Center

  • RCCA-9557 - fix broker flapping issue 7.0 - missed cherry-pick
  • Use docker image tag 7.1.6
  • Increase wait start time for control center in test
  • Make sure control center stop properly
  • Change PORT to avoid conflict with TelemetryReporterIntegrationTest
  • Change to make this compatible with 6.2.x
  • KC-2261 timeout websocket connection via jwt token
  • Update CODEOWNERS to match latest definition from master
  • MMA-13013: Override SSL Stores priority
  • Refactoring- made code more functional and improved readability.
  • Updated NotFoundException message in scenario where topicName provided returns 0 no consumergroupdata.
  • Updated code based on PR feedback.
  • Updated consumerGroupId API to also support topicName queryParam.
  • Improving test cases with scenarios of single consumergroup covering multiple topics. 2. updated consumerGroupId to not support topicName queryParam, it is not required.
  • Adding unit test for CachedConsumerOffsetsResource
  • Adding support for topicName in query param to return only consumer groups for that topic
  • [MMA-12439] Update Websocket configuration for Control-Center if BasePath supplied
  • Reading password using getPassword instead of getString, this fixes the casting exception
  • Adding a config to force armeria health check to use HTTP1
  • Fixing failing test case, due to merge issue
  • Fixing check style error, removing unused import
  • Renaming Health_check to health check
  • MMA-12911: Propagate numPartitions and replication factor from CreateTopicsResult
  • MMA-12912: Disable offset commits, don’t supply group ID
  • Adding config based functionality to force http1 for armeria health checks
  • MINOR: Update repo to use mvn-wrapper to speed up builds
  • Fix codeowners to make c3 default ownens as well
  • Cherry pick 7746 codewoner

MQTT Proxy

  • Remove the pin of netty version to use the (current) version defined in common

Metadata Service

  • Clean testng
  • [Fix build] Drop down to testNG 7.5 - which is the last version to support jdk8
  • Adapt 7.1.x to testNg 7
  • Clean up surefire
  • CIAM-2615 - Bump TestNG to 7.7.0
  • Fix notnull in 7.1.x
  • Fix NotNull import
  • Squashed ‘mk-include/’ content from commit 7df56b0fff
  • CIAM-2579: update pgsql
  • CIAM-2570: We need port CIAM-2183 fix to 7.1.x
  • CIAM-2570: Backport CIAM-2183 fix to CP in active 6.1.x
  • CIAM-2570: Backport CIAM-2183 fix to CP in active 6.0.x
  • CIAM-2577: Turned off test RbacExtractorMetricsTest
  • Fixed build error relate to FIPS jar for 7.0.x branch
  • Increase buffer size for Proxy Servlet

Replicator

  • Fixed deprecated method usage
  • Changed to 5 sec to honor TASK_SHUTDOWN_GRACEFUL_TIMEOUT_MS_CONFIG
  • Changed default to 120 sec to match topic config sync
  • KGLOBAL-3060 added consumer poll timeout ms property

Version 7.1.6

Released February 2023

Commercial Features

Security

Version 7.1.5

Released December 2022

Community Features

Common

  • PR-492 - APIF-3112: Upgrade jmx_prometheus_javaagent to 0.17.2.
  • f5764727 - Merge conflict resolution.
  • PR-489 - APIF-3117: Upgrade scala to 2.13.10.
  • PR-485 - APIF-2957: Upgrade Jackson and Protobuf versions.

Kafka

  • PR-845 - CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
  • PR-842 - DP-9030: Use the new withGradleFile closure
  • PR-832 - KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
  • PR-818 - KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2

ksqlDB

  • PR-9713 - Bump netty tcnative version to 2.0.54.Final
  • PR-9708 - APIF-3117: Unpin scala version.
  • PR-9697 - APIF-3122: Unpin Protobuf version.

REST Proxy

  • PR-1072 - KREST-7107 Shouldn’t provide overly detailed info or stack traces in responses -

Schema Registry

  • PR-2460 - DGS-5544 Support nulls when using optional fields in proto2
  • PR-2459 - DGS-5400 Support subjectPrefix containing wildcard context and subject
  • PR-2450 - Pin Snakeyaml version
  • PR-2443 - DGS-5423 Check subject when looking up ID across contexts
  • PR-2440 - Update the Snakeyaml dependency
  • 8e5ce168 - Unpin Protobuf version.
  • PR-2414 - DGS-5253 Handle cycles when resolving references
  • PR-2405 - DGS-4754 Handle empty record default at field level
  • PR-2399 - DGS-5084 Ignore compat check in IMPORT mode
  • PR-2395 - DGS-4971 Handle map types with enhanced.protobuf.schema.support

Commercial Features

Confluent Server

  • CONFLUENT: Skip publishing for projects with no scala suffix when the scala version is not the default
  • DP-9030 - Switch to codeartifact repo
  • Increase vagrant aws timeout for system test
  • Bouncycastle issue fix for connect service [7.1.x]
  • KSECURITY-792: Upgrade from Scala 2.13.8 to 2.13.10
  • KSECURITY-788: Upgrade jackson-databind version to 2.13.4.2
  • KSECURITY-832: Upgrade bc fips drivers
  • KSECURITY-789: Upgrade to snakeyaml version 1.32
  • CIAM-2424 - Name RestClient threads for stackdump identification (#7887)
  • KSECURITY-794: Upgrade to protobuf version 3.19.6
  • CIAM-2304: Add SecurityMetadata:Describe to SecurityAdmin
  • RCCA-8564: log a warning if LDAP login fails becuase of network issues

Security

  • Exclude bcpkix-jdk15on as compile dep and add as test dependency
  • Pin bc-nonfips dep as a test dependency
  • Pin bouncycastle fips dependency in confluent-security-plugins
  • Removed bcfips from common
  • excluded bcfips jars
  • AUTHN-1087: [7.0.x only] Enforce Netty versions in kafka-rest plugin
  • AUTHN-1087: Upgrade vertx to 3.9.14
  • AUTHN-1087: Upgrade vertx to 3.9.14
  • AUTHN-1087: Upgrade vertx to 3.9.14
  • AUTHN-1087: Upgrade vertx to 3.9.14
  • AUTHN-1087: Upgrade vertx to 3.9.14
  • AUTHN-1325: Update bc-fips version to 1.0.2.3
  • fixed checkstyle error
  • Requesting permission only for Subject Read operation instead of any of the subject operation in case of GET /schemas

Secret Registry

  • Exclude Non-FIPS dep and use bc-fips dependency

Control Center

  • MMA-9023: memory bounding for streams applications [5.4.x, master]
  • Revert “MMA-12206 upgrade io.netty:netty-codec-http”

MQTT Proxy

  • Fixing the tests and config validator, to incorporate new changes in validation of MqttDecoder brought in by netty upgrade
  • Upgrading netty

Metadata Service

  • MMA-12624 Fix testng dependencies
  • CIAM-2392 Bump PSQL Version
  • Remove spaces
  • Add back newline
  • Implement writeTransactionMarkers in DelegateConfluentAdmin
  • CIAM-2304: Add unit tests for SecurityAdmin

Replicator

  • KGLOBAL-2436: Don’t seek to beginning on pause for schema translator
  • REPL-2055-: Log WakeupException at debug level
  • RCCA-7678: Reverse proxy header check added

Version 7.1.4

Released August 2022

Community Features

Common

  • PR-472 - Upgrade Netty to 4.1.79.Final
  • PR-464 - :Migrate from confluent-log4j to reload4j [5.4.x – 7.0.x]
  • PR-465 - : Confluent log4j to reload4j [7.1.x Only]
  • PR-467 - : Upgrade JUNIT in 5.4.x only

Kafka

  • PR-793 - Upgrade Jetty to 9.4.48.v20220622
  • PR-763 - : migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (3.1)
  • 0f6420dd - : upgrading netty to v4.1.79.Final (3.1)

ksqlDB

  • PR-9337 - bugfix: remove log4j from the classpath (#9334)
  • PR-9334 - bugfix: remove log4j from the classpath

REST Proxy

Schema Registry

  • PR-2359 - DGS-4395 Fix message indexes of normalized Protobuf with map
  • PR-2351 - DGS-4358 Fix NPE in Protobuf converter for null map value
  • 0577dffb - Add comments
  • 042d3e3c - DGS-4303: Support different stat types in SchemaRegistryMetric
  • PR-2342 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
  • PR-2339 - APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.
  • PR-2336 - DGS-4249 Fix name resolution during Protobuf normalization

Commercial Features

Confluent Server

  • Upgrade com.squareup.okhttp3.okhttp to 4.9.3 (#7116)
  • KAFKA-14107: Upgrade Jetty version.
  • Migrating log4j12 to reload4j, slf4j-log4j12 to slf4j-reload4j.jar (7.1.x)
  • Avoid sending partial updates during LDAP group manager startup
  • Fix exception handling in RBAC writer coordinator (#6576)
  • Upgrading google-cloud-storage to v2.6.1 (7.1.x)
  • Upgrading aws-java-sdk-s3 to v1.12.268 (7.1.x)
  • Upgrading netty version to 4.1.79.Final (backport 7.1.x)
  • Update bouncy castle to 1.70 (back port 7.1.x)

Security

  • add auth token provider to ksql security extensions
  • revert pom
  • add auth token provider to ksql security extensions
  • AUTHN-1085: Fix NPE in debug log
  • APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.

Secret Registry

  • Migrate from log4j to reload4j for connect-plugin
  • CC-17641,CC-17644 Bump jetty version
  • APIF-2714: Migrate from confluent-log4j to slf4j-reload4j.

Control Center

  • MMA-12201 Fix the mistake in resolving merge conflict
  • MMA-12201 remove slf4j from transitive deps
  • MMA-12206 upgrade transitive dependency of netty
  • MMA-11617: allow for trailing slash in path
  • MMA-12206 upgrade io.netty:netty-codec-http
  • MMA-12201 Exclude slf4j in transitive dependency
  • MMA-12201: Use reload4j version properties from common
  • MMA-12201 Add logredactor [7.1.x]
  • MMA-8781: Respect SR Config and handle nulls in CachingSchemaRegistryClientSupplier
  • MMA-12201 Replace log4j with reload-4j
  • MMA-12127: Change type from String to Password for CONTROL_CENTER_EMAIL_PASSWORD

MQTT Proxy

  • CC-17620: remove dependency on slf4j-log4j12 and confluent-log4j

Metadata Service

  • CIAM-2164 - Migrate from confluent-log4j to reload4j for MDS

Replicator

  • Add log redactor.
  • Migrate confluent-log4j to reload4j.

Version 7.1.3

Released July 2022

Community Features

ksqlDB

  • PR-9260 - fix: Excludes Guava from Guava-retrying in order to manage Guava depe
  • 617df451 - merge conflicts
  • 4ef7a1df - merge conflicts

Schema Registry

  • PR-2331 - DGS-4172 Bound size of Avro datumReader/Writer caches
  • PR-2329 - DGS-4163 Pass topic to schema formatters
  • PR-2328 - DGS-4162 Handle refs at different levels in Protobuf converter
  • c4c0b7f7 - Fix merge issue
  • PR-2327 - DGS-1648 Allow Protobuf msg fullname to be passed for console producer
  • 5ea29980 - Fix test in Protobuf converter
  • PR-2321 - DGS-4016 Fix handling of UInt32Value in Protobuf converter
  • PR-2320 - Revert remove schema in exception

Commercial Features

Confluent Server

  • AUTHN-739: Oauth token refresh causes kafka client authentication to fail
  • KGLOBAL-1507: Filter out _schemas during cluster linking auto-mirroring
  • KSECURITY-429: Fix for test testWriterReelectionBeforeProduceComplete

Control Center

  • MMA-11938 Make connect healthcheck path configurable
  • Deprecate CONTROL_CENTER_STREAMS_RETRIES_CONFIG and remove default usage

Metadata Service

  • Update rbac-api-server/src/test/java/integration/rbacapi/api/v1/RbacClustersSmokeTest.java
  • CIAM-2141 - Remove usages of MiniKdcWithLdapService

Version 7.1.2

Released June 2022

Community Features

Common

  • Upgrading gson
  • PR-456 - DGS-3674 Upgrade gson to 2.8.6
  • PR-446 - Upgrade jackson related packages
  • PR-445 - Unify protocol buffer version.

Kafka

  • face4918 - Bump version to 3.1.1
  • f8061ae3 - Revert “MINOR: reload4j build dependency fixes (#12144)”
  • PR-708 - KSECURITY-220: Update jackson packages to 2.13.2
  • PR-11401 - KAFKA-13255: Use config.properties.exclude when mirroring topics (#11401)
  • PR-12096 - KAFKA-13794: Fix comparator of inflightBatchesBySequence in TransactionsManager (round 3) (#12096)
  • PR-12006 - KAFKA-13794: Follow up to fix producer batch comparator (#12006)
  • PR-11980 - fix: make sliding window works without grace period (#kafka-13739) (#11980)
  • PR-12001 - 3.1.1 release notes (#12001)
  • PR-11991 - KAFKA-13794; Fix comparator of inflightBatchesBySequence in TransactionManager (#11991)
  • PR-11995 - KAFKA-13782; Ensure correct partition added to txn after abort on full batch (#11995)
  • PR-11908 - KAFKA-13748: Do not include file stream connectors in Connect’s CLASSPATH and plugin.path by default (#11908)
  • PR-11962 - KAFKA-13775: Upgrade jackson-databind to 2.12.6.1 (#11962)
  • PR-11869 - KAFKA-13719: Fix connector restart cause duplicate tasks (#11869)
  • PR-11966 - KAFKA-13418: Support key updates with TLS 1.3 (#11966)
  • PR-11946 - KAFKA-13770: Restore compatibility with KafkaBasedLog using older Kafka brokers (#11946)
  • PR-11939 - KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)
  • PR-11933 - KAFKA-13759: Disable idempotence by default in producers instantiated by Connect (#11933)
  • PR-11931 - Fix LICENSE-binary (#11931)
  • PR-11922 - Update upgrade.html and quickstart.html for 3.1.1 (#11922)
  • PR-11912 - KAFKA-13752: Uuid compare using equals in java (#11912)
  • PR-11909 - KAFKA-13750; Client Compatability KafkaTest uses invalid idempotency configs (#11909)
  • PR-11891 - KAFKA-13727; Preserve txn markers after partial segment cleaning (#11891)
  • PR-11875 - KAFKA-13721: asymetric join-winodws should not emit spurious left/outer join results (#11875)
  • PR-11872 - KAFKA-12879: Remove extra sleep (#11872)
  • PR-11871 - KAFKA-12879: Addendum to reduce flakiness of tests (#11871)
  • PR-11797 - KAFKA-12879: Revert changes from KAFKA-12339 and instead add retry capability to KafkaBasedLog (#11797)
  • PR-11788 - :KAFKA-13673: disable idempotence when config conflicts (#11788)
  • PR-7082 - KAFKA-8659: fix SetSchemaMetadata failing on null value and schema (#7082)
  • PR-11656 - KAFKA-13579: Patch upgrade of netty/jetty/jackson (#11656)
  • PR-11745 - KAFKA-13661; Consistent permissions in KRaft for CreatePartitions API (#11745)
  • PR-11742 - KAFKA-13636: Fix for the group coordinator issue where the offsets are deleted for unstable groups (#11742)
  • PR-11631 - KAFKA-13563: clear FindCoordinatorFuture for non consumer group mode (#11631)
  • PR-11691 - KAFKA-13598: enable idempotence producer by default and validate the configs (#11691)
  • PR-11726 - KAFKA-13637: Use default.api.timeout.ms as default timeout value for KafkaConsumer.endOffsets (#11726)
  • PR-11689 - KAFKA-12841: Fix producer callback handling when partition is missing (#11689)
  • PR-11452 - KAFKA-13412; Ensure initTransactions() safe for retry after timeout (#11452)
  • PR-11671 - KAFKA-13388; Kafka Producer nodes stuck in CHECKING_API_VERSIONS (#11671)
  • 37edeed0 - Bump version to 3.1.0

ksqlDB

  • PR-8967 - fix: preserve old schema behavior for protobuf wrapped primitives (7.1.x)
  • PR-8968 - fix: preserve old schema behavior for protobuf wrapped primitives (7.0.x)

REST Proxy

  • PR-1026 - Backport consumer deadlock fix to 5.4.x
  • PR-1021 - KREST-6139 Deflake produce rate limiter test
  • b22efb66 - Patch gson to be 2.9.0
  • PR-1016 - KREST-5096 Fix NPE in Topics Resource when no key, or no value specified
  • PR-942 - Fix(#341): Remove deadlock when rebalancing by multiple consumer instances

Schema Registry

  • PR-2306 - DGS-3783 Allow same schema w/diff ids during IMPORT
  • PR-2299 - DGS-3141 - adding new v1/metadata/version endpoint
  • PR-2294 - Patch Gson to be 2.9.0
  • PR-2290 - DGS-3674 Upgrade to Protobuf 3.19.4
  • PR-2288 - DGS-3672 Allow CachedSchemaRegistryClient to register same schema
  • PR-2283 - DGS-3659 Handle null default in JSON Schema converter
  • PR-2282 - DGS-3654 Ensure isKey is passed in deserializers
  • 96471721 - Upgrade kotlin to 1.6.0
  • PR-2259 - Fix upgrade to Jackson 2.13
  • PR-2257 - DGS-3548 Optimize cache for JSON Schema converter
  • PR-2253 - DGS-3535: Remove enforcement of building with JDK8
  • PR-2241 - DGS-3484 Allow NaN when parsing JSON
  • PR-2223 - DGS-3342 Add reserved fields for Protobuf enums
  • PR-2227 - DGS-3417 Fix default value conversions in JsonSchemaConverter
  • PR-2318 - DGS-3974 Fix StackOverflow with recursive JSON schema in converter

Commercial Features

Confluent Server

  • Revert “CIAM-1419: Restructure DefaultAuthCache to lookup access rules for principal (#4758)”
  • Revert “CONFLUENT: Rename DataPlaneAuthCache class name to CloudAuthCache (#4885)”
  • KSECURITY-354: Fix failing Authorization related tests
  • build: kafka system tests should support larger EBS volumes for newer instances (#6371)
  • Upgrade Gson
  • CONFLUENT: Convert more modules to JUnit 5 and fix related build issues
  • Revert “KAFKA-13660: Switch log4j12 to reload4j (#11743)”
  • Revert “MINOR: reload4j build dependency fixes (#12144)”
  • KGLOBAL-1108: Fix OffsetsForLeaderEpoch version for cluster link source version < 2.8
  • Added 7.0.1 CP URL in dockerfile
  • Disable idempotent producer by default in Connect centralized licenses (#6193)
  • Fix backward incompatibility issue in license store (#6152)
  • KSECURITY-221: Update jackson packages to 2.13.2 version
  • KSTORAGE-228: optimize replica fetcher thread tier state transitions
  • KSECURITY-195: Update org.jetbrains.kotlin packages to 1.6.0
  • KDATA-399: back port fix for flaky test in ObjectStorePoolImplTest for 7.1.x
  • KSECURITY-198: Update protobuf-java packages to 3.19.4
  • Fix broken tests because of KAFKALESS-1032 cherry-pick
  • KAFKALESS-900: Fix replication factor in BrokerFailureDetectorTest and use monotonic time
  • KAFKALESS-1032: Use wall clock to prune task history
  • KGLOBAL-1234: Disable the link coordinator change tests in 7.1.x
  • KSTORAGE-2007: fix bad conflict resolution in backport
  • Revert “KAFKA-13598: enable idempotence producer by default and validate the configs (#11691)”
  • Revert “:KAFKA-13673: disable idempotence when config conflicts (#11788)”
  • METRICS-513 Log event log
  • Bump version to 3.1.0
  • RCCA-6743: Set hostname verifier to accept everything when identification algorithm is empty

Security

  • Added exclusion since MDC class is failing because of reload4j

Control Center

  • Bump maven-shade-plugin to 3.3.0
  • Fix build break
  • Override transitive dependency for analytics library of kotlin-stdlib-common
  • Add exclusion to override dependency kotlin-stdlib-jdk8
  • LOG_DIR to match rest of confluent components
  • (MINOR): Revert change that commented testArmeriaServiceHealthCheck
  • MMA-11694: Add retry logic for ControlCenterIntegrationTest

Metadata Service

  • Revert “MINOR: Use CloudAuthCache in DBAuthCache (#758)”
  • CIAM-2020 - Fix null string in MDSLoggingFilter
  • CIAM-2019 - Fix 6.x MDS builds from Jackson update
  • MDS-38 MDS-39 - Make MDSLoggingFilter case-insensitive
  • MDS-37 MDS-39 - Improve redaction rule for auth header
  • MINOR SEC-3087 SEC-3028 Avoid NPE in integration test
  • This release version includes a fix to prevent stack trace exposure from getting surfaced in HTTP responses for malformed requests.

Version 7.1.1

Released April 2022

Commercial Features

Metadata Service

  • MDS-37 MDS-39 - Improve redaction rule for auth header
  • MDS-38 MDS-39 - Make MDSLoggingFilter case-insensitive

Version 7.1.0

Released March 2022

Community Features

Common

  • PR-440 - AUDIT-971: Update confluent-log4j version for 7.1.x
  • PR-428 - Upgrade Netty
  • 8333c7ad - use 3.17.3 protobuf for version after 7.0.x
  • 605b9b9d - use 3.17.0 protobuf for version after 5.5.x
  • PR-422 - MMA-11603: Add centralized protobuf version from 5.3.x onwards
  • PR-421 - Update confluent-log4j version
  • PR-418 - [SEC-2896] Update confluent-log4j version to cp8
  • PR-413 - Remove redundant Netty Version definition.
  • PR-416 - Upgrade Avro to 1.11.0
  • PR-415 - Upgrade Avro to 1.11.0
  • PR-414 - Upgrade Avro to 1.11.0
  • PR-412 - CVE-2021-43797: Upgrade Netty
  • PR-411 - Upgrade Log4j2 to 2.17.1
  • PR-406 - Bump confluent-log4j to 1.2.17-cp6
  • PR-403 - Upgrade Log4j2
  • PR-391 - Sync Netty Override From 6.2.x
  • PR-384 - Update confluent-log4j version to 1.2.17-cp4
  • PR-383 - Upgrade Netty to 4.1.68.Final
  • PR-379 - [AUDIT-415] Update confluent-log4j version
  • PR-375 - Add Protobuf version to common pom
  • PR-370 - fix change in checkstyle behavior since 8.20
  • PR-364 - update checkstyle to 8.44 and fix checks
  • PR-363 - update maven dependency plugin to 3.2.0
  • PR-355 - update easymock for Java 16 support
  • PR-354 - update jacoco to 0.8.7 for Java 16 support
  • PR-356 - update mockito to 3.11.2
  • PR-353 - update spotbugs for recent Java version support

Kafka

  • PR-652 - KC-1933; Upgrade confluent-log4j to 1.2.17-cp8
  • PR-11664 - KAFKA-13582: TestVerifiableProducer.test_multiple_kraft_security_protocols fails (#11664)
  • PR-11607 - KAFKA-13544: fix FinalizedFeatureChangeListener deadlock (#11607)
  • PR-11552 - KAFKA-13488: Producer fails to recover if topic gets deleted midway (#11552)
  • PR-11503 - KAFKA-13456; Tighten KRaft listener config checks/constraints (#11503)
  • f05a1797 - KAFKA-13512: Avoid duplicating maps in ZkMetadataCache topic accessors
  • f068a791 - KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode #11416
  • PR-11563 - KAFKA-13461: Don’t re-initialize ZK client session after auth failure if connection still alive (#11563)
  • PR-11524 - KAFKA-13469: Block for in-flight record delivery before end-of-life source task offset commit (#11524)
  • PR-11526 - KAFKA-13472: Correct last-committed offsets tracking for sink tasks after partial revocation (#11526)
  • PR-11538 - KAFKA-10712; Update release scripts to Python3 (#11538)
  • PR-11431 - KAFKA-13397: MirrorMaker should not mirror topics ending with .internal (#11431)
  • PR-11459 - KAFKA-13394: Topic IDs should be removed from PartitionFetchState if they are no longer sent by the controller (#11459)
  • PR-11490 - KAFKA-13439: Deprecate eager rebalance protocol in kafka stream (#11490)
  • PR-11004 - KAFKA-12257; Consumer mishandles topics deleted and recreated with the same name (#11004)
  • PR-11484 - KAFKA-13443: Kafka broker exits when OAuth enabled and certain configuration not specified (#11484)
  • PR-11486 - KAFKA-13444: Fix OAuthCompatibilityTool help and add SSL options (#11486)
  • PR-11502 - KAFKA-13071; Deprecate support for changing acls through the authorizer (#11502)
  • 3c2dab10 - Revert “MINOR: Remove redundant argument from TaskMetricsGroup#recordCommit (#9642)”
  • PR-11439 - KAFKA-13406: skip assignment validation for built-in cooperativeStickyAssignor (#11439)
  • PR-11448 - KAFKA-13417; Ensure dynamic reconfigurations set old config properly (#11448)
  • PR-11323 - KAFKA-12226: Commit source task offsets without blocking on batch delivery (#11323)
  • PR-11489 - KAFKA-13446: Remove JWT access token from logs (#11489)
  • PR-11331 - KAFKA-13111: Re-evaluate Fetch Sessions when using topic IDs (#11331)
  • PR-10563 - KAFKA-12487: Add support for cooperative consumer protocol with sink connectors (#10563)
  • PR-11429 - KAFKA-13396: Allow create topic without partition/replicaFactor (#11429)
  • PR-11463 - KAFKA-13430: Remove broker-wide quota properties from the documentation (#11463)
  • PR-11368 - KAFKA-13261: Add support for custom partitioners in foreign key joins (#11368)
  • PR-11417 - KAFKA-13340: Change ZooKeeperTestHarness to QuorumTestHarness (#11417)
  • PR-11284 - KAFKA-13202: KIP-768: Extend SASL/OAUTHBEARER with Support for OIDC (#11284)
  • PR-11385 - KAFKA-13385: In the KRPC request header, translate null clientID to empty (#11385)
  • PR-10593 - KAFKA-10800; Enhance the test for validation when the state machine creates a snapshot (#10593)
  • PR-11426 - KAFKA-13391: don’t fsync directory on Windows OS (#11426)
  • PR-10772 - KAFKA-12697: Add FencedBrokerCount and ActiveBrokerCount metrics to the QuorumController (#10772)
  • PR-11379 - kafka-12994: Migrated SlidingWindowsTest to new API (#11379)
  • PR-11405 - KAFKA-12648: Wrap all exceptions thrown to handler as StreamsException & add TaskId field (#11405)
  • PR-11356 - KAFKA-10539: Convert KStreamImpl joins to new PAPI (#11356)
  • PR-11394 - KAFKA-13284: Use sftp to upload artifacts in release.py (#11394)
  • PR-11397 - KAFKA-13377: Close Stream to avoid resource leak (#11397)
  • PR-11380 - KAFKA-13345: Use “delete” cleanup policy for windowed stores if duplicates are enabled (#11380)
  • PR-11234 - KAFKA-13212: add support infinite query for session store (#11234)
  • PR-11188 - KAFKA-13021: disallow grace called after grace set via new API (#11188)
  • PR-11362 - KAFKA-13319: Do not commit empty offsets on producer (#11362)
  • PR-11058 - KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker restarts. (#11058)
  • PR-11220 - KAFKA-10777: Add additional configuration to control MirrorMaker 2 internal topics naming convention - KIP-690 (#11220)
  • PR-11311 - KAFKA-13280: Avoid O(N) behavior in KRaftMetadataCache#topicNamesToIds (#11311)
  • PR-11355 - KAFKA-13298: Improve documentation on EOS KStream requirements (#11355)
  • PR-11315 - KAFKA-10540: Migrate KStream aggregate operations (#11315)
  • PR-11337 - KAFKA-13309: fix InMemorySessionStore#fetch/backwardFetch order issue (#11337)
  • PR-11076 - KAFKA-12486: Enforce Rebalance when a TaskCorruptedException is throw… (#11076)
  • PR-11215 - KAFKA-12994: Migrate TimeWindowsTest to new API (#11215)
  • PR-11352 - Add DirectoryConfigProvider to the service provider list (#11352)
  • PR-11361 - KAFKA-13324: KRaft: fix validateOnly in CreateTopics (#11361)
  • PR-11327 - KAFKA-13305: fix NullPointerException in LogCleanerManager “uncleanable-bytes” gauge (#11327)
  • PR-11347 - KAFKA-13296: warn if previous assignment has duplicate partitions (#11347)
  • PR-11170 - KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path (#11170)
  • PR-11351 - KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown (#11351)
  • d3cebd79 - DGS-2161: Update netty version to 4.1.65.Final
  • PR-11310 - KAFKA-13279: allow CreateTopicsPolicy, AlterConfigsPolicy in KRaft mode (#11310)
  • PR-11316 - KAFKA-10544: Migrate KTable aggregate and reduce (#11316)
  • PR-11227 - KAFKA-13211: add support for infinite range query for WindowStore (#11227)
  • PR-11334 - KAFKA-13246: StoreQueryIntegrationTest#shouldQueryStoresAfterAddingAndRemovingStreamThread now waits for the client state to go to REBALANCING/RUNNING after adding/removing a thread and waits for state RUNNING before querying the state store. (#11334)
  • PR-11344 - KAFKA-13312; ‘NetworkDegradeTest#test_rate’ should wait until iperf server is listening (#11344)
  • PR-11289 - KAFKA-13254; Fix deadlock when AlterIsr response returns (#11289)
  • PR-11329 - KAFKA-13301 Config documentation optimized for ‘request.timeout. ms’ and ‘max.poll.interval.ms’. (#11329)
  • PR-10649 - KAFKA-12762: Use connection timeout when polling the network for new connections (#10649)
  • PR-11324 - KAFKA-13294: Upgrade Netty to 4.1.68 (#11324)
  • PR-11252 - KAFKA-13216: Use a KV with list serde for the shared store (#11252)
  • PR-10914 - [KAFKA-8522] Streamline tombstone and transaction marker removal (#10914)
  • PR-11186 - KAFKA-13162: Ensure ElectLeaders is properly handled in KRaft (#11186)
  • PR-11080 - KAFKA-13149; Fix NPE when handling malformed record data in produce requests (#11080)
  • PR-11292 - KAFKA-13264: fix inMemoryWindowStore backward fetch not in reversed order (#11292)
  • PR-11283 - KAFKA-13249: Always update changelog offsets before writing the checkpoint file (#11283)
  • PR-11312 - KAFKA-13224: Ensure that broker.id is set in KafkaConfig#originals (#11312)
  • PR-11317 - KAFKA-13287: Upgrade RocksDB to 6.22.1.1 (#11317)
  • PR-11319 - KAFKA-13288; Include internal topics when searching hanging transactions (#11319)
  • PR-11033 - KAFKA-12988 Asynchronous API support for RemoteLogMetadataManager add/update methods. (#11033)
  • PR-11287 - KAFKA-13256: Fix NPE in ConfigDef when documentation is null (#11287)
  • PR-11301 - KAFKA-13276: Prefer KafkaFuture in admin Result constructors (#11301)
  • PR-11302 - KAFKA-13243: KIP-773 Differentiate metric latency measured in ms and ns (#11302)
  • PR-11273 - KAFKA-13237; Add ActiveBrokerCount and FencedBrokerCount metrics to the ZK controller (KIP-748) (#11273)
  • PR-11294 - KAFKA-13266; InitialFetchState should be created after partition is removed from the fetchers (#11294)
  • PR-11250 - KAFKA-12766 - Disabling WAL-related Options in RocksDB (#11250)
  • PR-11288 - KAFKA-13258/13259/13260: Fix error response generation (#11288)
  • PR-11300 - KAFKA-13258/13259/13260: Fix error response generation (#11300)
  • PR-11308 - KAFKA-13277; Fix size calculation for tagged string fields in message generator (#11308)
  • PR-11214 - KAFKA-12994 Migrate JoinWindowsTest and SessionWindowsTest to new API (#11214)
  • PR-11213 - KAFKA-13201: Convert KTable suppress to new PAPI (#11213)
  • PR-11297 - KAFKA-10038: Supports default client.id for ConsoleConsumer, ProducerPerformance, ConsumerPerformance (#11297)
  • PR-11307 - KAFKA-13262: Remove final from MockConsumer.close() and delegate implementation (#11307)
  • PR-11074 - KAFKA-13101: Replace EasyMock and PowerMock with Mockito for RestServerTest (#11074)
  • PR-11295 - KAFKA-13270: Set JUTE_MAXBUFFER to 4 MB by default (#11295)
  • PR-11296 - KAFKA-13273: Add support for Java 17 (#11296)
  • PR-11027 - KAFKA-13066: Replace EasyMock with Mockito for FileStreamSinkConnectorTest (#11027)
  • PR-11051 - KAFKA-13088: Replace EasyMock with Mockito for ForwardingDisabledProcessorContextTest (#11051)
  • PR-11241 - KAFKA-13032: add NPE checker for KeyValueMapper (#11241)
  • PR-11086 - KAFKA-13103: add REBALANCE_IN_PROGRESS error as retriable error for AlterConsumerGroupOffsetsHandler (#11086)
  • PR-11255 - KAFKA-13225: Controller skips sending UpdateMetadataRequest when no change in partition state. (#11255)
  • PR-11174 - KAFKA-9747: Creating connect reconfiguration URL safely (#11174)
  • PR-11226 - KAFKA-13175; Optimization TopicExistsException,When a topic is marked for deletion. (#11226)
  • PR-11228 - KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)
  • PR-11267 - KAFKA-13234; Transaction system test should clear URPs after broker restarts (#11267)
  • PR-11149 - KAFKA-13229: add total blocked time metric to streams (KIP-761) (#11149)
  • PR-9769 - KAFKA-10774; Admin API for Describe topic using topic IDs (#9769)
  • PR-11262 - KAFKA-12963: Add processor name to error (#11262)
  • PR-11104 - KAFKA-13079: Forgotten Topics in Fetch Requests may incorrectly use topic IDs (#11104)
  • PR-11266 - KAFKA-13233 Log zkVersion in more places (#11266)
  • PR-11264 - KAFKA-13231; TransactionalMessageCopier.start_node should wait until the process if fully started (#11264)
  • PR-11164 - KAFKA-13155; Fix concurrent modification in consumer shutdown (#11164)
  • PR-11231 - KAFKA-13214; Consumer should not reset state after retriable error in rebalance (#11231)
  • PR-11127 - KAFKA-13134: Give up group metadata lock before sending heartbeat response (#11127)
  • PR-11230 - KAFKA-12840; Removing compact cleaning on a topic should abort on-going compactions (#11230)
  • PR-11239 - KAFKA-13219: BrokerState metric not working for KRaft clusters (#11239)
  • PR-11245 - KAFKA-13091; Ensure high watermark incremented after AlterIsr returns (#11245)
  • PR-11233 - HOTIFX: Disable spurious left/outer stream-stream join fix (#11233)
  • PR-11217 - KAFKA-13204: assignor name conflict check (#11217)
  • PR-11083 - KAFKA-13010: Retry getting tasks incase of rebalance for TaskMetadata tests (#11083)
  • PR-11184 - KAFKA-13172: Add downgrade guidance note for 3.0 (#11184)
  • PR-11216 - KAFKA-13198: Stop replicas when reassigned (#11216)
  • PR-11221 - KAFKA-13207: Skip truncation on fetch response with diverging epoch if partition removed from fetcher (#11221)
  • 17e2b249 - KAFKA-13209: Upgrade jetty-server to fix CVE-2021-34429
  • b923ec23 - KAFKA-13209: Upgrade jetty-server to fix CVE-2021-34429
  • PR-11199 - KAFKA-13194: bound cleaning by both LSO and HWM when firstUnstableOffsetMetadata is None (#11199)
  • PR-11154 - KAFKA-13068: Rename Log to UnifiedLog (#11154)
  • PR-11191 - KAFKA-13173; Ensure KRaft controller handles concurrent broker expirations correctly (#11191)
  • PR-11189 - KAFKA-13161; Update replica partition state and replica fetcher state on follower update (#11189)
  • PR-11194 - KAFKA-12779: rename namedTopology in TaskId to topologyName (#11194)
  • 9565a529 - KAFKA-12779: rename namedTopology in TaskId to topologyName #11192
  • PR-11179 - KAFKA-13165: Validate KRaft node id, process role and quorum voters (#11179)
  • PR-10784 - KAFKA-12862: Update Scala fmt library and apply fixes (#10784)
  • PR-11171 - KAFKA-13132; Upgrading to topic IDs in LISR requests has gaps introduced in 3.0 (part 2) (#11171)
  • PR-10788 - KAFKA-12648: Pt. 3 - addNamedTopology API (#10788)
  • PR-11156 - KAFKA-13046: add test coverage for AbstractStickyAssignorTest (#11156)
  • PR-11177 - KAFKA-13167; KRaft broker should send heartbeat immediately after starting controlled shutdown (#11177)
  • PR-11178 - KAFKA-13168: KRaft observers should not have a replica id (#11178)
  • PR-11168 - KAFKA-13160: Fix bug in BrokerMetadataPublisher handling of default resources (#11168)
  • PR-10602 - KAFKA-12724: Add 2.8.0 to system tests and streams upgrade tests. (#10602)
  • PR-11166 - KAFKA-13159 Enable additional transaction system tests in KRaft (#11166)
  • PR-11165 - Enable transactions_test.py system test in kraft mode (#11165)
  • PR-10074 - KAFKA-12305: Fix Flatten SMT for array types (#10074)
  • PR-10909 - KAFKA-12158; Better return type of RaftClient.scheduleAppend (#10909)
  • PR-11050 - KQE-289: Replace SerDe with Serde (#11050)
  • PR-11160 - update batch.size doc (#11160)
  • PR-11116 - KAFKA-13114; Revert state and reregister raft listener (#11116)
  • PR-11145 - KAFKA-13151; Disallow policy configs in KRaft since they are not yet supported (#11145)
  • PR-11146 - KAFKA-12648: minor followup from Pt. 2 and some new tests (#11146)
  • PR-11120 - Add support for infinite endpoints for range queries (#11120)
  • PR-11126 - KAFKA-13132; Ensure topicId is updated on replicas even when the leader epoch is unchanged (#11126)
  • PR-11131 - KAFKA-13137; KRaft Controller Metric MBean names incorrectly quoted (#11131)
  • PR-11143 - close TopologyTestDriver to release resources (#11143)
  • PR-10693 - KAFKA-12625: Fix the NOTICE file (#10693)
  • PR-11135 - KAFKA-13143; Remove Metadata handling from ControllerApis (#11135)
  • PR-10915 - KAFKA-13041: Enable connecting VS Code remote debugger (#10915)
  • PR-11099 - KAFKA-10542: Migrate KTable mapValues, passthrough, and source to new Processor API (#11099)
  • PR-11136 - KAFKA-13141; Skip follower fetch offset update in leader if diverging epoch is present (#11136)
  • PR-11098 - KAFKA-13099; Transactional expiration should account for max batch size (#11098)
  • PR-10683 - KAFKA-12648: Pt. 2 - Introduce TopologyMetadata to wrap InternalTopologyBuilders of named topologies (#10683)
  • PR-11134 - KAFKA-12851: Fix Raft partition simulation (#11134)
  • PR-11132 - KAFKA-13139: Empty response after requesting to restart a connector without the tasks results in NPE (#11132)
  • PR-11107 - KAFKA-13125: close KeyValueIterator instances in internals tests (part 2) (#11107)
  • PR-11106 - KAFKA-13124: close KeyValueIterator instance in internals tests (part 1) (#11106)
  • PR-11105 - KAFKA-13123: close KeyValueIterator instances in example code and tests (#11105)
  • PR-11002 - KAFKA-13026: Idempotent producer (KAFKA-10619) follow-up testings (#11002)
  • PR-11109 - KAFKA-13113; Support unregistering Raft listeners (#11109)
  • PR-11057 - KAFKA-13008: Try to refresh end offset when partitionLag returns empty (#11057)
  • PR-11053 - KAFKA-13015: Ducktape System Tests for Metadata Snapshots (#11053)
  • PR-11111 - KAFKA-13126: guard against overflow when computing joinGroupTimeoutMs (#11111)
  • PR-11113 - KAFKA-13128: wait for all keys to be fully processed in #shouldQueryStoresAfterAddingAndRemovingStreamThread (#11113)
  • PR-11118 - KAFKA-13127; Fix stray topic partition deletion for kraft (#11118)
  • PR-10196 - KAFKA-9203: Check for buggy LZ4 libraries and remove corresponding workarounds (#10196)
  • PR-11108 - KAFKA-13116: Fix message_format_change_test and compatibility_test_new_broker_test failures (#11108)
  • PR-11115 - KAFKA-13129: replace describe topic via zk with describe users (#11115)
  • PR-11097 - KAFKA-8529; Flakey test ConsumerBounceTest#testCloseDuringRebalance (#11097)
  • PR-10921 - KAFKA-13096: Ensure queryable store providers is up to date after adding stream thread (#10921)
  • b894e9ec - KAFKA-13104: Controller should notify raft client when it resigns #11082
  • 56ef9103 - KAFKA-13104: Controller should notify raft client when it resigns #11082
  • PR-8844 - KAFKA-9887 fix failed task or connector count on startup failure (#8844)
  • PR-11045 - KAFKA-13082: Replace EasyMock with Mockito for ProcessorContextTest (#11045)
  • PR-11084 - KAFKA-13100: Create KRaft controller snapshot during promotion (#11084)
  • PR-10811 - KAFKA-12598: ConfigCommand should only support communication via ZooKeeper for a reduced set of cases (#10811)
  • PR-11078 - KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724) (#11036) (#11078)
  • PR-10579 - KAFKA-9555 Added default RLMM implementation based on internal topic storage. (#10579)
  • PR-11016 - KAFKA-13058; AlterConsumerGroupOffsetsHandler does not handle partition errors correctly. (#11016)
  • PR-11036 - KAFKA-12944: Assume message format version is 3.0 when inter-broker protocol is 3.0 or higher (KIP-724) (#11036)
  • 637c0b0a - KAFKA-13090: Improve kraft snapshot integration test
  • b5cb02b2 - KAFKA-13090: Improve kraft snapshot integration test
  • d375c309 - KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • 06151721 - KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • PR-11071 - KAFKA-13098: Fix NoSuchFileException during snapshot recovery (#11071)
  • PR-10945 - KAFKA-13017: Remove excessive logging for sink task deserialization errors (#10945)
  • PR-11026 - KAFKA-13064: Make ListConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11026)
  • PR-11065 - Fix perf regression on LISR requests by asynchronously flushing the partition.metadata file (#11056) (#11065)
  • PR-10952 - KAFKA-12257; Consumer mishandles topics deleted and recreated with the same name (#10952)
  • PR-11059 - KAFKA-12930,KAFKA-12929: Deprecate Java 8 and Scala 2.12 (#11059)
  • PR-11056 - Fix perf regression on LISR requests by asynchronously flushing the partition.metadata file (#11056)
  • PR-10753 - KAFKA-12803: Support reassigning partitions when in KRaft mode (#10753)
  • PR-10941 - KAFKA-10847: Remove internal config for enabling the fix (#10941)
  • PR-11035 - KAFKA-13072: Make RemoveMembersFromConsumerGroupHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11035)
  • PR-11021 - KAFKA-13062: Make DeleteConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11021)
  • PR-11022 - KAFKA-13063: Make DescribeConsumerGroupsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11022)
  • PR-11019 - KAFKA-13059: Make DeleteConsumerGroupOffsetsHandler unmap for COORDINATOR_NOT_AVAILABLE error (#11019)
  • PR-10877 - KAFKA-12925: adding presfixScan operation for missed implementations (#10877)
  • 2abe6851 - KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • 301a07f4 - KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • PR-8259 - KAFKA-7421: Ensure Connect’s PluginClassLoader and DelegatingClassLoader are truly parallel capable and resolve deadlock occurrences (#8259)
  • PR-11007 - KAFKA-10589; Rename kafka-replica-verification CLI command line arguments for KIP-629 (#11007)
  • PR-11008 - KAFKA-10588; Rename kafka-console-consumer CLI command line arguments for KIP-629 (#11008)
  • PR-11006 - KAFKA-13049: Name the threads used for log recovery (#11006)
  • 0707a8d3 - KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • 5e5d5bff - KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • PR-10985 - KAFKA-12984: make AbstractStickyAssignor resilient to invalid input, utilize generation in cooperative, and fix assignment bug (#10985)
  • PR-10280 - KAFKA-12554: Refactor Log layer (#10280)
  • PR-11032 - KAFKA-13073: Fix MockLog snapshot implementation (#11032)
  • PR-11031 - KAFKA-13067 Add internal config to lower the metadata log segment size (#11031)
  • PR-11040 - KAFKA-13078: Fix a bug where we were closing the RawSnapshotWriter to early (#11040)
  • PR-11041 - KAFKA-13080: Direct fetch snapshot request to kraft controller (#11041)
  • PR-11034 - KAFKA-13075: Consolidate RocksDBStoreTest and RocksDBKeyValueStoreTest (#11034)
  • PR-10986 - KAFKA-12983: reset needsJoinPrepare flag before rejoining the group (#10986)
  • 72159ea8 - Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
  • da63f3db - Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
  • PR-11011 - KAFKA-13051; Require principal builders implement KafkaPrincipalSerde and set default (#11011)
  • PR-11003 - KAFKA-12360: Document new time semantics (#11003)
  • PR-10935 - KAFKA-13003: In kraft mode also advertise configured advertised port instead of socket port (#10935)
  • PR-10794 - KAFKA-12677: Return not_controller error in envelope response itself in KRaft mode (#10794)
  • PR-11013 - KAFKA-13056; Do not rely on broker for snapshots if controller is co-resident (#11013)
  • PR-11012 - KAFKA-13057; Add KRaft “broker” to several RPC’s listeners (#11012)
  • PR-11000 - KAFKA-13045: Adding a test for batched offsetFetch requests with one group repeating (#11000)
  • PR-9541 - KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message (#9541)
  • PR-11010 - KAFKA-13053; Bump kraft frame version for incompatible changes from 2.8 (#11010)
  • PR-10560 - KAFKA-12660; Do not update offset commit sensor after append failure (#10560)
  • PR-10903 - KAFKA-13023: make “range, cooperative-sticky” as the default assignor in V3.0 (#10903)
  • PR-9944 - KAFKA-10580: Add topic ID support to Fetch request (#9944)
  • PR-8571 - KAFKA-7613: Enable -Xlint:rawtypes for connect, fixing warnings (#8571)
  • PR-10962 - KAFKA-12234: Implement request/response for offsetFetch batching (KIP-709) (#10962)
  • PR-10931 - KAFKA-12998: Implement broker-side KRaft snapshots (#10931)
  • PR-10981 - Bump trunk to 3.1.0-SNAPSHOT (#10981)
  • 258ed5a7 - KAFKA-12155: Metadata log and snapshot cleaning #10864
  • 284ec262 - KAFKA-12155: Metadata log and snapshot cleaning #10864
  • PR-10975 - KAFKA-13035 updated documentation for connector restart REST API to … (#10975)

ksqlDB

  • PR-8830 - fix: Set the sslFactory properly for the SR REST client.
  • PR-8611 - feat: un-synchronize PullQueryQueue row queuing when limit absent (#8563)
  • PR-8610 - fix: make adding to pull query queue threadsafe (#8544)
  • PR-8599 - fix: back out post-3.1 changes to fix 7.1.x build
  • PR-8553 - feat: add support for connect specific https configs
  • PR-8551 - fix: SandboxKafkaTopicClient should use default replication factor if applicable
  • PR-8523 - feat: Drop query from assignor
  • PR-8520 - fix: Consider topics created by join operations internal
  • PR-8532 - feat: don’t book keep sources in runtime as well
  • PR-8449 - feat: Adds support to /query-stream for http1.1 and StreamedRow json format
  • PR-8524 - fix: Restore runtime assignor
  • PR-8529 - fix: only use uuid as alter system command id
  • PR-8525 - feat: add a config to control the number of rutimes
  • PR-8511 - fix: reject incorrectly typed HEADER columns
  • PR-8472 - feat: new runtime selection
  • PR-8480 - feat: Allow to plug-in custom error handling for Connect server errors
  • PR-8510 - fix: refactoring properties replacement before restarting runtime
  • PR-8357 - feat: support custom headers for connector requests
  • PR-8516 - feat: enable headers for source tables
  • PR-8518 - fix: Fix some typos
  • PR-8509 - fix: invoke per-query listeners upon state change in shared runtime
  • PR-8493 - fix: close all runtimes in sandbox
  • PR-8492 - feat: throw exception if feature flag is off
  • PR-8490 - fix: make HEADER and HEADERS non-reserved
  • PR-8506 - fix: disallow persistent query limit clause
  • PR-8496 - feat: enable pull queries for header columns
  • PR-8507 - fix: Prevents internal http2 requests from having shared connections closed
  • PR-8498 - refactor: eliminate static metrics registry
  • PR-8476 - feat: custom auth configs for ksql connector requests
  • PR-8481 - feat: restart runtime after alter system query
  • PR-8497 - fix: explicitly set confluent log4j version to 1.2.17-cp5
  • PR-8491 - fix: disallow negative integers in pull query limit clause
  • PR-8342 - feat: Continuation tokens and ALOS for SPQs
  • PR-8462 - fix: 404 for /topics connect endpoint logs a warn instead of showing up in CLI
  • PR-8445 - feat: Validate connector config before creating it
  • PR-8475 - feat: indicate header columns in source descriptions
  • PR-8477 - fix: clean up transient queries when using shared runtimes
  • PR-8474 - fix: use correct runtime to build copy of query’s NamedTopology
  • PR-8452 - feat: verify no other query is running before putting into command topic
  • PR-8434 - feature: allow shared runtimes to be restarted to pick up config changes
  • PR-8461 - feat: removed properties from allowlist
  • PR-8453 - feat: use blocking topology changes
  • PR-8451 - fix: remove unnecessary import
  • PR-8334 - Kci 1081/implement endpoint for query scaling server configs
  • PR-8426 - feat: Add int/bigint/double conversion functions from bytes
  • PR-8417 - feat: block inserting into sources with headers
  • PR-8422 - fix: allow valid @ character on JSON extract functions
  • PR-8333 - feat: pull query LIMIT clause
  • PR-8416 - feat: populate header columns
  • PR-8423 - fix: make to_bytes work with lowercase hex
  • PR-8392 - refactor: Consolidates the logic for launching queries in QueryExecutor
  • PR-8406 - fix: synchronizing writes to localcommand file
  • PR-8409 - fix: port changes from #7863 to 5.4.x
  • PR-8400 - fix: handle null input for average udaf
  • PR-8393 - feat: build on prem images
  • PR-8383 - feat: clean up based on query id for shared runtimes
  • PR-8366 - feat: add headers to the logical schema
  • PR-8397 - fix: fix warnings in pom
  • PR-8377 - fix: Fixes a timestamp bug with Scalable Push Queries
  • PR-8389 - fix: upgrade Netty for 5.5.x
  • PR-8327 - fix: return error message if table functions are used inside a CASE
  • PR-8346 - fix: Pull Query Key Extraction Optimizations
  • PR-8369 - fix: update client to fix build
  • PR-8329 - build: disable concurrent jobs up through 6.0.x
  • PR-8350 - feat: add syntax for HEADER(<key>) columns
  • PR-8330 - fix: make parse_date able to parse partial dates
  • PR-8314 - feat: add HEADERS column type to syntax
  • PR-8332 - Kci 1191/extend list properties
  • PR-8310 - fix: dont throw error on processing local commands, just log
  • PR-8309 - refactor: introduce new api to control schema translation
  • PR-8322 - fix: if not exists return type
  • PR-8299 - fix: Pull query table scans support LIKE and BETWEEN operators
  • PR-8311 - docs: add Go and Python community clients to the list of community clients
  • PR-8264 - feat: Add consistency vector handling to CLI and Java client
  • PR-8308 - feat: [UIF-1010] Add swiftype metatag for site of ksqldb
  • PR-8282 - fix: Print only failed line on parsing exception
  • PR-8238 - feat: Moves Scalable Push Queries to use a KafkaConsumer
  • PR-8297 - fix: allow insertion of null values in migrations tool
  • PR-8271 - feat: Add support for TIMESTAMP type in the WITH/TIMESTAMP property
  • PR-8276 - fix: Add user-friendly error message for SELECT null AS column
  • PR-8277 - fix: fix_application_id_to_work_with_acls
  • PR-8261 - feat: Refactors to use KeyValueMetadata which includes RowMetadata
  • PR-8161 - Enable fetching of subsets of partitions in HARouting
  • PR-8268 - fix: change metrics tag
  • PR-8257 - fix: During ksql startup, avoid recreating deleted command topic when a valid backup exists #7753
  • PR-8253 - fix: Specify source type in DROP referential integrity errors
  • PR-8252 - fix: Allows remote pull queries to be cancelled
  • PR-8180 - klip 56 schema id in create command
  • PR-8249 - feat: add configurations around endpoint logging
  • PR-8248 - fix: Fixes errors and increased latency for pull queries from closing connection
  • PR-8245 - feat: enable ROWPARTITION and ROWOFFSET pseudo columns (KLIP-50)
  • PR-8236 - feat: Re-enable GRACE period with new stream-stream joins semantics
  • PR-8159 - chore: add an scm block to the pom.xml
  • PR-8232 - fix: doesn’t print error
  • PR-8221 - fix: update error msg
  • PR-8217 - fix: don’t register stream pull queries as push
  • PR-8230 - fix: correctly capture rows returned for scalable push query metrics
  • PR-8206 - fix: skip adding invalid if not exists to cmd topic
  • PR-8208 - fix: close command runner when command topic is deleted
  • PR-8177 - refactor: Refactor ConnectFormatSchemaTranslator to take translator object instead of lamda function so that the translator can take in config to decide how to transform field names
  • PR-8205 - fix: ClassCastException when dropping sources with 2+ insert queries
  • PR-8178 - feat: add detailed scalable push query metrics with type breakdown
  • PR-8174 - feat: Add metrics for stream pull queries to WS
  • PR-8189 - fix: allow more time for consumer group cleanup
  • PR-8168 - feat: Add metrics for stream pull queries over http2
  • PR-8164 - fix: Ensure that we always close /query writer
  • PR-8165 - fix: Always dec. concurrency counter, even hitting rate limits
  • PR-8166 - fix: stream pull query internal overrides shouldn’t clash with query configs
  • PR-8147 - Kci 903/querylogger not logging to kibana
  • PR-8143 - feat: add stream pull queries to websocket endpoint
  • PR-8123 - fix: Allow multiple EXTRACTJSONFIELD invocations on different paths (#8122)
  • PR-8142 - refactor: clean up push query publisher
  • PR-8126 - feat: Add metrics and limiters for stream pull queries for HTTP1 endpoint
  • PR-8130 - fix: CREATE OR REPLACE TABLE on an existing query fails while initializing kafka streams
  • PR-8141 - add logical cluster id to observability metrics
  • PR-8114 - fix: allow insert into table using HTTP/2 endpoint and Java API
  • PR-8124 - feat: Add stream pull queries to http2
  • PR-8138 - refactor: simplify websocket endpoint
  • PR-8137 - refactor: rename QueryExecutor to QueryBuilder
  • PR-8132 - fix: Ensures background timer completes for scalable push queries
  • PR-8118 - feat: add methods and classes to execute low level HTTP requests in the ksqldb-api-client
  • PR-8119 - fix: fix spq bandwidth throttling on http2
  • PR-8115 - feat: disable stream pull queries by default
  • PR-8074 - feat: add shared runtime config to QueryPlan
  • PR-8095 - feat: update storage utilization metrics to start when app is initialized
  • PR-8099 - Fix title in KLIP-54
  • PR-8087 - feat: scalable push query bandwidth throttling
  • PR-8098 - fix: use a looser check on the error msg for overflow
  • PR-7993 - Optimize key-range queries in pull queries (#6105)
  • PR-8094 - fix: Enables ALPN for internal requests when http2 and tls are in use
  • PR-8064 - feat: introduce stream pull queries for HTTP/1
  • PR-7955 - feat: add persistent query saturation metric
  • PR-8091 - fix: add time types to Java client
  • PR-8073 - feat: add consumerGroupId to QueryDescription
  • PR-8085 - feat: add feature flag to disable source table materialization
  • PR-8062 - fix: Adds a new config check to insert SPQ Processor
  • PR-8081 - fix: add new types to udaf functions
  • PR-7987 - feat: add CumulativeSum total bytes metric
  • PR-8063 - feat: block DROP…DELETE TOPIC for read-only sources
  • PR-8061 - feat: execute source table query plans
  • PR-8054 - refactor: RQTT: Moves system thread exclusion
  • PR-8045 - feat: introduce branches for pull queries on streams
  • PR-8053 - fix: Lower session timeout for RQTT
  • PR-7721 - feat: shared runtimes
  • PR-8052 - refactor: make sink source optional in PersistentQueryMetadata
  • PR-7868 - feat: add storage utilization metrics
  • PR-8039 - feat: perform SchemaRegistry permissions on C*AS sink subjects
  • PR-8015 - fix: date parsing functions: case-insensitive name parsing
  • PR-8043 - feat: create and write a query plan for source tables
  • PR-8022 - feat: add SOURCE streams/tables to query anonymizer
  • PR-8026 - docs: cherry-pick: revert GRACE keyword in Joins (DOCS-9583)
  • PR-8009 - feat: reject INSERT statements on SOURCE streams and tables
  • PR-7988 - feat: Allow scalable push queries to handle rebalances
  • PR-8004 - feat: add CREATE SOURCE STREAM syntax and metadata info
  • PR-7781 - Update README.md
  • PR-7996 - feat: fail scalable push query if error is detected in subscribed persistent query
  • PR-7992 - fix: Fixes pull query latency distribution metrics
  • PR-7983 - feat: Propose KLIP 53: Pull Queries on Streams
  • PR-7991 - refactor: pull query metrics
  • PR-7945 - feat: add CREATE SOURCE TABLE syntax and metadata info
  • PR-7874 - Refactor pull query execution
  • PR-7947 - feat: terminate transient queries by id

REST Proxy

  • PR-991 - KREST-4825 Huge timeout in consume test
  • PR-984 - KREST-4608 port changes to 6.2.x onwards
  • PR-983 - KREST-4605 Merge the master changes to back levels too
  • PR-982 - KREST-4577 Port changes to 6.2.x onwards
  • PR-980 - Disable producer idempotency until KAFKA-13668 is fixed.
  • PR-970 - Fix context initialization in ConsumersResource.
  • PR-963 - KREST-4113 Don’t delete all ACLs if an empty DELETE request is received.
  • PR-969 - KREST-3836 update maven to pull from https
  • PR-962 - KREST-4112 Fix Create ACL request example.
  • PR-959 - Ignore all Resilience4jRateLimitTest tests.
  • PR-946 - KREST-3255 Fix typo in 10 MB
  • PR-950 - KREST-3430 Optimize rate-limiting exceptions
  • PR-949 - KREST-3255 fix rest producer count limit missing annotations
  • PR-929 - KREST-3136 process the produce responses in their own thread
  • PR-939 - KREST-3255: Rate limit REST Produce on size of data
  • PR-920 - KREST-2654 Rate limit per clusterId
  • PR-931 - Implement request rate-limiting for Kafka REST.
  • PR-933 - Bind NoSchemaRecordSerializer as Singleton.
  • PR-924 - KREST-1936 Remove mask filtering of producer configs.
  • PR-922 - Disabled concurrent jobs
  • PR-917 - MMA-10482 return authorised operations in topic description
  • PR-912 - KREST-1976 add producer rate limiting
  • PR-916 - KREST-2835: Move ownership of Admin/Producer to HK2.
  • PR-914 - Krest 2837 streamingresponse logging
  • PR-892 - KREST-1654 JMX metrics for produce v3
  • PR-907 - MINOR: fix format for spotless check
  • PR-901 - KREST-2303: Try harder to find ConsumerLag in integration test.
  • PR-899 - Actually pin google-java-format to 1.7.
  • PR-898 - Pin google-java-format version to 1.11.0.
  • PR-890 - KREST-1673 Add error responses to the OpenAPI spec.
  • PR-888 - Add Google-Java-Format to Maven configuration.
  • PR-879 - Remove KafkaRestContextProvider.
  • PR-875 - Pass KafkaRestContext to HK2 via ContainerRequestContext.
  • PR-876 - Add KAFKA_REST_CONTEXT_PROPERTY_NAME constant.
  • PR-873 - fix formatting issues with newer checkstyle versions
  • PR-871 - Add constructor with listenerName argument
  • PR-869 - fix issues identified by spotbugs
  • PR-307 - APPSEC-725:Override netty in 7.1.x only
  • 17d8169e - fix merge issue
  • 5514b21c - MMA-11468: Add option to enable nosniff protection
  • 19657b29 - Remove Redundant Netty Definition
  • PR-305 - Remove Redundant Netty Definition
  • PR-304 - MMA-11468: Add option to enable nosniff protection
  • PR-302 - APIF-2027: Upgrade jetty-server to 9.4.44.v20210927.
  • PR-301 - APIF-2005: Change the Maven repository protocol: http → https
  • PR-296 - DGS-2581: Catch TimeoutException and map to 408
  • PR-295 - Add Netty to the right project
  • PR-294 - Enforce Netty for children
  • 6106f1b7 - Revert the change of Guava during manual merge.
  • PR-292 - Upgrade Netty Version
  • PR-291 - Test “custom methods” support in rest-utils
  • PR-290 - Disabled concurrent jobs
  • PR-285 - KREST-1671 remove cumulative metric counts as we aren’t using them
  • PR-284 - KREST-1671 Add counter metrics
  • PR-278 - Add support for PROXY protocol
  • PR-277 - Fix request logger to print request time in milliseconds
  • PR-276 - Allow disabling OPTIONS on REST proxy
  • PR-273 - fix additional checkstyle issues
  • 76c79e8b - fix checkstyle violations with newer checkstyle versions
  • PR-268 - Support for named listeners v2

Schema Registry

Note: The Protobuf Converter will now convert Protobuf wrapper types to primitives instead of structs. To retain the old behavior, set wrapper.for.raw.primitives=false.

  • PR-2203 - DGS-3147 Include enum default in AvroSchema.equals
  • PR-2201 - DGS-3066 Perf optimizations for AvroSchema
  • PR-2200 - DGS-3133: Compare canonical string instead of jsonNode when comparing JsonSchema
  • PR-2195 - DGS-3118 Escape invalid chars in doc field when normalizing Avro
  • PR-2191 - DGS-3109 Ensure context lookups use unqualifed subjects
  • PR-2187 - DGS-3066 Handle unions in AvroSchema.metaEquals
  • PR-2186 - DGS-3061 Only cache top-level schemas in AvroData
  • PR-2184 - DGS-3040 Configure Schema Registry Client [m]TLS using PEM (#2062) [skip secret scan]
  • PR-2175 - DGS-2990: Backport PR 2174 to 5.2.x
  • PR-2174 - DGS-2990 Removing message value from exception to avoid logging custo
  • PR-2173 - DGS-2988: Disable producer idempotence in KafkaStore
  • PR-2169 - DGS-2965 Make context wildcard the default subjectPrefix
  • PR-2168 - DGS-2958 Search contexts when retrieving schema
  • PR-2164 - DGS-2937 Use default scale of 0 in Avro converter
  • PR-2155 - DGS-2885 Ignore complex custom Protobuf options
  • PR-2151 - DGS-2875: Return correct default value for connect union
  • PR-2148 - DGS-2830 Check for addition/removal of required Protobuf fields
  • PR-2147 - DGS-2826 Add Avro UUID conversion; minor optimization
  • PR-2140 - DGS-2814 Ensure id generation is thread-safe in converters
  • PR-2126 - DGS-2719: add explicit dependency for kotlin-scripting-jvm
  • PR-2125 - DGS-2727 Handle multiple refs to same Protobuf enum
  • PR-2124 - DGS-2722 Use identity comparison for DatumReader/Writer caches
  • PR-2122 - DGS-2707 Ensure Protobuf reserved range uses exclusive end
  • PR-2119 - DGS-2701 Pass topic to context strategy during deserialization
  • PR-2116 - Fix compile error from KAFKA-13456 fix: effectiveAdvertisedListeners
  • PR-2118 - DGS-2700 Fix doubling of context delimiter w/custom context strategy
  • PR-2117 - DGS-2680 Use exact schema returned by ID when looking up version
  • PR-2113 - DGS-1753 Add flag to support optional fields in Protobuf converter
  • PR-2111 - DGS-1654 Generate synthetic oneofs for proto3 optional
  • PR-2108 - DGS-2646 Support name method on ProtobufSchema with only enums
  • PR-2102 - DGS-2621 Support raw primitives in the Protobuf converter
  • PR-2107 - DGS-2636 Collect properties for allOf in JSON Schema converter
  • PR-2106 - Exclude log4j2 transitive dependencies
  • PR-2104 - Exclude log4j2 dependency
  • PR-2100 - Backport PR2039 to 5.5.x branch
  • PR-2103 - DGS-2626 Handle combined schema with title and/or object
  • PR-2101 - feat: Update wire-schema version to 4.0.0
  • PR-2089 - DGS-2541 Fallback to simple name comp in Protobuf compat
  • PR-2090 - DGS-2415 Reject subjects with control chars
  • PR-2086 - Backport Netty version Override change.
  • PR-2085 - Fix widening -> narrowing issue in casting
  • PR-2082 - DGS-2487: Adding missing cache to schema registry client
  • PR-2083 - DGS-2428 minor rename to normalizeSchemas
  • PR-2079 - DGS-2507 Use fullName if enhanced.protobuf.schema.support=true
  • PR-2080 - DGS-2506 Add reference.lookup.only for Protobuf serializer
  • PR-2064 - DGS-2428 Add config to ignore ordering when registering/looking up schema
  • PR-2075 - DGS-2491 Ensure ProtobufSchema.name returns qualified name
  • PR-2070 - DGS-2401 Add converter flag to allow old doc/default behavior
  • PR-2039 - Fixes #1930: support Jakarta EE 9 by eliminating dependencies on jersey ey-common and jakarta.ws.rs in schema-registry-client. (#1933)
  • PR-2036 - Fix default value for array of records
  • PR-2068 - Demostrate wrong cache hit for union type reader schema
  • PR-2065 - DGS-2293 Add serializer config to specify schema ID (#2048)
  • PR-2061 - DGS-2417 Change moving Protobuf messages as compatible
  • PR-2058 - DGS-2393 Fix special chars in options for oneof/enum
  • PR-2048 - DGS-2293 Add serializer config to specify schema ID
  • PR-2035 - Use value schema for array item record schema
  • PR-1184 - Allowing HTTP Headers on requests to schema registry
  • PR-1933 - Fixes #1930: support Jakarta EE 9 by eliminating dependencies on jersey-common and jakarta.ws.rs in schema-registry-client.
  • PR-1996 - Use protobuf.version from common
  • PR-1976 - DGS-1907 upgrade swagger to version-2.x
  • PR-1973 - fix checkstyle violations with newer checkstyle versions
  • PR-1960 - Fix errors due to spotbugs upgrade to 4.3.0

Commercial Features

Confluent Server

  • KAFKALESS-928: Remove possible port collision when starting multiple brokers in Embe
  • CONFLUENT: Use non-parameterized tests for EndToEndAuthorizationTest since it is extended for Confluent authorizer tests
  • KGLOBAL-863: Fix formatting to handle multiple filtered out topics (#
  • AUDIT-971: Update confluent-log4j version for 7.1.x
  • Ignore failing quota_test (#5651)
  • Upgrade TCNative version
  • Cherry-pick “Disbaled bubbling errors from JsonSerializer and Deserializers for connect that print the message in stack trace (#5558)” onto 7.1.x
  • Cherrypick to 7.1.x “KSTORAGE-1915: Check if usage of S3’s GetBucketLocation API can be eliminated”
  • CPKAFKA-8140: disable tier compaction system test in 7.1.x
  • [CLI-1430] Add GCM functionality to DecryptionEngine (#5563)
  • CONFLUENT: Check version to select ZK CLI class name
  • CONFLUENT: Reinstate MultiTenantClusterLinkTest that was deleted by mistake
  • KGLOBAL-1034: Support source initiated links with link coordinators based on controller or partition leader
  • KGLOBAL-1025: Auto mirror create should exclude all linked topics. (#
  • KC-1504: fix license topic consumer endOffsets TimeoutException
  • CPKAFKA-7648: retry fetching license topic metadata in LicenseStore
  • [CNC-9] Configure log redactor for Connect (#5440)
  • KGLOBAL-1021: Fix deadlock in ClusterLinkFetcherManager during excessive epoch updates
  • KCFUN-274: ClientQuotaCallback.updateDynamicQuotas needs a default implementation (#5403)
  • KGLOBAL-995: Change mirrorTopic in NewMirrorTopic to sourceTopic in t
  • KGLOBAL-1006: Use Kafka Uuid instead of Java UUID for cluster link APIs and metrics
  • [SEC-2896] Update confluent-log4j version to cp8
  • KGLOBAL-1009: Fix testDeleteAutoMirroredTopics intermittent failures
  • KGLOBAL-1014: Disable cluster link prefix in 7.1.x.
  • KGLOBAL-854: Fix function name style. (#5353)
  • KGLOBAL-991: Allow link ids to be specified in ACL filters to prevent circular sync
  • KAFKA-13582: TestVerifiableProducer.test_multiple_kraft_security_prot
  • CONFLUENT: Fix QuorumTestHarness.controllerListenerSecurityProtocol
  • KC-2002: Disallow TS with KRaft for CP 7.1
  • Backport: AuthN-256 Fix Oauth system test with new configuration
  • Fix merge for Connect source task logic
  • CONFLUENT: Fix IntegrationTestHarness and QuorumTestHarness
  • CONFLUENT: Ensure dynamic reconfigurations are setup correctly in KRaft mode
  • Fix NPE
  • KGLOBAL-854: Introduce DescribeClusterLinks (#5014)
  • AuthN-256: authn configuration change preparation for cc-spec-kafka
  • KC-2005: update more tier integration tests to run in both zk and kraft modes
  • MINOR: Change default cmd in restore pod
  • KDATA-237 change restore job to restore stateful set
  • KSTORAGE-1859: Retag TierLogCleanerManager metrics
  • CNK-664: Throttle clients using TokenBucket algorithm
  • AUTHN-14: Add ResourceType for FederatedIdentity configuration operations
  • KSTORAGE-1785: change prefix of some tier compaction configs from “tier.compact” to “tier.cleaner”
  • KCFUN-167: Quota Publishing with existing AutoTuning Integration
  • [KSTORAGE-1913] Integrate TierObjectGarbageCollector with TierDeletedPartitionsCoordinator [skip secret scan]
  • KGLOBAL-923: Allow mirror topic offsets to be committed under a transaction
  • KC-213 Minor: fix visibility for test api
  • Make libs.logredactorMetrics compileOnly
  • KENGINE-97: Wrap all api calls with engine metrics
  • KSTORAGE-1627: block DeletionTask from deleting segments being used by CompactionTask
  • Remove Outdated Last Value for Deltas When Registering Metric
  • [skip secret scan] Make partition moves aware of Partition ineligibleBrokers
  • KC-213: Restructure ACL authorizer code to lookup ACLs for principals quickly
  • [skip secret scan] KAFKALESS-724: SBC internal implementation for trigger even cluster load
  • [skip secret scan] Cell PBTs for replicaMoves in CCloudClusterModel
  • [skip secret scan] Drop assumption that CellAwareGoal has been optimized before CapacityGoal
  • Drop assumption that CellAwareGoal has been optimized before RackAwareGoal
  • MINOR CIAM-1221 Cookies can exclude the “Bearer ” prefix
  • MINOR cleanup; use hasReplicaOfPartition instead of direct check
  • [skip secret scan] KAFKALESS-632: Revert reassignments in case a ExecutionTask is DEAD
  • [skip secret scan] MINOR: Add method in Broker to check if it hosts a replica of a TopicPartition
  • AUDIT-651: Ensure that log redaction metrics can flow to datadog
  • KC-2007; Add IsEligibleController gauge to track controller eligibility
  • CONFLUENT: Update zookeeper functions to use provided version
  • KGLOBAL-987: Ensure introduction of cluster link prefix does not hinder consumer group offset migration
  • KMETA-15: KRaft broker must initialize tenant metadata early
  • KGLOBAL-980: Ensure introduction of cluster link prefix does not hinder auto mirror topic creation
  • Remove circular dependency in startup order introduced in AUDIT-528
  • Revert concurrent shutdown in EmbeddedKafkaCluster
  • KCFUN-207: Fix startup ordering for logical cluster metadata pipeline
  • fix: replace s3 link with packages.confluent.io
  • KCFUN-266: Don’t track principals without defined tags
  • Kdata 178 integration
  • KGLOBAL-954: Introduce cluster link prefix and enhance mirror topic creation
  • KSTORAGE-1908: improve validation of illegal CommitAndSwap operations
  • KAFKALESS-793: Change kafka api timeouts to 30 seconds
  • KENGINE-106: handle SchemaException from deserializeAssignment
  • KAFKA-13111: Re-evaluate Fetch Sessions when using topic IDs (#11331)
  • Fixing breaking tests from https://github.com/confluentinc/ce-kafka/pull/4929
  • Drop assumption that CellAwareGoal has been optimized before ReplicaCapacityGoal
  • KAFKA-9648: Add configuration to adjust listen backlog size for Accep
  • [KSTORAGE-1885] tier object garbage collector implementation for TS with KRaft [skip secret scan]
  • KCFUN-238: Update TenantQuotaCallback to support service account quotas
  • Added suppression of NPathComplexity and ParameterNumber Checkstyle rules for SaslServerAuthenticator
  • KCFUN-243: SaslAuthenticatorTest.[testPublicCredential, testSslClientAuthRequestedForSaslSslListener, testSslClientAuthRequiredForSaslSslListener] fail frequently
  • AUTHN-280: Java client: implement async auth
  • CNK-1798 fix bug formatting last two octets in v4 broker names
  • Move imagePullSecrets section to where it is supposed to be
  • KCFUN-169: Hookup with QuotaCoordinator for coordinator lookup.[skip secret scan]
  • Revert “KC-1930 Enabling SBC APIs in KRaft (#4929)”
  • KENGINE-102: add metric to measure time to load logs
  • KENGINE-99: add metric to measure leader election
  • KENGINE-98: add metric to measure controlled shutdown
  • KAFKALESS-745: Temporarily disable disk PBT
  • [skip secret scan] KAFKALESS-452 Update ReplicaPlacementGoal to be cell-aware / update rebalanceForBroker
  • KAFKALESS-451 MINOR use shouldExclude in CellAwareGoal
  • KC-1930 Enabling SBC APIs in KRaft
  • CONFLUENT: Add Multi Tenant ACL system test for KRaft
  • Revert “CIAM-1218 CRN Authority to support filling environment (#4971)”
  • KGLOBAL-941: Fix matchError and uses of getCause for ClusterLinkStopMirrorTopic task
  • KAFKA-13394: Topic IDs should be removed from PartitionFetchState if they are no longer sent by the controller
  • KAFKALESS-556: Batch entities in describeConfigs call
  • KGLOBAL-969: Part 2 - Universal Fix for broker uuid in RBAC test
  • CIAM-1221 Support credentials passed as cookies
  • CIAM-1218 CRN Authority to support filling environment
  • KENGINE-29: Shrunk ISR before leader crash makes the partition offline
  • SEC-2796: Remove dead code, force initial keyset refresh
  • KGLOBAL-969: Fix cluster link ACL sync tests to work with RBAC changes
  • Add metrics for dynamic quota reporting
  • KGLOBAL-836: Reduce WARN level logs produced by ClusterLinkSyncAcls when RBAC not configured on source cluster
  • [skip secret scan] MINOR: Flat Buffer definition for restore database
  • AUTHN-263: OAuth Kafka system test - Kafka Federated Identity Authent
  • KSTORAGE-1812: skip flush for deleted log
  • Bump confluent-log4j to 1.2.17-cp6
  • KGLOBAL-958: Do broker side validation if trying to create a link with improper task configs
  • KC-1969 - Add Metrics for KC-1786
  • KGLOBAL-955: testObserverElectionWithUnderReplicatedObserverPromotion
  • KAFKA-13456; Tighten KRaft listener config checks/constraints (#11503)
  • [KSTORAGE-1905] handle NoSuchElementException in segment flush
  • Properly register HttpServerAuthFilter as a prematch filter
  • KGLOBAL-965: Fix ReplicaManagerTest.testDeltaFollowerStopFetcherBeforeCreatingInitialFetchOffset that was broken due to arg with default added in KGLOBAL-959
  • KPERF-224: Make alterLeadershipPriorityAPI idempotent
  • Security patch - Bump confluent-log4j to mitigate log4j2 jndi vulnerability - RCCA-5119
  • KC-1969/KSTORAGE-1861 - Add static and dynamic broker configs for KC-1786/ Handle Logging Volume Outage/ Add Storage probe to list of background threads
  • KC-1943: Introduce ExclusionGoal [skip secret scan]
  • KCFUN-250: Improve propagation and processing of SSL handshake failures
  • KDATA-198 Check if the log exists before checking if it is in frozen state
  • KDATA-247: integrate with kafka APIs
  • KAFKA-13533: Clean up resources correctly when task initialization fails in Connect
  • KGLOBAL-959: Track fetcher thread for partitions in cluster link fetcher manager
  • CIAM-1606: Don’t block internal listeners during Authorizer startup for CCloud
  • KSTORAGE-1887: Add configs for tier topic garbage collection
  • KAFKA-13512: Avoid duplicating maps in ZkMetadataCache topic accessors
  • KAFKA-13512: Avoid duplicating maps in ZkMetadataCache topic accessors
  • KENGINE-84; Simplify mocking logic
  • KCFUN-255: Disable reauth in testPublicCredential to avoid timing issues in test
  • KMETA-10: KRaft must honor confluent.apply.create.topic.policy.to.create.partitions
  • MINOR: add cluster link admin manager
  • [MINOR] Fix resetting halt procedure irrespective if the test fails/passes
  • [skip secret scan]KCFUN-169: Add throttled bit as part of quota consumption reporting
  • Fix race condition in BrokerHealthManagerIntegrationTest.
  • MINOR: Rename StandardMetadataRequestBenchmark to MetadataRequestBenchmark
  • CNK-1774: SocketServerTest.testIdleConnection fails
  • KGLOBAL-732: Add a new metric for unavailable link count due to authentication issues.
  • KSTORAGE-1735 / KSTORAGE-1783/ KSTORAGE-1889 - Controlled Shutdown + Background thread for monitoring availability - New PR
  • KAFKALESS-744: Temporarily disable chain PBT
  • [skip secret scan] KDATA-107: Rest API support for orchestrating tier partition state
  • MINOR: remove dead MIN_REPLICA and MAX_REPLICA code
  • [skip secret scan] KCFUN-171: Add QuotaCoordinator to track computed quotas
  • KGLOBAL-932: Fix timing issues in cluster link integration tests
  • KCFUN-242: SslCertificateManagerTest tests fail frequently
  • KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode #11416
  • Add Log Message When Calling metricChange if Delta Already Exists
  • KDATA-208: System test for corrupted checksum, detection by DA and manual recovery
  • KAFKA-13490: Fix createTopics and incrementalAlterConfigs for KRaft mode #11416
  • CIAM-1262 Role Def update for Quotas
  • KSE-700, KSE-691: Add new RBAC roles for ksql connector management
  • KGLOBAL-908: Ignore ACL filter validation on the source side of the source initiated link.
  • fix: upgrade cc-base to v12.8.0-jdk-16, async profiler 2.5
  • KENGINE-80: Failing test KafkaAuthStoreTest.testCacheFailureStatus
  • KGLOBAL-935: Fix timing issues with auto mirror creation test and its metrics
  • KDATA-256 Improved metric for fenced tiered partitions
  • KCFUN-169: First Cut of Quota Consumption Reporting Implementation
  • Rename AsssumeRole API to AssumePrincipal
  • KGLOBAL-934: Increase wait time in tests for auto-mirroring to make tests reliable
  • KGLOBAL-947: Fix timing issue in ClusterLinkFetcherManagerTest.testSourceNotAvailable
  • Make JwtPrincipal fields not mandatory
  • KENGINE-85: ReplicaManagerTest.testDeltaFollowerWithNoChange fails
  • KMETA-7: Don’t schedule ZK client reinitialize() after auth failure if connection still alive
  • CIAM-1442: Allow dataplane authtopic reader to continue on deserialisation/processing errors
  • KGLOBAL-943: Recreate local admin client for destination when cluster link resumed
  • KCFUN-220: Broker load metric missing request handler idle metric
  • KDATA-169: Add support for TopicId in Durability audits
  • KGLOBAL-932: Wait and verify no expected threads before and after each cluster link test
  • CPKAFKA-6730: Increase log segment size in upgrade tests to avoid failures with larger batches
  • KAFKA-12257: Consumer mishandles topics deleted and recreated with the same name
  • CPKAFKA-7919 Fix broker_replica_exclusion test regression
  • KDATA-250: Fix incorrect handling of register partition request in DA
  • Remove “Bearer” from javadoc and test of TokenUserLoginCallbackHandler
  • KGLOBAL-940: Ensure non-cluster-link operations don’t fail when cluster linking disabled
  • KGLOBAL-906: Track cluster link ids in migrated ACLs to ensure deletes are not lost
  • KENGINE-75; Improve visibility of the KafkaScheduler used by the GroupCoodinator
  • KAFKALESS-465: Metrics for average and largest-deviation from avg resource usage for distribution goals [skip secret scan]
  • Revert “KSTORAGE-1735 / KSTORAGE-1783 Controlled Shutdown + Background thread for monitoring availability (#4908)
  • KDATA-218: implement restore objects in store request
  • KC-1974: Pass an empty AliveBrokersSnapshot to SBC, instead of null from kraft callback
  • KGLOBAL-888: Removing AdminZkClient references in CL integration tests
  • KDATA-252: enable server side encryption of restoreObjectByCopy api for S3TierObjectstore
  • KSTORAGE-1735 / KSTORAGE-1783 Controlled Shutdown + Background thread for monitoring availability
  • [skip secret scan] KAFKALESS-451: Adding CellAwarePlacementGoal
  • MINOR: add more log for buildOffsetMap progress
  • KAFKALESS-710: Remove PreferredLeaderElectionGoal.
  • KAFKALESS-723: Add trigger even cluster load admin api
  • KC-1948: Integrate TS, DA and KRaft
  • Add a system test for leadership priority API.
  • KPERF-19: Disable SBC when there are demoted brokers.
  • KGLOBAL-890: Add dedicated admin client for dest reverse connections to avoid timeouts with unavailable links
  • KCFUN-210: Change sequence-id headers for cdc
  • KCFUN-217: PROXY connection state not updated correctly when handshake fails
  • KGLOBAL-863: fix –pending-stopped-only to print topics only in the PENDING_STOPPED state and with partitions in the PENDING_STOPPED state
  • KAFKALESS-722: Add protocol for TriggerEvenClusterLoad
  • KCFUN-182: Move quota multiplier config to TenantQuotaCallbackConfig
  • CIAM-1451 Create CCloudRoleBindingViewer role
  • Add back missing truncation log message in AbstractFetcherThread
  • KAFKALESS-453: Make RackAwareGoal cell aware
  • SBC: Add a class to iterate over subset of entites
  • DeterministicCluster extension for KAFKALESS-452, KAFKALESS-451
  • Remove dubious code in LogCleanerTest
  • KGLOBAL-859: Write cluster linking metadata from AlterIsrRequest to KRaft metadata quorum.
  • CPKAFKA-5446: unnecessary check of non-existing log segments metric
  • KGLOBAL-891: Only display topic names in exception message while deleting cluster link with active mirror
  • KSTORAGE-1875: CompactionTask handles IOExcepton while reading data from object store
  • KENGINE-82, KENGINE-83: Fix failing RBAC tests
  • KGLOBAL-599: Issue raised to fix the reverted solution of KGLOBAL-553 due to healthcheck incompatibility.
  • Revert “MINOR: Remove redundant argument from TaskMetricsGroup#recordCommit (#9642)”
  • MINOR: fix compile error with scala 2.12
  • KSE-594: Add KSQL roles to CCloud RBAC policies
  • CIAM-1484 StreamShare is a control plane resource to manage set of sh
  • KENGINE-68: Topic IDs in Fetch Metadata causes issues when IBP < 2.8
  • KREST-3802 added non streaming option to rp produce worker
  • KPLATFORM-247: Reverting base image url for ce-kafka for trogdor
  • KENGINE-45: Gather metrics to indicate engine health/progress
  • KGLOBAL-923: Don’t add mirror partitions to transactions, fail as invalid request
  • [KSTORAGE-1831] Add partition creation offset and epoch to TierTopicInitLeader
  • [skip secret scan] KREST-2513 RP Produce Trogdor worker timer bug fix
  • Added a new fault related to network AZ isolation
  • KGLOBAL-758: Add schemas for cluster links and mirror topics
  • [skip secret scan] CNK-1658: Broker leadership priority API on Zookeeper Kafka
  • CONFLUENT: Rename DataPlaneAuthCache class name to CloudAuthCache
  • CIAM-1484: Split cloud_rbac_roles.json file to Control Plane and Data Plane specific files
  • [skip secret scan] KREST-2513 add reconnection logic to RP Produce trogdor worker
  • KSTORAGE-1761: fix test case in LogCleanerIntegrationTest
  • KSTORAGE-1761: add test to ensure no metrics callback grab log lock
  • KCFUN-170: Add new PublishQuotaTarget RPC
  • KC-1925 Adding Dynamic Config Handling to SBC when running with KRaft
  • [Availability] ThreadLocalCounters Framework and Broker’s Storage Metrics Implementation KSTORAGE-1734
  • KSTORAGE-1761: remove disk access when reporting log size metrics
  • KGLOBAL-867: Creation of internal to manage the cluster link coordina
  • KENGINE-60: add internal metrics for consumer lag emitter
  • Revert “KGLOBAL-867: Creation of internal to manage the cluster link coordinator leadership election”
  • KSTORAGE-1857,KSTORAGE-1858: accept TierSegmentDeleteInitiate and TierSegmentDeleteComplete event for partition deletion
  • [skip secret scan] DEBUG KAFKALESS-715: adjusting runtime
  • KGLOBAL-867: Creation of internal to manage the cluster link coordinator leadership election
  • KDATA-241 Fix for CLM thread not exiting on broker shutdown
  • KCFUN-207: Move SecretsStore startup just before SocketServer startup
  • AUDIT-528: fix for audit logging with pkc-id or dedicated lkc-id if applicable, on failed authentication [skip secret scan]
  • Revert “KAFKALESS-715: Don’t generate invalid movements for dead brokers in RackAwareGoal (#4820)
  • Allow configurable anonymous access for HTTP Auth
  • KSTORAGE-1759: evict deleted segment IDs from tier partition state
  • BUGFIX: use ConfluentCloudCrnAuthority in all non-CP cases
  • KSTORAGE-1456: Tier Compaction Transactions
  • [skip secret scan] KAFKALESS-715: Don’t generate invalid movements for dead brokers in RackAwareGoal
  • KC-1944: Add confluent.eligible.controllers configuration
  • Fix BearerCredential Compile issue and disable MDS docker test
  • Cherry-pick upstream AlterIsr fixes (KAFKA-13091 and KAFKA-13254)
  • KREST-2513 provide cloud credentials to RP trogdor worker topic create command
  • AuthN-253, AuthN-259, AuthN255: Introduce Kafka server functionality for OAUTH authentication and authorization
  • [AUTHN-176][AUTHN-284] Create authn filter to request token auth from MDS
  • KSTORAGE-1591: implement max.compaction.lag.ms for tier compacted topics
  • KREST-2513 fix for typo in RestProxyProduceV3Worker.toString()
  • SBC: Add unit test for Cell class
  • Update confluent-log4j version to 1.2.17-cp4
  • [KC-1939] Integrating SBC with the new MetadataImageListener Interface exposed by KRaft nodes.
  • KGLOBAL-913: Disable auto-topic creation in cluster link tests to workaround license timing issues
  • CONFLUENT: update comment on supporting license topics
  • KC-1889: Implement KraftInternalAdmin
  • CDX-47 add CDXStreamShareReader role for cdx phase 1 service accounts
  • Kdata 206 kafka tier partition status request
  • [KC-1924] Adding MetadatImageListener Interface invoked by a MetadataPublisher
  • MINOR: fix broken KafkaConfigTest and LogConfigTest
  • Exception for azure unsupported operation
  • CIAM-1462 Create CCloudHealthChecker role
  • CIAM-1476 Give OrgOp and EnvOp Describe permission on networking
  • KAFKA-13312; ‘NetworkDegradeTest#test_rate’ should wait until iperf server is listening
  • KAFKALESS-701: Add metrics for self-healing enablement
  • CCDB-4277: Proactively port KAFKA-13327, KAFKA-13328, and KAFKA-13329 fixes to CE Kafka
  • CONFLUENT: Fix auth topic record header parsing logic
  • CONFLUENT: Fix TierCompactionIntegrationTest
  • KGLOBAL-912: Filter out excluded groups when committing offsets for mirror promotion
  • KSTORAGE-1723: add minimum size threshold for compacted segment
  • KAFKALESS-454: Update ReplicaCapacityGoal to be cell aware
  • KMETA-8 Only update the LISR cache if we had successful ISR updates
  • Kdata 205 copyobject
  • KGLOBAL-742: Fix comment
  • KC-1889: Abstract ZK from TierTopicAdmin
  • KAFKALESS-508: Introduce total and per-broker metrics for remaining data to move in reassignments [skip secret scan]
  • KCFUN-132: Move all broker load configs into the BrokerLoadConfig class
  • CIAM-1475: Minor memory usage improvements for RBAC Cache
  • KCFUN-170: Add new RPC ReportQuotaConsumption
  • KAFKALESS-681: Fix pending replica action metric to show its accurate amount when a broker removal is on-going [skip secret scan]
  • Use time.milliseconds() instead of time.hiResClockMs() for timestamp fields for add broker
  • CONFLUENT: Use single RoleBinding instance in all the AccessRules
  • CIAM-1419: Restructure DefaultAuthCache to lookup access rules for principal
  • KSTORAGE-1766: Fence in progress compaction upload upon new SegmentUploadInitiate
  • KAFKALESS-464: Track and log when hot partitions are detected [skip secret scan]
  • EVENTS-356: add a readme file for ce-events
  • Fix testReadCommittedFetchLimitedAtLSO
  • KGLOBAL-870: Disable ACL sync for cluster linking in CP 7.0.0
  • KC-1785 Add an SLO metrics collector
  • KPERF-208: Pipeline under min ISR topic config reads
  • KSTORAGE-1790: implement basic object existence DA audits of tier compacted topics
  • Address review comments from Kowshik and Justine
  • KENGINE-14: minor fix to consumer lag emitter integration test
  • [AUTHN-282] Move common code to ce-security-extensions
  • give environment admin full access on networking resources
  • KAFKALESS-473: Track & log aggregate goal-level stats about proposals
  • KPERF-211: ControllerContext.isReplicaOnline should not copy broker IDs
  • Kdata-86 ObjectStorePool Implementation
  • KC-1913: Add metadata offset to the partition
  • CONFLUENT: Fix thread cleanup in KafkaReader
  • Port logredactor to 7.0.x
  • KCFUN-182: Adds quota multiplier property, adjusts auto-tuning algorithm
  • KENGINE-52; Add a replication system test which simulates a slow replica
  • CONFLUENT: Add support to disable deny logic in DefaultAuthCache
  • CPKAFKA-7709: Fix ClusterLinkTest.test_mirror_using_sbk.
  • KREST-2827 [skip secret scan] add support for username/password in trogdor rest proxy load generator
  • KAFKALESS-455: Add rebalanceForBroker for CapacityGoal
  • CONFLUENT: move TopicMetadataListener to ConfluentPartitionsPerTopicListener
  • Move antlr generated classes to generated build dir
  • [KSTORAGE-1833] Introduce TierPartitionDeletePreInitiate event type and logic to block TierSegmentUploadInitiate
  • KGLOBAL-712: Fixing bug that didn’t update task ms configs
  • KSTORAGE-1813; Retain only latest transactional batch after cleaning
  • KENGINE-14: add tenant prefix to external assignments in consumer lag emitter
  • AUTHN-119: OAuth TrustPolicy Rule engine implementation
  • KGLOBAL-852: Send the correct consumer groups to the coordinator (#4718)
  • KENGINE-14: reuse admin client for consumer lag emitter
  • [SEC-2729] Consolidate Metadata Response Classes
  • KDATA-110, KDATA-111, KDATA-108: Rest API Support in Kafka for various restore requests
  • KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path
  • KDATA-41 Custom Lifecycle Manager
  • KGLOBAL-852: Send the correct consumer groups to the coordinator
  • KDATA-84 kafka connection pool
  • MINOR: fix flaky test - clear metrics before tier compaction integration test run
  • CPKAFKA-3053: Streamline Tombstone and Transaction Marker Removal
  • Replace DescribeTopicsResult all() with allTopicNames()
  • MINOR: Fix start offset alert type for type 3 case
  • CONFLUENT: Disable deny permissions check in DataplaneProvider
  • Upgrade Azure storage blob jar, force netty DNS resolver versions.
  • Upgrade Azure storage blob jar, force netty DNS resolver versions.
  • KGLOBAL-834: Prevent early cancellation of DelayedFetches which fetch from tiered storage
  • KPERF-206: lastStableOffset resulting from completedTxn is expensive when many txns for a producer id are appended
  • KGLOBAL-840: Fix bug with backward compatibility to old source clusters w/ consumer offset sync
  • KGLOBAL-834: Prevent early cancellation of DelayedFetches which fetch from tiered storage
  • Unfence KRaft before joining plugin futures
  • KC-608: TLSv1.3 support with Netty OpenSSL engine
  • CC-15871: upgrade azure-storage-blob.jar to fix the netty version issue
  • KCFUN-79: Adds a ReplicaFetcherManager TotalLag metric
  • KSTORAGE-1678: Add metrics for tier compaction
  • KDATA-85 implement ObjectStoreManager
  • This change is part of AK merge and fixes the following tests - * Bumps up the Metadata API version in Muti-tenant/clusterLink tests. * Fixes EasyMock usage in SBC MetadataClientTest. * Changes usage of KafkaFutureImpl to KafkaFuture in ExecutorIntegrationTest. (Not caught on compilation as this uses reflection.)
  • KGLOBAL-845: Close cluster link network client if admin client creation fails due to invalid config
  • KENGINE-33/KENGINE-51; lastCaughtUpTime is always reset by leader / Expansion should only consider caught-up followers
  • Bump golang for cc-kafka-init to 1.16
  • KENGINE-14: emit consumer lag metrics
  • KENGINE-3: Keep failed partition in loadingPartition map and add metrics for par
  • CONFLUENT: Fix MultiTenantRbacAuthorizerTest
  • KSTORAGE-1834: followerRestorePoint is the Last Tiered Segment with offsets before localLogStartOffset
  • KSTORAGE-1834: followerRestorePoint is the Last Tiered Segment with offsets before localLogStartOffset
  • KAFKALESS-686: Handle exceptions in KafkaCruiseControlConfig on SBC Startup
  • Fixes for AK merge
  • KGLOBAL-721: Fix output error for kafka-cluster-links –list
  • KAFKALESS-668: Log reason for shutting down SBC
  • CIAM-1380: Added SupportPan for DeveloperWrite
  • CONFLUENT: Fix rbac-access-rules-count metric
  • [TRAFFIC-886] define NetworkAdmin RBAC role
  • CNK-1757: Remove TenantAuthenticationStats metrics
  • KGLOBAL-832: Fix local listener override for source initiated links, add multi-listener tests
  • KGLOBAL-827: Fix MirrorCommand output for partitions without leader, source offset for stopped mirror
  • KSTORAGE-1537: Handle scenarios where the firstUnstableOffset is in the tiered portion of the log
  • KGLOBAL-215: Source origination link system test
  • KAFKALESS-454 partitionActionAcceptance() for ReplicaPlacementGoalAction
  • RCCA-4228 skip collecting metrics with errors
  • CIAM-1282 crn.resourceType case fix for scope
  • CIAM-1303: Add jmh benchmark for DataPlaneProvider
  • Kdata 100 pause resume restore
  • Add OpenId certified Provider test fixture for integration testing
  • CIAM-1373 - Operator roles should be in “dataplane” namespace.
  • Kdata-97 metric and log setup
  • SBC: Add partitionActionAcceptance for CapacityGoal
  • Cherry-pick KAFKA-13305: fix NullPointerException in LogCleanerManager “uncleanable-bytes” gauge
  • KC-1906 Call maybeExpandIsr in all cases when handling AISR response
  • SBC: Add partitionActionAcceptance for ReplicaCapacityGoal
  • SBC: partitionActionAcceptance() for MaxReplicaMovementParallelismGoal
  • CONFLUENT: Upgrade netty to 4.1.65.Final and disable io.netty.handler.ssl.openssl.useTasks (7.0.x)
  • Add version number to IdentityPool records
  • CONFLUENT: Upgrade netty to 4.1.68.Final and disable io.netty.handler.ssl.openssl.useTasks
  • KENGINE-49; Cache ReplicaManager.fetchPartitionPruningEnabled to avoid lock contention
  • KGLOBAL-294: Reverse connection calls are stuck when source controlle
  • SBC: Add RackAware actionAcceptance for partition move
  • KFUNC-143 for KRaft BrokerServer
  • KAFKALESS-616: Make a broker shutdown race harder to hit.
  • CIAM-1354 Operator roles should be able to View Clusters
  • KGLOBAL-294: Reverse connection calls are stuck when source controlle
  • kafka manager implementation
  • Update gcs cloud storage client to 2.1.2
  • KSTORAGE-1816: TierPartitionStatePropertyTest metadata lookup failure
  • AUDIT-720 : Modify ce-kafka in order to include the request id in authorization audit logs
  • KSTORAGE-1796, KSTORAGE-1797, KSTORAGE-1798: improve record batch iteration and offset translation perf
  • KC-1910; Add configuration to skip SSL certificate compatibility check
  • KCFUN-144: Find a better way to tag the broker load metric with the correct LKC for dedicated clusters
  • KGLOBAL-818: Handle NOT_CONTROLLER error for initiateReverseConnection
  • KGLOBAL-822: Fix bug for altering topic filters for cluster links
  • CONFLUENT: KRaft: pass KafkaPrincipal to AlterConfigsPolicy
  • KSTORAGE-1744: remove COMPACTED_SEGMENT_UPLOAD_INITIATE
  • KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown (#11351)
  • KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown
  • KAFKALESS-653: Propagate exception messages from Balancer APIs in the controller
  • Revert “DGS-2161: Update netty version to 4.1.65.Final”
  • Fix KRaft MDS check for CCloud
  • Attempt #2 to fix gradle HTTP timeouts
  • Increase Gradle HTTP timeout
  • DGS-2161: Update netty version to 4.1.65.Final
  • DGS-2161: Update netty version to 4.1.65.Final
  • KDATA-98 initial implement partition restore workflow and FSM
  • Format storage for KRaft in CCloud
  • CIAM-1283 Use SchemaRegistry in CRNs not SchemaRegistryCluster
  • Fix NPE in KRaft Controller for auditLogProvider
  • KCFUN-154: TopicConfigPolicy should respect dynamic broker configs on startup
  • AUTHN-167: Add IdentityPool records to auth store
  • KAFKALESS-666: Fix cosmetic issues in kafka-remove-brokers –broker-ids describe command
  • KAFKALESS-655: Clear Executor state on exceptions encountered before proposals start executing
  • KAFKALESS-556: Simplify MetadataClient conditional topic placement fetching and consolidate MetadataClient usage in SBC
  • CIAM-1084: Added SupportPlan Describe permission.
  • Skip alter-config/create-topic policy checks for KRaft in CCloud
  • CAPAC-255: Ignore load metric spikes during cluster provision, expansion
  • AUTHN-207: Add caching to IdP grant requests
  • AUTHN-166: Add support for JWT Issuer AuthEntry types
  • KC-1902: Disallow MDS with KRaft
  • KSTORAGE-1801: add an internal config to gate linux-diskstats metric
  • KAFKALESS-341: Retry CDBE startup on failure
  • KREST-1531 Adding Rest proxy produce load generation to trogdor
  • KGLOBAL-801: Don’t include topic filters from source links for validating auto-mirroring overlap
  • KCFUN-143: Don’t start the log consumer from createSaslServer
  • KSTORAGE-1751: rename kafka log dir block device stats
  • KAFKA-13079: Forgotten Topics in Fetch Requests may incorrectly use topic IDs
  • KGLOBAL-742: LINK_FAILED replica status sholdn’t stop failover. (#4531)
  • KGLOBAL-742: LINK_FAILED replica status sholdn’t stop failover.
  • KAFKA-10580: Add topic ID support to Fetch request (#9944)
  • CONFLUENT: Do not use MetadataServerConfig in DataplaneProvider
  • CPKAFKA-7455: upgrade system test archive failures (#4528)
  • CPKAFKA-7455: upgrade system test archive failures
  • Add multi-tenant support to forwarding
  • KDATA-104 restore / reconcile segment state in TopicPartition State file
  • CC-13747 CIAM-1263 FF-4419: Limit max retries in MDS URL Refresh loop
  • Convert ce-broker-plugins from JUnit 4 to JUnit 5
  • SEC-2731 - Improve Ldap handling for stale connections
  • CPKAFKA-7608: reassign license topic in unclean leader election syste
  • CPKAFKA-7608: reassign license topic in unclean leader election system test
  • KSTORAGE-1802: remove linux-diskstats metric from CP 7.0
  • CC-13747 CIAM-1263 FF-4419: Limit max retries in MDS URL Refresh loop
  • Make $buildDir/dependant-libs a constant for copyDependantLibs task
  • KDATA-96 implement wrapper tool to start restore job
  • CIAM-1192 Added resource type “OwnClusterApiKey” to allow data plane roles access to own api keys
  • KDATA-160: Add more logging information for DA
  • KDATA-185: Race with local audit and hotset reduction and TopicDeletion check should happen before stale epoch check.
  • KGLOBAL-779: Avoid truncation in circular mirror test when recreating source topic
  • EVENTS-428: add event emit enabled flag
  • Make Jwt Verifier public and extensible
  • KENGINE-26: KAFKA-13237; Add ActiveBrokerCount and FencedBrokerCount metrics to the ZK controller (KIP-748)
  • KPERF-196: Cache and look up MetricsKey in Kafka and Yammer metrics collectors
  • KGLOBAL-751: Improve error handling for source initiated link creation without dest link
  • KGLOBAL-750: Allow failed links to be reconfigured after password encoder secret update
  • KGLOBAL-741: Improve error message for invalid cluster link configuration
  • Fix GCS tiered storage system tests for CP 5.4/5.5
  • KDATA-184: fix GCS object store existence check and overall object store loss check
  • CIAM-1253 Fix Logging of various timeouts in ce-rest-auth RestClient
  • KAFKALESS-556 Fixes to main PR
  • [AUDIT-415] Update confluent-log4j version
  • KDATA-162 generating configmap
  • KSTORAGE-1637: new tier compaction efficiency heuristic
  • KSTORAGE-1751: fix kafka log dir block device stats
  • KAFKALESS-617: Set a low retry-on-reassignment-failure value for tests.
  • SEC-1537 CIAM-1253 Fix Logging of various timeouts in ce-rest-auth RestClient
  • KSTORAGE-1778: deleted partition coordinator handling of compacted segments
  • KGLOBAL-739: Notify cluster link fetcher thread on high watermark update to resume fetching
  • KSTORAGE-1717: add more metrics to measure total compaction lag
  • CONFLUENT: Fix NoSuchMethodError in ExecutorIntegrationTest
  • Fix ReplicaManagerTest.testReadCommittedFetchLimitedAtLSO
  • KAFKALESS-556 SBC MetadataClient should only call describeConfigs when necessary
  • Revert “MINOR: System test enhancements for muckrake w/outZooKeeeper”
  • AUTHN-97: Mutli-Issuer JwtAuthenticaator
  • KSTORAGE-1721: avoid reading a whole segment into one big memory buffer in CompactionTask
  • KSTORAGE-1773: fix NPE in LogCleanerManager “uncleanable-bytes” gauge
  • KAFKA-13234; Transaction system test should clear URPs after broker restarts (#11267)
  • MINOR: Remove duplicate confluent.balancer config in KafkaConfigTest
  • KDATA-88: ObjectStorePool
  • MINOR: Lower the minimum encryption key manager rotation interval to 1 minute from 30 minutes
  • Handle EOF when reading from channel to handle PROXY headers
  • KC-1850: Ensure topic ID upgrade does not overwrite pending isr state when upgrading to topic IDs
  • KSTORAGE-1164, KSTORAGE-1765: Tier Compaction - implement property tests
  • EVENTS-11: non-blocking Kafka Exporter
  • CIAM-1190: Update DataplaneAuthReader to handle out of order sequence IDs
  • CONFLUENT: Remove force_use_zk_connection from topic command
  • SBC: MINOR. Update Retryer to not rely on Optional
  • KPLATFORM-70: Upgrade docker base image to v11.2.0. This changes the jdk version to 16.0.2
  • KENGINE-17; Fetch from follower stuck after unclean leader election
  • KSTORAGE-1770: namespace confluent configs
  • KAFKA-13225: Controller skips sending UpdateMetadataRequest when no change in partition state. (#11255)
  • [DevProductivity] Make system tests run on EC2 machine with temporary credentials for AWS
  • Updating Readme for using temporary credentials with AWS in EC2 machine
  • KAFKALESS-10: Fix LeaderBytesInDistributionGoal low util threshold
  • KSTORAGE-1775: Compact feature flag results in FTPS header size change and incorrect layout
  • SEC-1537 RestClient Retry Logic Fixed (#4395)
  • SEC-1537 Fix Busy-Loop in RestClient Retry Logic
  • KSTORAGE-1105: Refactor log cleaner code for reuse in tier compaction
  • KC-1878 Debug Logging for LDAP Context creation
  • CNK-1372: Load PhysicalClusterMetadata from topic (in proto format)
  • Test failures follow-up ce-kafka/pull/4704
  • KAFKALESS-607: Take care of offline nodes when creating cluster metadata
  • KSTORAGE-1048: end to end compaction for tiered topics
  • KGLOBAL-709: Receive buffer auto tuning for cluster linking connections
  • KAFKALESS-613: Update AlterBrokerReplicaExclusionResponse to contain reason for broker exclusion
  • KCFUN-130: Incrementally update topic-based sasl secrets on api key updates.
  • KC-1888: Initialize dynamic logins in KRaft before plugins start
  • KGLOBAL-499: Add a link config to specify the topic configs to sync
  • KDATA-166, KDATA-153 Handle missing delete partition event
  • KC-1780: MultiTenantPrincipalBuilder KafkaPrincipalSerde
  • CONFLUENT: Include core test jar to dependant-testlibs directory
  • cherry-pick KAFKA-13134
  • Change Guava from andriod to jre
  • Change Guava from andriod to jre
  • CIAM-1227 - Add View operation to MetricsViewer Role
  • KSTORAGE-1692: Tier state committer takes log creation log blocking new partition creation
  • KAFKALESS-592: Revert rebalance storm prevention code to require full fixes.
  • KSTORAGE-1712, KAFKA-12840; Removing compact cleaning on a topic should abort on-going compactions
  • KGLOBAL-686: Detect circular mirrors for older IBPs by tracking epoch update state
  • KGLOBAL-713: Use topic ids to detect source topic deletion and recreation to avoid circular mirror
  • KGLOBAL-720: Don’t fail LeaderAndIsr request for mirror partitions if link metadata not yet processed
  • KC-1800: Upgrade Guava to 30.0-android
  • KAFKALESS-592: Revert the rebalance storm prevention code to require full goal violation fixes. #4179
  • KC-1800: Upgrade Guava to 30.0-android
  • KGLOBAL-719: Report link count metric only on controller
  • MINOR: Update attribute passed to ServiceQuorumInfo from using_raft to using_kraft
  • KSTORAGE-1713: Document the code asking user to ignore KIP-405 tiered storage configuration in CE-Kafka
  • KC-1873: Upgrade jetty-server to fix CVE-2021-34429
  • KENGINE-23: Add User Resource Id tag to the Tenant Metrics
  • KCFUN-131: SaslSecretsStore should be resilient to malformed messages
  • KC-1850: Ensure topic ID upgrade does not overwrite pending isr state when upgrading to topic IDs (7.0.x)
  • KAFKALESS-528: Add shutdown support to multi-broker removal.
  • KGLOBAL-700: Add more TRACE level logging for periodic tasks
  • KAFKALESS-354: Update the Broker Removal State Machine to include all necessary phases
  • KAFKALESS-586: Do all removal ProgressListener callbacks in the PhaseExecutor
  • KC-1865: KRaft BrokerServer missing certain features/fixes
  • KSTORAGE-1668: Collapse Log class and move implementation into MergedLog (refactoring part 2)
  • KCFUN-125: broker load improvements
  • KSTORAGE-1668: Incorporate LocalLog class into Log class (refactoring part 1)
  • KDATA-127: durability audits on GCP performs 2MB reads per segment existence check
  • KDATA-156: Refactor AsyncService Messages into own classes and delete MessageType enum
  • KC-1875: Do not enable both KRaft and CL in ducktape
  • KSTORAGE-1636: Partial revert of KSTORAGE-390 (#3894)
  • KGLOBAL-692: Don’t truncate mirror topic for source epoch 0 when source topic is recreated
  • CIAM-1197 Give access to TransactionalIds to Admin roles
  • KDATA-82: Async Task Scheduler
  • KSTORAGE-905,KSTORAGE-1725: Property Test for MergedLog Actions
  • KAFKALESS-575: Retry ApiPersistenceStore initialization and increase its timeout to 60s
  • KC-1869; IllegalStateException in ControlMetadataBatch due to topicId mismatch
  • SBC: Add method to accept partition movement across cells
  • KC-1852: [CVE-2021-35516] Upgrade commons-compress
  • KGLOBAL-478: Replace references to source connection origination with source initiated link
  • KAFKA-13209: Upgrade jetty-server to fix CVE-2021-34429
  • KAFKA-13209: Upgrade jetty-server to fix CVE-2021-34429
  • KAFKALESS-437: Return BrokerShutdownStatus based on request version in order to not break older clients
  • CIAM-1127: Added Operator role with only view operations.
  • KGLOBAL-703: Allow TLS cluster link configs in MT interceptor to support hybrid mTLS links
  • KAFKA-13194: bound cleaning by both LSO and HWM when firstUnstableOffsetMetadata is None (#11199)
  • KAFKALESS-398: Add UNKNOWN status value to balancer operation statuses
  • KGLOBAL-696: Remove unnecessary purgatory when cluster linking is disabled
  • MINOR: Controlled shutdown is triggered twice when a Broker shuts down
  • KGLOBAL-695: Change purgatory to val from method to mitigate RCCA-3631
  • Revert “KGLOBAL-553: Do not filter out the non-updatable cluster link configs in the multitenant interceptor.”
  • KCFUN-119: Plumbing through ACL limit control in MultiTenantAuthorizor.
  • SBC: Reafctor CapacityGoal::rebalanceForBroker to allow additional balancing steps
  • KGLOBAL-548: Rename topic mirror to mirror topic in codebase
  • KGLOBAL-689: Adding lag to kafka-mirrors –describe
  • KGLOBAL-682: Propagate SSL handshake failures during cluster link validation to clients
  • KAFKALESS-595: Harden regex to only emit metrics processed by SBC to local telemetry topic
  • KGLOBAL-672: Process cluster link pause state during link creation and broker startup
  • KAFKALESS-588: Update system tests to remove multiple brokers at once
  • Cherry pick MMA-9932 Max timestamp changes from KIP-734
  • AUTHN-175: OIDC Http Client Implementation
  • KSTORAGE-1686: Intern OpaqueData instances
  • MMA-9932 Max timestamp changes from KIP-734
  • KAFKALESS-594: Extend BrokerShutdownManager to shut down multiple brokers
  • KGLOBAL-665: Fixed bug with auto mirror topic create and cluster link config updates
  • KAFKA-12155: Metadata log and snapshot cleaning #10864
  • KAFKA-12155: Metadata log and snapshot cleaning #10864
  • KCFUN-39: Add config to fallback to non-PROXY request when no PROXY h
  • Revert cell [2/2]
  • Revert cell [1/2]
  • TelemetryReporter self metrics [METRICS-2864]
  • KCFUN-123: Make SSL engine factory class reconfigurable
  • KAFKA-12779: rename namedTopology in TaskId to topologyName #11192
  • KSTORAGE-1713: Document the code asking user to ignore KIP-405 tiered storage configuration in CE-Kafka
  • KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-m
  • KDATA-80: Create Message and API Declaration for Async Service Manager Framework
  • KGLOBAL-482: Remove –stop-lag-ms and –stop-lag-records from kafka-m
  • KCFUN-38: Enforce IP quotas in processor for proxy protocol
  • KDATA-148: Handle duplicate events in DA due to producer retry. Also fixed certain scenario of out of order delivery of high watermark events.
  • KGLOBAL-664: Revert KGLOBAL-382 which blocks cluster link update operations using API keys in Cloud
  • Revert “MINOR: Bump ce-kafka to 7.1.0-ce”
  • KAFKA-13104: Controller should notify raft client when it resigns #11082
  • KAFKA-13104: Controller should notify raft client when it resigns #11082
  • KAFKA-13090: Improve kraft snapshot integration test
  • KAFKA-13090: Improve kraft snapshot integration test
  • KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • KAFKA-12777: Fix a potential NPE in AutoTopicCreationManager
  • KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • KAFKA-13083: Fix KRaft ISR in createPartitions, createTopics
  • KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • KAFKA-13037: “Thread state is already PENDING_SHUTDOWN” log spam
  • Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
  • Revert “MINOR: Upgrade Gradle to 7.1.1 and remove JDK 15 build (#10968)”
  • KAFKA-12155: Metadata log and snapshot cleaning #10864
  • KAFKA-12155: Metadata log and snapshot cleaning #10864

Security

  • DGS-3112: Add operations for exporter endpoints
  • Added fix for empty bulkAuthorize requests
  • Upgrading ksql Jetty JAAS version.
  • Exclude netty kafka rest
  • Disable producer idempotence in SR security plugins
  • Upgrade vertx to sync with ksql 7.1.x
  • APPSEC-878: Override io.netty:netty-codec and io.netty:netty-codec-http [7.1.x only]
  • DG-1249, DG-1257, DG-1353, DG-1423, DG-1462, DG-1491, DGS-2950: Fix testGetLastOffsetWithExistingMessages
  • CIAM-1735 Add debug logging to KSQL Permission Validator
  • APPSEC-827:Upgrade bc-fips
  • KSQL-7811: support JwtPrincipal wrapped in DefaultKsqlPrincipal (7.1.x)
  • DGS-2730: Use fail-open logic in license validation
  • BUGFIX: KC-1948 changed TestSslUtils to return String not Password (#
  • DGS-2570: add ScopeResource for SchemaRegistrySecurityResourceExtension
  • Override Netty-codec version
  • Fix Netty version in confluent-security-plugins 5.4.x
  • Upgrade Vertx
  • Upgrade Vertx
  • Made suggested changes
  • Added method to add more resource actions into the map
  • Disabled concurrent jobs
  • KREST-2835: Override KafkaModule bindings with RequestScoped bindings.
  • Fix compile error from ce-kafka change
  • Add logging info to KSQL/SR permission checks
  • Set SR ‘restService’ accessible on KSQL security plugins
  • Perform Schema Registry checks on KSQL security plugin
  • Remove KafkaRestContextProvider.
  • Use ContainerRequestContext instead of KafkaRestContextProvider.
  • Fix signature of ConnectorsResource.restartConnector
  • fix formatting issues with newer checkstyle versions
  • Update KsqlRestAuthorizer with new checkPrivileges() method
  • SEC-2533 upgrade commons-io to 2.11.0

Secret Registry

  • Rename advertisedListeners to effectiveAdvertisedListeners (#52)
  • Disabled concurrent jobs
  • CONFLUNET: Remove usage of WorkerConfig.REST_PORT_CONFIG config (#47)
  • Fix errors due to spotbugs upgrade to 4.3.0

Control Center

  • (MINOR): add retry for test
  • MMA-11641: increase retry count
  • MMA-11641[Test]: wait longer for c3 to start
  • MMA-11642: Increase retry count
  • MMA-11637: use AdminClient to create topic
  • MMA-11603: Update protobuf version
  • MMA-11121: Fix compatibility issues for partition size metric
  • (MINOR): Fix formatting issue
  • MMA-11478: enable cors check on websocket
  • MMA-11451: fix @JsonProperty usage to yield expected output
  • (MINOR): update armeria version and exclude netty artifacts
  • (MINOR): fix merge
  • MMA-11468: Add option to enable nosniff protection
  • MMA-10986: fix health check resource for connect
  • MMA-10571: add SR basic auth to armeria healthcheck
  • (MINOR): fix armeria test
  • MMA-9126: Fix broken RocksDB stats endpoint
  • MMA-9292: unit test for ClusterDataExporter extraction of security protocol
  • MMA-9292: control-center-export script reads security.protocol from prop file
  • MMA-10114: Change default values for command topic configuration
  • MMA-10668: reduce the broker id retention to 30ms
  • MMA-11374: fix multiple entries with same key
  • MMA-9479: Remove JDK dependency from control center stop script
  • DGS-2570: call /scope instead of /permissions to get SR scope in RbacServiceVisibilityFilter
  • (MINOR): fix build break
  • PG-383: Allow component-specific SSL configurations
  • (MINOR): fix SR test due to SR method signature change
  • (MINOR): Disable concurrent builds 53x
  • (MINOR): Disabled concurrent jobs
  • PG-382: Allow default truststore with custom truststore in jetty proxy layer
  • MMA-11217: use test containers for integration test
  • MMA-11236: clean and reorganize checkstyle suppressions
  • MMA-8609: DescribeConsumerGroups catch UnsupportedVersionException and retry
  • KEXP-171 disable last produce time offset commit
  • KREST-2835: Make KafkaRestContext handling in line with Kafka REST.
  • PG-406: Add additional documentation for Docker and getting a license key
  • PG-334: Add retry for failing tests
  • KGLOBAL-690: Remove CL endpoints in Kafka API
  • PG-371: refactor CommandStore to support multiple store types
  • PG-298: set context path to allow non-root hosting
  • (MINOR): fix some flakiness in integration test
  • MMA-11113: Feature flag changes for new C3 broker settings
  • PG-399: validate leading and trailing slash in base path
  • KEXP-33 consumer offsets partial failures
  • (MINOR): Add dependency to kafka-protobuf-provider
  • PG-358: flat list of compatible metrics
  • (MINOR): fix build break
  • (MINOR): Remove use of KafkaRestContextProvider
  • MMA-10910: fix test
  • MMA-10870: class header for ControlCenterConfig
  • (MINOR): Use protobuf.version from common
  • dev docker setup
  • (MINOR): fix formatting for newer checkstyle versions
  • PG-303: use mustache to resolve tokens in html page
  • PG-317: add mustache dependency
  • (MINOR): fixes for upstream common pom dependency updates
  • Added support for bigint format for numbers coming from ksql JSON
  • MMA-11702: Fixed the “New action” button on the Alerts Actions list page
  • KEXP-114: Fixed broker details page to show correct values under Production and Consumption latency

MQTT Proxy

  • CCMSG-1677 handle enable idempotence producer by default by explicitly setting the idempotence based on delivery guarantees
  • CCMSG-1677 handle enable idempotence producer by default by explicitly setting the idempotence based on delivery guarantees
  • Fixing error in repeated manual merge.
  • [CC-16389] Changing the maven URL in pom.xml from http to https
  • Revert “[CC-16389] Changing the maven URL in pom.xml from http to https”
  • [CC-16389] Changing the maven URL in pom.xml from http to https
  • Disabled concurrent jobs

Metadata Service

  • MDS-33 - Add AES GCM enable variable
  • CIAM-1761 - Update postgres db driver to 42.3.3 for cve issue
  • MDS-25 Prevent Credentials from be logged when Jetty prints debug auth headers
  • CIAM-1737 Bump PSQL Version
  • Don’t release ports on timeout
  • SEC-2937 Don’t fail on shutdown timeout in ProxyServletRewriteTest
  • KGLOBAL-969 : Adapt MDS tests to lookup brokerId
  • test: update tests for new RBAC roles (CCloudConnectorManager + CCloudClusterRoleBindingAdmin)
  • AUTHN-263: Fix interface API compatibility
  • SEC-2886 CIAM-1610 - Remove explicit usage of log4j2 from extractor class
  • SEC-238 MDS Test : Silent Proxy of Writes
  • SEC-2805 - Manually tested fix for MDS forwarding logic to handle unicode and request with query params
  • Implement equals and hashcode in data classes
  • AUTHN-124 Extend RBAC Extractor to publish Trust Service data
  • Disabled concurrent jobs
  • CIAM-1540: Add self field to metadata, JsonIgnore V2RoleBinding
  • FIX-CHART: Fixed chart issue.
  • CIAM-1540: Sync V2SRBR and V2LRBR with Public API minispec
  • CIAM-1215:Cloud Dataplane only mds
  • CIAM-1492: Added kubernetes resource limit.
  • CIAM-1536: Change Public API create endpoint success code to 201
  • CIAM-1484: Changes after Cloud Roles json file split
  • Update public api timestamp format
  • CIAM-1511 Nov Fix Master MDS build
  • CIAM-1447 - Fix CVE for commons-codec
  • Add TrustServiceConfig for PolicyEngine and DB Store.
  • Update roles prefix
  • AUTH-253: Decouple RBAC/TrustService API dependencies
  • CIAM-1447 - Fix CVE for commons-codec
  • CIAM-1447 Fix CVE for commons-codec
  • CIAM-1302: Handle a bad format protobuf from lifecycle events
  • CIAM-1081 Fix bug with resourceRole add/remove endpoints and add tests
  • Revert “Hack it”
  • CIAM-1462 Add Health Checker Role
  • CIAM-1379 /roles endpoint can take multiple namespaces
  • Hack it
  • CIAM-766: Catch missing org in MdsScope
  • CIAM-1456: Remove inclusive parameter from list endpoint
  • CIAM-871: Make DB v2alpha1 match topic v2alpha1 w.r.t RoleBindingFilter logic
  • CIAM-1441: Added flag for dataplane rbac-extractor.
  • CIAM-1434-CIAM-1435: Handled invalid PatternType and Limit ResourceName length
  • CIAM-1422 Limit Rolebindings to fixed value
  • CIAM-1138: New Authorizer Java client
  • CIAM-1240: Rename resourcePattern to crn_pattern in public list endpoint(#734)
  • CIAM-1417: Created events.logic_cluster topic for testing.
  • AUTHN-211 Trust Service DB implementation
  • CIAM-1251: Copy over /roles endpoints from v2alpha1 to v2
  • CIAM-1145: Add get by id public API endpoint
  • CIAM-1418 Fix error on adding role that already exists after delete
  • CIAM-507: Remove authorization from internal endpoint
  • Remove Policy Engine update security-extensions dependency information
  • CIAM-1372: Added metrics for lifecycle republish.
  • CIAM-1144: Add delete by id public API endpoint
  • CIAM-1382: renamed sync.rbac topic and sync.logical_cluster name.
  • CIAM-1214:Add e2e functionality for v2 authorize API
  • CIAM-1388 - Make trust and policy modules play nice with MDS build
  • AUDIT-720 : Modify MDS to include the request id in the authorization logs emitted
  • CIAM-1385: Revert back to sequenceId as String.
  • CIAM-1346: Convert _sequence_id to long bytes.
  • CIAM-1369: Fixed the issue that wrong resources associated with wrong RBAC roles.
  • AUTHN-119: OAuth Policy Rule Engine Implementation
  • CIAM-1282 Fix mds build for a related ce-kafka change
  • CIAM-1373 - Operator roles should be in “dataplane” namespace.
  • Add TestNg depenency as test runner for junit
  • Trust Service Scaffolding
  • Add version number to IdentityPool record
  • CIAM-1209 Sort roles by defined order
  • CIAM-1184 Collapse Org/Env/CloudClusterOperator to just Operator
  • CIAM-1250: Add inclusive parameter to public API list endpoint
  • AUTHN-167: Add IdentityPool records to AuthStore
  • Add missing AuthWriter methods to StubAuthWriter
  • Rename Public API rolebindings path to role-bindings
  • CIAM-766: Remove v2alpha1 CP tests
  • CIAM-1258 Allow Resource Owners to lookup role bindings on their resources
  • AUDIT-571 Remove the format for X-Request-Id from openapi spec
  • AUTHN-166: Add support for JWTIssuer AuthEntry types
  • CIAM-1260 Used key/secret from events path in vault.
  • SEC-2731 - Improve Ldap handling for stale connections
  • CIAM-742: Implemented handling new lkc events from events kafka cluster.
  • Rest rest-utils-parent as parent pom
  • AUDIT-571 : Add a header for request Id
  • CIAM-1271: Switch back to the oracle jdk image
  • CIAM-155 Update OpenApi doc to use resource IDs
  • CIAM-1178 : CLI: Support resources in iam commands
  • CAM-1261: Fixed authentication error for RBAC Extractor
  • CIAM-1212:Swagger file for v2 authorize api
  • CIAM-1135:Added stub V2 Authorize API call for CCloud
  • CIAM-1238: Turned on sending RBAC extrator metrics to datadog.
  • CIAM-1087 Operation Create endpoint for Cloud
  • CIAM-507: Add internal MDS endpoint for Admin Dash get rolebindings (followup)
  • CIAM-507: Add internal MDS endpoint for Admin Dash get rolebindings
  • CIAM-1224: Added publish_type column for stag and prod.
  • CIAM-1224: Added a new column publish_type to extractor_publisher_state in DEVEL
  • CIAM-967 Create DB User for RBAC Extractor
  • CIAM-1191 Can’t see rolebinding with lookup at cluster scope
  • CIAM-1150 Managed lookup returns resource roles
  • CIAM-1097 Create RDS User for Data Sci ETL service
  • CIAM-1127: Fixed test cases failures caused by ce-kafka changes in CIAM-1127
  • CIAM-740: Added consumer for sync.logical_cluster
  • SEC-2525 - ClusterRegistry authroization logic
  • SEC-2525: Create new ‘BootstrapCluster’ operation
  • CIAM-687 Implement List Endpoint for Public API
  • CIAM-730 - Extractor Ops : Deploy to Stag/Prod
  • CIAM-689 Add Delete endpoint for Public API
  • CIAM-1142: Disabled emitting metrics to datadog due to cc-otel-collector is not deployed to mothership yet.
  • CIAM-1129: Fixed configmap.ymal
  • CIAM-732: Implement METRICS for Data Plane RBAC Extractor using OpenTelemetry
  • CIAM-1116: Removed hack added by CIAM-1099
  • CIAM-1016 Use org.resource_id LaunchDarkly value
  • fix formatting for newer checkstyle versions
  • CIAM-1093 Remove references to H2
  • remove KafkaHttpApplicationProvider.provide
  • Try to speedup CI
  • Update providers for new KafkaHttpApplicationProvider interface
  • CIAM-1104: Fixed URL path of Extractor
  • CIAM-741 Query to publish all rolebindings relevant to scopes
  • fixed sync topic
  • CIAM-1099: Hard code _logical_cluster_id to lkc-3nrjo2 for demo purpose.
  • CIAM-697: Framework for rbac public api setup
  • Hackthon POC - MDS can run w/out Kafka and talk to DB
  • CIAM-1091: Changes the _app_name of RBAC Extrator application to rbac
  • CIAM-989 Move Data Governance roles to own namespace

Replicator

  • REPL-1884: Download artifacts over secure channel
  • REPL-1843: Track uncommitted records given to Connect framework.