Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Version 6.0.0

Community Features

Common

  • PR-294 - SEC-1307: Backport “log4j replacement with confluent repackaged version”
  • PR-278 - SEC-1334: update confluent-log4j version
  • PR-277 - KC-613: Update slf4j version to match AK’s.
  • PR-274 - ST-3988: Update jackson version to 2.10.5
  • PR-275 - SEC-1305: update jackson-bom version
  • PR-272 - METRICS-1918: bump gson from 2.8.5 -> 2.8.6
  • PR-270 - SEC-1034: log4j migration to confluent repackaged version
  • PR-269 - Upgrade Jackson to 2.10.2 and Avro to 1.9.2
  • PR-265 - downgrade maven-deploy-plugin to 2.8.2
  • PR-258 - update various maven plugins
  • PR-257 - ST-1323: Enable source jars if Apache or Confluent license files are found
  • PR-262 - fix: Pin the Scala version
  • PR-261 - Update ZooKeeper to 3.5.8
  • PR-260 - Update Scala version to 2.13
  • PR-259 - add property for jackson-jsonschema version
  • PR-248 - Upgrade Avro to 1.9.2

Kafka

  • PR-9197 - Revert KAFKA-9309: Add the ability to translate Message to JSON
  • PR-9172 - KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect
  • PR-392 - KAFKA-2715: Removed previous system_test folder
  • PR-384 - KAFKA-2714: Added integration tests for exceptional cases in fetching
  • PR-9089 - KAFKA-10224: Update jersey license from CDDL to EPLv2
  • PR-9066 - KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets
  • PR-9086 - FIX: Remove staticmethod tag to be able to use logger of instance
  • PR-9075 - KAFKA-10306: GlobalThread should fail on InvalidOffsetException
  • PR-9065 - KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates
  • PR-9051 - KAFKA-10268: dynamic config like “–delete-config log.retention.ms” doesn’t work
  • PR-9026 - KAFKA-10274; Consistent timeouts in transactions_test
  • PR-8979 - KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers
  • PR-8239 - KAFKA-9666: Don’t increase transactional epoch when trying to fence if the log append fails
  • PR-9009 - KAFKA-6453: Document how timestamps are computed for aggregations and joins
  • PR-8994 - KAFKA-10247: Correctly reset state when task is corrupted
  • PR-9010 - KAFKA-10262: Ensure that creating task directory is thread safe
  • PR-9005 - KAFKA-10263: Do not assign standby for revoking stateless tasks
  • PR-8996 - KAFKA-10249: don’t try to read un-checkpointed offsets of in-memory stores
  • PR-8934 - KAFKA-10134: Use long poll if we do not have fetchable partitions
  • PR-8989 - KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups
  • PR-8962 - KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory
  • PR-8983 - KAFKA-8398: Prevent NPE in forceUnmap
  • PR-8858 - KAFKA-10153: Error Reporting in Connect Documentation
  • PR-8917 - KAFKA-10180: Fix security_config caching.
  • PR-8905 - KAFKA-10173: Fix suppress changelog binary schema compatibility
  • PR-8926 - KAFKA-10166: always write checkpoint before closing an (initialized) task
  • PR-8924 - KAFKA-10198: guard against recycling dirty state
  • PR-8900 - KAFKA-10169: swallow non-fatal KafkaException and don’t abort transaction during clean close
  • PR-8887 - KAFKA-10135: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager…
  • PR-8896 - KAFKA-10185: Restoration info logging
  • PR-8890 - KAFKA-9891: add integration tests for EOS and StandbyTask
  • PR-8822 - KAFKA-10113; Specify fetch offsets correctly in LogTruncationException
  • PR-8876 - KAFKA-10167: use the admin client to read end-offset
  • PR-8841 - KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition
  • PR-8853 - KAFKA-10147 MockAdminClient#describeConfigs(Collection<ConfigResource…
  • PR-8882 - KAFKA-10165: Remove Percentiles from e2e metrics
  • PR-8856 - KAFKA-10150: task state transitions/management and committing cleanup
  • PR-8880 - KAFKA-10169: Error message when transit to Aborting / AbortableError / FatalError
  • PR-8872 - Fix log message for transition from standby to active
  • PR-8873 - Avoid WARN log message when re-init from checkpoint skipped
  • PR-8849 - KAFKA-10144: clean up corrupted standby tasks before attempting a commit
  • PR-8764 - KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes
  • PR-8787 - KAFKA-10085: correctly compute lag for optimized source changelogs
  • PR-8818 - KAFKA-10086: Integration test for ensuring warmups are effective
  • PR-8069 - KAFKA-9374: Make connector interactions asynchronous
  • PR-8828 - KAFKA-9216: Enforce that Connect’s internal topics use compact cleanup policy
  • PR-8829 - KAFKA-10115: Incorporate errors.tolerance with the Errant Record Reporter
  • PR-8455 - KAFKA-9845: Warn users about using config providers with plugin.path property
  • PR-8502 - KAFKA-9066: Retain metrics for failed tasks
  • PR-8847 - KAFKA-7833: Add missing test
  • PR-8453 - KAFKA-9841: Revoke duplicate connectors and tasks when zombie workers return with an outdated assignment
  • PR-8663 - KAFKA-9985: Sink connector may exhaust broker when writing in DLQ
  • PR-8833 - KAFKA-9441: remove prepareClose() to simplify task management
  • PR-8825 - KAFKA-7833: Add StateStore name conflict check in InternalTopologyBuilder
  • PR-8630 - KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation
  • PR-8775 - KAFKA-10079: improve thread-level stickiness
  • PR-7384 - KAFKA-8938: Improve allocations during Struct validation in ConnectSchema
  • PR-8827 - KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used
  • PR-8805 - KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group
  • PR-8783 - KAFKA-10063 UnsupportedOperation when querying cleaner metrics after shutdown
  • PR-8803 - KAFKA-10102: update ProcessorTopology instead of rebuilding it
  • PR-8676 - KAFKA-10005: Decouple RestoreListener from RestoreCallback
  • PR-8684 - KAFKA-10012 Reducing memory overhead associated with strings in SelectorMetrics
  • PR-8270 - KAFKA-9216: Enforce internal config topic settings for Connect workers during startup
  • PR-8820 - KAFKA-10097: Internalize checkpoint data
  • PR-8804 - KAFKA-9851: Revoking Connect tasks due to connectivity issues should also clear the running assignment
  • PR-8486 - KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable
  • PR-8135 - KAFKA-9570: Define SSL configs in all worker config classes, not just distributed
  • PR-8814 - KAFKA-10111: Make SinkTaskContext.errantRecordReporter() a default method
  • PR-8811 - KAFKA-10110: Corrected potential NPE when null label value added to KafkaMetricsContext
  • PR-8776 - KAFKA-9441: Improve Kafka Streams task management
  • PR-8759 - KAFKA-10066: TestOutputTopic should pass record headers into deserializers
  • PR-8755 - KAFKA-10069: Correctly remove user-defined “predicate” and “negate” configs from transformation properties
  • PR-8644 - KAFKA-9313: Set use_all_dns_ips as the new default for client.dns.lookup (KIP-602)
  • PR-8784 - KAFKA-9788: Use distinct names for transaction and group load time sensors
  • PR-8792 - KAFKA-10089 The stale ssl engine factory is not closed after reconfigure
  • PR-8782 - KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends
  • PR-8785 - KAFKA-10084: Fix EosTestDriver end offset
  • PR-8786 - KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests
  • PR-8737 - KAFKA-9945: TopicCommand should support –if-exists and –if-not-exists when –bootstrap-server is used
  • PR-8695 - KAFKA-9320: Enable TLSv1.3 by default (KIP-573)
  • PR-8777 - KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment
  • PR-8706 - KAFKA-10030 allow fetching a key from a single partition
  • PR-8760 - Kafka-10064 Add documentation for KIP-571
  • PR-8668 - KAFKA-9987: optimize sticky assignment algorithm for same-subscription case
  • PR-7956 - KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing
  • PR-8769 - KAFKA-10074: Improve performance of matchingAcls
  • PR-8723 - KIP-569-KAFKA-9494: DescribeConfigsResponse - include additional metadata information
  • PR-8739 - KAFKA-10056; Ensure consumer metadata contains new topics on subscription change
  • PR-8705 - KAFKA-10029; Don’t update completedReceives when channels are closed to avoid ConcurrentModificationException
  • PR-8238 - KAFKA-9130: KIP-518 Allow listing consumer groups per state
  • PR-8248 - KAFKA-9501: convert between active and standby without closing stores
  • PR-8699 - KAFKA-9673: Filter and Conditional SMTs
  • PR-8720 - KAFKA-9971: Error Reporting in Sink Connectors (KIP-610)
  • PR-8736 - KAFKA-9802; Increase transaction timeout in system tests to reduce flakiness
  • PR-8731 - KAFKA-10050: kafka_log4j_appender.py fixed for JDK11
  • PR-8589 - KAFKA-9146: KIP-571 Add option to force delete active members in StreamsResetter
  • PR-8691 - KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter
  • PR-8709 - KAFKA-9952; Remove immediate fetch completion logic on high watermark updates
  • PR-8692 - KAFKA-10018: Change command line tools from /bin/sh to /bin/bash
  • PR-8697 - KAFKA-9983: KIP-613, add INFO level e2e latency metrics
  • PR-8376 - KAFKA-9724 Newer clients not always sending fetch request to older brokers
  • PR-8504 - KAFKA-9298: reuse mapped stream error in joins
  • PR-8716 - KAFKA-6145: KIP-441: Fix assignor config passthough
  • PR-6824 - KAFKA-7523: Add ConnectedStoreProvider to Processor API
  • PR-8221 - KAFKA-9561: update task input partitions after rebalance
  • PR-7886 - KAFKA-9353: Add groupInstanceId to DescribeGroup for better visibility
  • PR-8722 - KAFKA-5295: Allow source connectors to specify topic-specific settings for new topics (KIP-158)
  • PR-8395 - Added doc for KIP-535 and updated it for KIP-562
  • PR-8118 - KAFKA-9472: Remove deleted Connect tasks from status store
  • PR-2604 - KAFKA-4794: Add access to OffsetStorageReader from SourceConnector
  • PR-8698 - KAFKA-10022:console-producer supports the setting of client.id
  • PR-6284 - KAFKA-6755: Allow literal value for MaskField SMT
  • PR-8357 - KAFKA-9767: Add logging to basic auth rest extension
  • PR-8620 - KAFKA-9944: Added supporting customized HTTP response headers for Kafka Connect.
  • PR-8511 - KAFKA-9888: Copy connector configs before passing to REST extensions
  • PR-8654 - KAFKA-9931: Implement KIP-605 to expand support for Connect worker internal topic configurations
  • PR-8713 - KAFKA-6145: Add unit tests for assignments of only stateless tasks
  • PR-8696 - KAFKA-6145: KIP-441: Enforce Standby Task Stickiness
  • PR-8679 - KAFKA-10003: Mark KStream.through() as deprecated
  • PR-8658 - KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas
  • PR-8379 - KAFKA-9780: Deprecate commit records without record metadata
  • PR-8651 - kafkatest: Deploy VerifiableClient in constructor to avoid test timeouts
  • PR-8472 - KAFKA-9855 - return cached Structs for Schemas with no fields
  • PR-8608 - KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties
  • PR-8444 - KAFKA-8869: Remove task configs for deleted connectors from config snapshot
  • PR-7942 - KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect
  • PR-8661 - KAFKA-9603: Do not turn on bulk loading for segmented stores on stand-by tasks
  • PR-8671 - KAFKA-9859 / Add topics generated by KTable FK join to internal topic matching logic
  • PR-8681 - KAFKA-10010: Should make state store registration idempotent
  • PR-8682 - KAFKA-10011: Remove task id from lockedTaskDirectories during handleLostAll
  • PR-8673 - KAFKA-9992: EmbeddedKafkaCluster.deleteTopicAndWait not working with kafka_2.13
  • PR-8689 - KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173
  • PR-8675 - KAFKA-10004: ConfigCommand fails to find default broker configs without ZK
  • PR-8569 - KIP-551: Expose disk read and write metrics
  • PR-8678 - Update Gradle to 6.4.1
  • PR-8659 - KAFKA-9617 Replica Fetcher can mark partition as failed when max.message.bytes is changed
  • PR-8618 - KAFKA-9955: Prevent SinkTask::close from shadowing other exceptions
  • PR-8558 - KAFKA-8611 / KIP-221 documentation
  • PR-8670 - KAFKA-10001: Should trigger store specific callback if it is also a listener
  • PR-8090 - KAFKA-9537 - Cleanup error messages for abstract transformations
  • PR-8674 - KAFKA-9996: Upgrade zookeeper to 3.5.8
  • PR-8667 - KAFKA-9994: Handle task migrated inside corruption path
  • PR-8665 - KAFKA-9984 Should fail the subscription when pattern is empty
  • PR-8222 - KAFKA-9650: include human readable units in ms and bytes configs
  • PR-8588 - KAFKA-6145: KIP-441: Improve assignment balance
  • PR-313 - KAFKA-2641: Upgrade path for ZK authentication
  • PR-8550 - KAFKA-9850 Move KStream#repartition operator validation during Topology build process
  • PR-8648 - KAFKA-9966: add internal assignment listener to stabilize eos-beta upgrade test
  • PR-8596 - KAFKA-9821: persist followup rebalance in assignment and consolidate rebalance triggering mechanisms
  • PR-8647 - KAFKA-9669; Loosen validation of inner offsets for older message formats
  • PR-8254 - KIP-557: Add Emit On Change Support
  • PR-8626 - KAFKA-9921: explicit handling of null values with retainDuplicates
  • PR-8643 - KAFKA-9956: Authorizer APIs may be invoked more than once for a given request
  • PR-8632 - KAFKA-9972: Only commit tasks with valid states
  • PR-8628 - KAFKA-9942: Fixes ConfigCommand client quotas w/ default users.
  • PR-8613 - KAFKA-6145: Set HighAvailabilityTaskAssignor as default in streams_upgrade_test.py
  • PR-8230 - KAFKA-9667: Connect JSON serde strip trailing zeros
  • PR-8483 - KAFKA-9865: Expose output topic names from TopologyTestDriver
  • PR-8360 - KAFKA-9768: Fix handling of rest.advertised.listener config
  • PR-7950 - KAFKA-9419: Integer Overflow Possible with CircularIterator
  • PR-8602 - KAFKA-9947; Ensure proper shutdown of components in TransactionsBounceTest
  • PR-8591 - KAFKA-6342: Move workaround for JSON parsing of non-escaped strings
  • PR-8609 - KAFKA-9946; StopReplicaRequest deletePartition changes may cause premature topic deletion handling in controller
  • PR-8496 - KAFKA-9748: Add Streams eos-beta integration test
  • PR-8607 - KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
  • PR-8527 - Remove dependencies on deprecated –zookeeper command flags in junit tests
  • PR-8551 - KAFKA-9918 SslEngineFactory is NOT closed when channel is closing
  • PR-8565 - KAFKA-9798: Send one round synchronously before starting the async producer
  • PR-8533 - KAFKA-9589: Fixed bug in V2 log validator tests
  • PR-8577 - use appropriate fn for readability. (maybe)
  • PR-8586 - KAFKA-9939; Fix overcounting delayed fetches in request rate metrics
  • PR-8595 - KAFKA-8410: Revert Part 1: processor context bounds (#8414)
  • PR-8597 - KAFKA-6145: KIP 441 remove balance factor
  • PR-8590 - KAFKA-6145: Remove check to reuse previous assignment
  • PR-8204 - KAFKA-9633: Ensure ConfigProviders are closed
  • PR-8554 - KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd
  • PR-8567 - KAFKA-9652: Fix throttle metric in RequestChannel and request log due to KIP-219
  • PR-8559 - KAFKA-9922:Update examples README
  • PR-8442 - KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses
  • PR-8578 - KAFKA-9875: Make integration tests more resilient
  • PR-8582 - KAFKA-9932: Don’t load configs from ZK when the log has already been loaded
  • PR-8574 - KAFKA-9925: decorate pseudo-topics with app id
  • PR-8580 - KAFKA-9832: fix attempt to commit non-running tasks
  • PR-8540 - KAFKA-9127: don’t create StreamThreads for global-only topology
  • PR-8568 - KAFKA-9176: Retry on getting local stores from KafkaStreams
  • PR-8548 - KAFKA-9823: Follow-up, check state for handling commit error response
  • PR-8541 - KAFKA-6145: KIP-441: Add TaskAssignor class config
  • PR-8564 - KAFKA-9921: disable caching on stores configured to retain duplicates
  • PR-8566 - Fix minor code issue
  • PR-8525 - KAFKA-9885; Evict last members of a group when the maximum allowed is reached
  • PR-8524 - KAFKA-9866: Avoid election for topics where preferred leader is not in ISR
  • PR-8509 - KAFKA-9839: Broker should accept control requests with newer broker epoch
  • PR-8224 - KAFKA-9704: Fix the issue z/OS won’t let us resize file when mmap.
  • PR-8326 - KAFKA-8639: Replace AddPartitionsToTxn with Automated Protocol
  • PR-8445 - KAFKA-9823: Remember the sent generation for the coordinator request
  • PR-8536 - KAFKA-9883: Add better error message when REST API forwards a request and leader is not known
  • PR-8537 - KAFKA-9907: Switch default build to Scala 2.13
  • PR-8464 - KAFKA-9852: Change the max duration that calls to the buffer pool can block from 2000ms to 10ms
  • PR-8417 - KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy
  • PR-8530 - KAFKA-9388: Refactor integration tests to always use different application ids
  • PR-8482 - KAFKA-9863: update the deprecated –zookeeper option in the documentation into –bootstrap-server
  • PR-8497 - KAFKA-6145: KIP-441 Build state constrained assignment from balanced one
  • PR-8520 - Add explicit grace period to tumbling window example
  • PR-8522 - KAFKA-9868: Reduce transaction log partitions for embed broker
  • PR-8523 - Adding github whitelist
  • PR-8475 - KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence
  • PR-8501 - KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test
  • PR-8448 - KAFKA-9796; Broker shutdown could be stuck forever under certain conditions
  • PR-8471 - KAFKA-9854 Re-authenticating causes mismatched parse of response
  • PR-8476 - KAFKA-9838; Add log concurrency test and fix minor race condition
  • PR-8378 - KAFKA-9779: Add Stream System Test 2.5
  • PR-6210 - KAFKA-7885: TopologyDescription violates equals-hashCode contract.
  • PR-8092 - KAFKA-9540: Move “Could not find the standby task while closing it” log to debug level
  • PR-8256 - KAFKA-9675: Fix bug that prevents RocksDB metrics to be updated
  • PR-8440 - KAFKA-9832: extend Kafka Streams EOS system test
  • PR-8492 - KAFKA-9739: Fix for 2.5 branch
  • PR-8403 - KAFKA-9797; Fix TestSecurityRollingUpgrade.test_enable_separate_interbroker_listener
  • PR-8477 - KAFKA-9864: avoid expensive QuotaViolationException usage
  • PR-8469 - [KAFKA-9826] Handle an unaligned first dirty offset during log cleaning.
  • PR-8470 - KAFKA-8611 / Refactor KStreamRepartitionIntegrationTest
  • PR-8257 - KAFKA-9539; Add leader epoch in StopReplicaRequest (KIP-570)
  • PR-8353 - KAFKA-9764: Remove stream simple benchmark suite
  • PR-8457 - KAFKA-9842; Add test case for OffsetsForLeaderEpoch grouping in Fetcher
  • PR-8077 - KAFKA-9583: use topic-partitions grouped by node to send OffsetsForLeaderEpoch requests
  • PR-8474 - KAFKA-9853: Improve performance of Log.fetchOffsetByTimestamp
  • PR-7015 - KAFKA-8436: use automated protocol for AddOffsetsToTxn
  • PR-8399 - KAFKA-3720: Change TimeoutException to BufferExhaustedException when no memory can be allocated for a record within max.block.ms
  • PR-8443 - KAFKA-9832: Extend Streams system tests for EOS-beta
  • PR-8436 - KAFKA-6145: KIP-441 avoid unnecessary movement of standbys
  • PR-7170 - KIP-221 / Add KStream#repartition operation
  • PR-7844 - KAFKA-9309: Add the ability to translate Message classes to and from JSON
  • PR-8441 - KAFKA-9748: Extend Streams integration tests for EOS beta
  • PR-8212 - KAFKA-9642: Change “BigDecimal(double)” constructor to “BigDecimal.valueOf(double)”
  • PR-8317 - KAFKA-9691: Fix NPE by waiting for reassignment request
  • PR-8315 - KAFKA-9433: Use automated protocol for AlterConfigs request and response
  • PR-8425 - KAFKA-6145: KIP-441 Move tasks with caught-up destination clients right away
  • PR-8451 - KAFKA-9835; Protect FileRecords.slice from concurrent write
  • PR-8409 - KAFKA-6145: KIP-441 Pt. 6 Trigger probing rebalances until group is stable
  • PR-8338 - KAFKA-8890: KIP-519- Make SSL context/engine configuration extensible
  • PR-8405 - KAFKA-9801: Still trigger rebalance when static member joins in CompletingRebalance phase
  • PR-8414 - KAFKA-8410: Part 1: processor context bounds
  • PR-8423 - KAFKA-9818: improve error message to debug test
  • PR-8420 - KAFKA-9815; Ensure consumer always re-joins if JoinGroup fails
  • PR-8371 - KAFKA-9753: A few more metrics to add
  • PR-8402 - KAFKA-9793: Expand the try-catch for task commit in HandleAssignment
  • PR-8383 - KAFKA-9784: Add OffsetFetch to group concurrency test
  • PR-8422 - KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator
  • PR-8418 - KAFKA-9807; Protect LSO reads from concurrent high-watermark updates
  • PR-8408 - KAFKA-9810: Document Connect Root REST API on /
  • PR-8412 - KAFKA-9750; Fix race condition with log dir reassign completion
  • PR-8411 - KAFKA-9812: fix infinite loop in test code
  • PR-8400 - KAFKA-9739: Fixes null key changing child node
  • PR-8375 - KAFKA-9776: Downgrade TxnCommit API v3 when broker doesn’t support
  • PR-8407 - KAFKA-9809: Shrink transaction timeout for streams
  • PR-8359 - KAFKA-9778: Add validateConnector functionality to the EmbeddedConnectCluster
  • PR-8337 - KAFKA-6145: Pt. 5 Implement high availability assignment
  • PR-8331 - KAFKA-9748: extend EosIntegrationTest for EOS-beta
  • PR-7755 - KAFKA-9233: Fix IllegalStateException in Fetcher retrieval of beginni…
  • PR-8269 - KAFKA-9659: Add more log4j when updating static member mappings
  • PR-8356 - KAFKA-9441: Cleanup Streams metrics for removed task commit latency metrics
  • PR-8370 - KAFKA-9753: Add active tasks process ratio
  • PR-8389 - KAFKA-9777; Remove txn purgatory to fix race condition on txn completion
  • PR-8279 - KAFKA-9706: Handle null keys/values in Flatten transformation
  • PR-8367 - KAFKA-9719: Streams with EOS-beta should fail fast for older brokers
  • PR-8388 - KAFKA-9759: Add doc change for KIP-562
  • PR-8350 - KAFKA-9760: Add KIP-447 protocol change to upgrade notes
  • PR-8382 - KAFKA-9685: PT2, avoid unnecessary set creation in ACL matching
  • PR-8164 - KAFKA-9600: EndTxn should enforce strict epoch checking if from client
  • PR-8368 - KAFKA-9770: Close underlying state store also when flush throws
  • PR-8369 - KAFKA-9771: Port patch for inter-worker Connect SSL from Jetty 9.4.25
  • PR-8358 - KAFKA-9756: Process more than one record of one task at a time
  • PR-8320 - KAFKA-8470: State change logs should not be in TRACE level
  • PR-8280 - KAFKA-9707: Fix InsertField.Key should apply to keys of tombstone records
  • PR-8332 - KAFKA-9688: kafka-topic.sh should show KIP-455 adding and removing replicas
  • PR-8346 - KAFKA-9373: Reduce shutdown time by avoiding unnecessary loading of indexes
  • PR-6741 - Improve performance of checkpointHighWatermarks, patch 1/2
  • PR-8138 - KAFKA-9573: Fix JVM options to run early versions of Kafka on the latest JVMs
  • PR-8343 - KAFKA-9758: Doc changes for KIP-523 and KIP-527
  • PR-8290 - KAFKA-9677: Fix consumer fetch with small consume bandwidth quotas
  • PR-8339 - KAFKA-9752; New member timeout can leave group rebalance stuck
  • PR-8285 - KAFKA-9700:Fix negative estimatedCompressionRatio issue
  • PR-8287 - KAFKA-9711 The authentication failure caused by SSLEngine#beginHandsh…
  • PR-8334 - KAFKA-6145: Add balanced assignment algorithm
  • PR-8336 - KAFKA-9749: TransactionMarkerRequestCompletionHandler should treat KAFKA_STORAGE
  • PR-8327 - KAFKA-9743: Catch commit offset exception to eventually close dirty tasks
  • PR-7939 - KAFKA-9407: Return Immutable List from SchemaSourceTask
  • PR-8330 - KAFKA-9742: Fix broken StandbyTaskEOSIntegrationTest
  • PR-8226 - KAFKA-9651: Fix ArithmeticException (÷ by 0) in Partitioner impls
  • PR-8318 - KAFKA-9451: Enable producer per thread for Streams EOS
  • PR-8205 - KAFKA-9634: Add note about thread safety of ConfigProvider
  • PR-8252 - KAFKA-6145: Pt 2.5 Compute overall task lag per client
  • PR-8325 - KAFKA-9741: Update ConsumerGroupMetadata before calling onPartitionsRevoked()
  • PR-8324 - KAFKA-9701 (fix): Only check protocol name when generation is valid
  • PR-8307 - KAFKA-9727: cleanup the state store for standby task dirty close and check null for changelogs
  • PR-8262 - KAFKA-6145: Add constrained balanced assignment algorithm
  • PR-8319 - KAFKA-9734: Fix IllegalState in Streams transit to standby
  • PR-8155 - KAFKA-9553: Improve measurement for loading groups and transactions
  • PR-8244 - KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API
  • PR-8223 - KAFKA-9654 ReplicaAlterLogDirsThread can’t be created again if the pr…
  • PR-8284 - KAFKA-9225: Bump rocksdb 5.18.3 to 5.18.4
  • PR-8218 - KAFKA-9441: Unify committing within TaskManager
  • PR-7972 - KAFKA-9435: DescribeLogDirs automated protocol
  • PR-7039 - KAFKA-8618: Replace Txn marker with automated protocol
  • PR-8299 - KAFKA-9568: enforce rebalance if client endpoint has changed
  • PR-8313 - KAFKA-5604: Remove the redundant TODO marker on the Streams side
  • PR-8253 - KAFKA-9656: Return COORDINATOR_NOT_AVAILABLE for older producer clients
  • PR-8161 - KAFKA-9047: Making AdminClient group operations respect retries and backoff
  • PR-7936 - KAFKA-9404: Use ArrayList instead of LinkedList in Sensor Class
  • PR-8260 - KAFKA-9625: Fixing IncrementalAlterConfigs with respect to Broker Configs
  • PR-8278 - KAFKA-8803: Remove timestamp check in completeTransitionTo
  • PR-8289 - KAFKA-9712: Catch and handle exception thrown by reflections scanner
  • PR-8236 - KAFKA-9670: Reduce allocations in Metadata Response preparation
  • PR-8083 - KIP-546 (1/2): Implements describeClientQuotas() and alterClientQuotas().
  • PR-8267 - KAFKA-6647: Do note delete the lock file while holding the lock
  • PR-8246 - KAFKA-6145: Pt 2. Include offset sums in subscription
  • PR-8293 - KAFKA-9715: Eliminate reference to interBrokerProtocolVersion in TransactionStateManager
  • PR-8294 - KAFKA-9718; Don’t log passwords for AlterConfigs in request logs
  • PR-7957 - KAFKA-8768: DeleteRecords request/response automated protocol
  • PR-8265 - KAFKA-9657: Throw upon offset fetch unsupported stable flag protocol
  • PR-8266 - KAFKA-9695; Handle null config values for createTopics, alterConfigs
  • PR-8272 - KAFKA-9701: Add more debug log on client to reproduce the issue
  • PR-8177 - KAFKA-9605: Do not attempt to abort batches when txn manager is in fatal error
  • PR-7876 - KAFKA-9344:Logged consumer config does not always match actual config…
  • PR-8232 - KAFKA-9658: Fix removing user quotas
  • PR-8255 - KAFKA-9686 MockConsumer#endOffsets should be idempotent
  • PR-8235 - KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode
  • PR-283 - KAFKA-2391: add timeout to KafkaConsumer blocking calls
  • PR-282 - KAFKA-2428
  • PR-8220 - KAFKA-9645: Fallback to unsubscribe during Task Migrated
  • PR-8228 - KAFKA-9663: Doc some null returns in KafkaStreams
  • PR-284 - KAFKA-2620: Introduce Scalariform
  • PR-8233 - KAFKA-9668: Iterating over KafkaStreams.getAllMetadata() results in ConcurrentModificationException
  • PR-279 - KAFKA-2479: Transient exception
  • PR-8227 - KAFKA-9662: Wait for consumer offset reset in throttle test to avoid losing early messages
  • PR-8121 - KAFKA-6145: Pt 1. Bump protocol version and encode task lag map
  • PR-8213 - KAFKA-9615: Clean up task/producer create and close
  • PR-8229 - KAFKA-9661: Propagate includeSynonyms option to AdminClient in ConfigCommand
  • PR-254 - KAFKA-2588 ReplicaManager partitionCount metric should actually be re…
  • PR-277 - Kafka-2587: Only notification handler will update the cache and all verifications will use waitUntilTrue.
  • PR-8209 - KAFKA-9632; Fix MockScheduler synchronization for safe use in Log/Partition tests
  • PR-8186 - KAFKA-9618: Directory deletion failure leading to error task RocksDB open
  • PR-275 - KAFKA-2527; System Test for Quotas in Ducktape
  • PR-8208 - KAFKA-8895: delete all topics before recreating
  • PR-8197 - KAFKA-9573: Fix VerifiableProducer and VerifiableConsumer to work with older Kafka versions
  • PR-1286 - KAFKA-3618: Handle ApiVersionsRequest before SASL authentication
  • PR-8087 - KAFKA-9525: add enforceRebalance method to Consumer API
  • PR-8190 - KAFKA-9623: Keep polling until the task manager is no longer rebalancing in progress
  • PR-8187 - KAFKA-9620: Do not throw in the middle of consumer user callbacks
  • PR-273 - KAFKA-2517; Performance Regression post SSL implementation (zero copy)
  • PR-8168 - KAFKA-9607: Do not clear partition queues during close
  • PR-8169 - KAFKA-9610: Do not throw illegal state when remaining partitions are not empty
  • PR-8173 - KAFKA-9614: Not initialize topology twice in StreamTask
  • PR-8165 - KAFKA-9601: Stop logging raw connector config values
  • PR-8166 - KAFKA-9602: Close the stream internal producer only in EOS
  • PR-8062 - KAFKA-9498; Topic validation during the topic creation triggers unnecessary TopicChange events
  • PR-8029 - KAFKA-8147: Add changelog topic configuration to KTable suppress
  • PR-8167 - KAFKA-9533: Revert ValueTransform forwards null
  • PR-8009 - KAFKA-9308: Reworded the ssl part of the security documentation
  • PR-8159 - KAFKA-9599 create unique sensor to record group rebalance
  • PR-8145 - KAFKA-9581: Remove rebalance exception withholding
  • PR-8142 - KAFKA-9577: SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version
  • PR-8105 - KAFKA-9441: Add internal StreamsProducer
  • PR-263 - KAFKA 2578 Client Metadata internal state should be synchronized
  • PR-8143 - KAFKA-9582: Do not abort transaction in unclean close
  • PR-8111 - KAFKA-9206: throw a KafkaException when encountering CORRUPT_MESSAGE
  • PR-8058 - KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted
  • PR-8116 - KAFKA-9562: part 1: ignore exceptions while flushing stores in close(dirty)
  • PR-7337 - KAFKA-8910: Incorrect javadoc at KafkaProducer.InterceptorCallback#onCompletion
  • PR-8108 - KAFKA-9533: ValueTransform forwards null values
  • PR-7913 - KAFKA-9343: Add ps command for Kafka and zookeeper process on z/OS.
  • PR-8119 - KAFKA-9558: Fixing retry logic for getListOffsetsCalls
  • PR-8089 - KAFKA-6266: Repeated occurrence of WARN Resetting first dirty offset
  • PR-8125 - KAFKA-9515: Upgrade ZooKeeper to 3.5.7
  • PR-8106 - KAFKA-9319: Fix generation of CA certificate for system tests.
  • PR-8109 - KAFKA-9545: Fix subscription bugs from Stream refactoring
  • PR-8088 - KAFKA-9535: Update metadata upon retrying partitions for ListOffset
  • PR-7389 - KAFKA-8805: Bump producer epoch on recoverable errors
  • PR-8060 - KAFKA-9274: Gracefully handle timeout exception
  • PR-7781 - KAFKA-8904: Improve producer’s topic metadata fetching.
  • PR-8085 - KAFKA-9556: Fix two issues with KIP-558 and expand testing coverage
  • PR-8112 - KAFKA-9557: correct thread process-rate sensor to measure throughput
  • PR-8023 - KAFKA-8507 kip 499 Unify connection name flag for command line tool
  • PR-7674 - KAFKA-9106 make metrics exposed via jmx configurable
  • PR-7731 - KAFKA-9204: allow ReplaceField SMT to handle tombstone records
  • PR-7733 - KAFKA-9192: fix NPE when for converting optional json schema in structs
  • PR-8000 - KAFKA-9417: New Integration Test for KIP-447
  • PR-8031 - KAFKA-9447: Add new customized EOS model example
  • PR-8091 - KAFKA-6607: Commit correct offsets for transactional input data
  • PR-8053 - KAFKA-9499; Improve deletion process by batching more aggressively
  • PR-7955 - KAFKA-9423: Refine layout of configuration options on website and make individual settings directly linkable
  • PR-8015 - KAFKA-9500: Fix topology bug in foreign key joins
  • PR-8065 - KAKFA-9503: Fix TopologyTestDriver output order
  • PR-7996 - KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS
  • PR-8054 - KAFKA-9390: Make serde pseudo-topics unique
  • PR-8024 - KAFKA-9483; Add Scala KStream#toTable to the Streams DSL
  • PR-8040 - KAFKA-6607: Commit correct offsets for transactional input data
  • PR-8059 - KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent fields
  • PR-8061 - KAFKA-9517: Fix default serdes with FK join
  • PR-8033 - KAFKA-9487: Follow-up PR of Kafka-9445
  • PR-8081 - KAFKA-9523: Migrate BranchedMultiLevelRepartitionConnectedTopologyTest into a unit test
  • PR-8018 - KAFKA-9480: Fix bug that prevented to measure task-level process-rate
  • PR-8039 - KAFKA-9505: Only loop over topics-to-validate in retries
  • PR-8027 - KAFKA-7658: Follow up to original PR
  • PR-8057 - KAFKA-9507 AdminClient should check for missing committed offsets
  • PR-8056 - KAFKA-9519: Deprecating ZK for ConfigCommand
  • PR-8048 - KAFKA-9509: Fixing flakiness of MirrorConnectorsIntegrationTest.testReplication
  • PR-252 - KAFKA-2586; Enable SSL for inter-broker communication when SSL is enabled in tests
  • PR-255 - KAFKA-2593 Key value stores can use specified serializers and deserializers
  • PR-7770 - KAFKA-9261; Client should handle unavailable leader metadata
  • PR-7997 - KAFKA-9113: Clean up task management and state management
  • PR-8037 - KAFKA-9491; Increment high watermark after full log truncation
  • PR-8007 - KAFKA-9477 Document RoundRobinAssignor as an option for partition.assignment.strategy
  • PR-7568 - KAFKA-9074: Correct Connect’s Values.parseString to properly parse a time and timestamp literal
  • PR-8030 - KAFKA-9492; Ignore record errors in ProduceResponse for older versions
  • PR-8028 - KAFKA-9490: Fix generics for Grouped
  • PR-8026 - Bump trunk to 2.6.0-SNAPSHOT
  • PR-7725 - KAFKA-9027, KAFKA-9028: Convert create/delete acls requests/response to use generated protocol

ksqlDB

  • PR-6126 - fix: create /var/lib/kafka-streams directory on RPM installation
  • PR-6165 - fix: allow expressions in flat map (6.0.x)
  • PR-5952 - METRICS-1988: add confluent-telemetry directory to ksql-run-class
  • PR-5937 - fix: avoid losing cause of processing errors
  • PR-5895 - fix: adds a handler to gracefully shutdown
  • PR-5885 - fix: close query on invalid use of HTTP/2 with /query endpoint (6.0.x)
  • PR-5871 - fix: remove schema compat check if schema exists
  • PR-5844 - fix: Reuse KsqlClient instance for inter node requests (#5742)
  • PR-5837 - fix: always use the changelog subject in table state stores
  • PR-5786 - fix: better error on DECIMAL type with PROTOBUF format
  • PR-5818 - fix: Allow backported ALPN to be used and warn if no OpenSSL
  • PR-5783 - fix: SEC-1034: log4j migration to confluent repackaged version
  • PR-5788 - fix: circumvent KAFKA-10179 by forcing changelog topics for tables
  • PR-5763 - fix: ksqlDB should not truncate decimals
  • PR-5761 - support UDTFs on streams without keys
  • PR-5756 - fix: do not overwrite schemas from a CREATE STREAM/TABLE
  • PR-5730 - fix: make sure old query stream doesn’t block on close
  • PR-5684 - fix: join queries with SELECT * and WHERE fail
  • PR-5703 - fix: support CASE statements returning NULL
  • PR-5656 - fix: allow empty structs in schema inference
  • PR-5645 - fix: ensure only deserializable cmds are written to command topic
  • PR-5644 - fix: support GROUP BY with no source columns used
  • PR-5638 - feat: implements ARRAY_JOIN as requested in (#5028) (#5474)
  • PR-5617 - fix: don’t fail if broker does not support AuthorizedOperations
  • PR-5621 - fix: /inserts-stream endpoint now supports nested types
  • PR-5563 - feat: new split_to_map udf
  • PR-5559 - feat: add CHR UDF
  • PR-5597 - fix: fail AVRO/Protobuf/JSON Schema statements if SR is missing
  • PR-5591 - fix: make stream and column names case-insensitive in /inserts-stream
  • PR-5592 - fix: disallow requests to /inserts-stream if insert values disabled
  • PR-5576 - fix: Stream with no key should not fail if key has data
  • PR-5568 - fail aggregations that don’t select any value columns
  • PR-5516 - feat: reload TLS certificate without restarting server
  • PR-5560 - fix: set retention.ms to -1 instead of Long.MAX_VALUE
  • PR-5548 - feat: new UDFs for set-like operations on Arrays
  • PR-5552 - fix: improve print topic format detection
  • PR-5528 - feat: create MetricsContext for ksql metrics reporters
  • PR-5534 - feat: new CONCAT_WS UDF
  • PR-5531 - fix: inserts now support zero-valued decimals
  • PR-5536 - feat: new UDFs for working with Maps
  • PR-5546 - feat: new string UDFs LPad, RPad
  • PR-5523 - feat: Add encode udf
  • PR-5542 - feat: support TIMESTAMP being a key column
  • PR-5535 - feat: new UUID UDF
  • PR-5532 - fix: Prevent memory leaks caused by pull query logging
  • PR-5533 - feat: keys can now be named using the KEY syntax
  • PR-5529 - fix: deals with issue #5521 by adding more descriptive error message
  • PR-5513 - fix: update Concat UDF to new framework and make variadic
  • PR-5504 - feat: Adds UDF regexp_replace
  • PR-5501 - feat: Adds udf regexp_split_to_array
  • PR-5495 - feat: turn on snappy compression for produced data
  • PR-5515 - fix: Retry on connection closed
  • PR-5505 - feat: new UDFs for array max/min/sort
  • PR-5507 - feat: Adds UDF regexp_extract_all
  • PR-4909 - feat: add extra log messages for pull queries
  • PR-5506 - fix: allow dynamic construction of an ARRAY of STRUCTS with duplicate values #5436
  • PR-5461 - fix: KSQL does not accept queries when running QueryLimit - 1 queries
  • PR-5497 - build: reference plugin URL as property (MINOR)
  • PR-5478 - fix: Remove stacktrace from error message
  • PR-5435 - fix(client): fail queryID future if executeQuery() request fails (MINOR)
  • PR-5469 - fix: /inserts-stream endpoint now accepts complex types
  • PR-5477 - fix: Improve pull query error logging
  • PR-5470 - ST-1323: Add license file so sources get attached
  • PR-5459 - fix: avoid unnecessary warnings from PRINT
  • PR-5448 - feat(client): support (non-streaming) insert into in Java client
  • PR-5455 - fix: allow setting auto.offset.reset=latest on /query-stream endpoint
  • PR-5425 - fix: exclude window bounds from persistent query value & schema match
  • PR-5431 - fix: fail on WINDOW clause without matching GROUP BY
  • PR-5340 - feat: Expose Vert.x metrics
  • PR-5444 - Fix markdown for function references
  • PR-5385 - feat: INSTR function #881
  • PR-5412 - feat: introduce RegexClassifier for classifying errors via cfg
  • PR-5392 - feat: Pull Queries: QPS check utilizes internal API flag to determine if forwarded
  • PR-5355 - How-to guides: first set
  • PR-5391 - fix(client): expose query ID from executeQuery() return type
  • PR-5386 - fix: Block writer thread if response output buffer is full
  • PR-5374 - feat: expose JMX metric that classifies an error state
  • PR-5363 - feat: drop WITH(KEY) syntax
  • PR-5371 - feat(client): support push query termination in Java client
  • PR-5382 - MINOR: Use kafka.scala.version variable in pom file
  • PR-5380 - MINOR: Update mbknor-jackson-jsonschema_2.12 artifact to mbknor-jackson-jsonschema_2.13
  • PR-5381 - fix: Override Scala version in our build
  • PR-5360 - fix(client): Java client error handling
  • PR-5270 - fix: remove leading zeros when casting decimal to string
  • PR-5273 - feat: implement earliest_by_offset() UDAF
  • PR-4975 - fix: fix NPE in latest_by_offset if first element being processed has null…
  • PR-4782 - feat: Implement latest_by_offset() UDAF
  • PR-5212 - feat: Adds the ability have internal endpoints listen on ksql.internal.listener
  • PR-5332 - fix: Allow value delimiter to be specified for datagen
  • PR-5093 - Any key name
  • PR-5200 - feat(client): Java client with push + pull query support
  • PR-5287 - fix: allow structs in schema provider return types
  • PR-5205 - fix: /query-stream endpoint should serialize Struct (MINOR)
  • PR-5239 - fix: Move Cors handler in front of /chc handlers
  • PR-5124 - Backport array len
  • PR-5213 - fix: use sr’s jackson-jsonschema version
  • PR-5158 - feat: Remove dependency to ksqldb-common
  • PR-5198 - fix: use schema in annotation as schema provider if present
  • PR-5193 - feat: Do not allow access to new streaming endpoints using HTTP1.x
  • PR-5189 - build: pin streams to 5.5.0 instead of 5.5.0-SNAPSHOT
  • PR-5162 - bug: stop multi-way joins failing on ambiguous column errors
  • PR-5150 - feat: klip-14 - rowtime as pseduocolumn
  • PR-5145 - fix: do not allow implicit casting in UDAF function lookup
  • PR-5122 - Event-driven microservice tutorial
  • PR-5104 - feat: fail startup if command contains incompatible version
  • PR-5105 - feat: transient queries added to show queries output
  • PR-5137 - BREAKING CHANGE: minor: allow more precise configuration of rocks bounds
  • PR-5142 - fix: support quotes in explain statements (MINOR)
  • PR-5126 - feat: Implement cluster health check endpoints
  • PR-5123 - refactor: Migrate old websocket query endpoint
  • PR-5139 - fix: include lower-case identifiers among those that need quotes (#3723)
  • PR-5081 - feat: add multi-join expression support
  • PR-5075 - fix: replace ‘null’ in explain plan with correct op type (MINOR)
  • PR-5023 - fix: Improve error message for where/having type errors
  • PR-5064 - feat: support multi-way joins
  • PR-5054 - fix: add content type header to ported endpoints response (MINOR)
  • PR-5069 - fix: make endpoints available while waiting for precondition
  • PR-5048 - fix: reject requests to new API server if server is not ready
  • PR-5047 - fix: make endpoints available while waiting for precondition
  • PR-5059 - fix: Make sure internal client is configured for TLS
  • PR-4985 - fix: Don’t set chunked transfer encoding for HTTP2 proxy
  • PR-4974 - fix: Catch server startup exceptions
  • PR-4945 - feat: Support wildcards for unauthed paths
  • PR-4944 - feat: CORS support for Vert.x based ksqlDB
  • PR-4851 - feat: Migrate /ksql and /ksql/terminate endpoints to Vert.x
  • PR-5019 - fix: improve handling of NULLs
  • PR-5052 - fix: json creator for QueryStatusCount should match getter (MINOR)
  • PR-5029 - feat: add KsqlQueryStatus to decouple from KafkaStreams.State
  • PR-4921 - Two new tutorials: materialized views + streaming ETL pipelines
  • PR-5013 - feat: support more advanced suite of LIKE expressions
  • PR-4952 - Support aliases in PARTITION BY and GROUP BY
  • PR-5002 - feat: speed up restarts by not building topologies for terminated queries
  • PR-4986 - Distinguish keys for tables by naming them PRIMARY KEY
  • PR-4875 - feat: scatter gather query status from all servers in cluster for ‘SHOW QUERIES [EXTENDED]’ statement
  • PR-4993 - fix: Allows unclosed quote characters to exist in comments
  • PR-5012 - fix: allow trailing slash in listeners config (MINOR)
  • PR-5005 - fix: query id for TERMINATE should be case insensitive
  • PR-4977 - fix: generated aliases for struct field access no longer require quoting
  • PR-5006 - Remove unused / duplicate methods (MINOR)
  • PR-4951 - feat: Adds rate limiting to pull queries
  • PR-4972 - fix: When pull query metrics enabled, throws NPE
  • PR-4956 - fix: output valid multiline queries when running SHOW QUERIES
  • PR-4966 - fix: Pass correct request configs when making pull query request
  • PR-4942 - fix: do not allow grouping sets
  • PR-4926 - fix: throw error when column does not exist during INSERT INTO
  • PR-4940 - fix: do not allow GROUP BY and PARTITION BY on boolean expressions
  • PR-4908 - fix: Don’t wait for streams thread to be in running state
  • PR-4878 - fix: handle leap days correctly during timestamp extraction
  • PR-4893 - fix: Infer TLS based on scheme of server string
  • PR-4889 - fix: If startup hangs (esp on preconditions), shutdown server correctly
  • PR-4872 - fix: ‘drop (stream|table) if exists’ fails if source does not exist
  • PR-4827 - fix: avoid duplicate column name errors from auto-generated aliases
  • PR-4822 - fix: Stop PARTITION BY and UDTFs that fail from terminating the query
  • PR-4850 - feat: expand IFNULL to support all types
  • PR-4859 - fix: Filter out hosts with no lag info by default
  • PR-4868 - test: add test coverage to ensure that historical commands from previous versions always work in the current version
  • PR-4837 - fix: Returns empty lag info for a dead host rather than last received lags
  • PR-4867 - fix: avoid long possible format lists on PRINT TOPIC for nulls
  • PR-4829 - feat: add COALESCE function
  • PR-4653 - fix: CommandRunner metric has correct metric displayed when thread dies
  • PR-4823 - fix: do not filter out rows where PARTITION BY resolves to null
  • PR-4814 - fix: use describeTopics() for Kafka healtcheck probe
  • PR-4771 - feat: add internal request to KsqlRequestConfig and SessionProperties
  • PR-4816 - fix: do not lose old key values on repartition
  • PR-4830 - fix: add deserializer for SqlType
  • PR-4809 - fix: Removes unnecessary error logging for heartbeat since this is ex…
  • PR-4754 - fix: better error code on shutdown
  • PR-4805 - fix: only create processing log stream if it doesn’t exist
  • PR-4743 - fix: make CLI unit-tests platform independent
  • PR-4785 - fix: docker image directories in light of renames (MINOR)
  • PR-4608 - feat: Add metrics for pull queries endpoint
  • PR-4778 - feat: Security plumbing
  • PR-4777 - fix: add transfer-encoding:chunked header if Jetty omits this
  • PR-4733 - feat: support for tunable retention, grace period for windowed tables
  • PR-4728 - feat: add REGEXP_EXTRACT UDF
  • PR-4717 - fix: create schemas at topic creation
  • PR-4709 - fix: stop logging about command topic creation on startup if exists (MINOR)
  • PR-4725 - feat: add ARRAY_LENGTH UDF
  • PR-4713 - feat: Integrate new API with existing security framework
  • PR-4714 - test: avoid NPE if user puts input topic name in output
  • PR-4685 - feat: basic auth for new API server
  • PR-4681 - fix: fix the error message for failing to wait for CommandRunner to catch up in DistributingExecutor
  • PR-4632 - fix: change default exception handling for timestamp extractors to log-and-continue
  • PR-4675 - fix: change configOverrides back to streamsProperties
  • PR-4597 - fix: added special handling for forwarded pull query request
  • PR-4650 - fix: Return immutable list of streams metadata
  • PR-4643 - fix: idempotent terminate that can handle hung streams
  • PR-4489 - fix: csas/ctas with timestamp column is used for output rowtime
  • PR-4625 - fix: support partial schemas
  • PR-4509 - feat: display errors in CLI in red text
  • PR-4621 - fix: patch KafkaStreamsInternalTopicsAccessor as KS internals changed
  • PR-4606 - fix: StoreQueryParameters name change
  • PR-4573 - feat: add the topic name to deserialization errors
  • PR-4575 - feat: log groupby errors to processing logger
  • PR-4590 - feat: Simple perf tool for new API
  • PR-4581 - feat: Provide upper limit on number of push queries
  • PR-4511 - [SECURITY] Use HTTPS to resolve dependencies in Maven Build
  • PR-4512 - feat: Plug insert streams into backend
  • PR-4551 - feat: enhance PRINT TOPIC’s format detection

REST Proxy

  • PR-738 - KC-613: Remove share/java/kafka-rest dependencySet.
  • PR-737 - KC-613: Split share/java/kafka-rest in kafka-rest-bin and kafka-rest-lib.
  • PR-734 - CLIENTS-1659: Remove V1 API Kafka core dependency and associated Zookeeper Client configs
  • PR-731 - CLIENTS-1657: Explicitly depend on Guava.
  • PR-729 - METRICS-1988: add confluent-telemetry directory to kafka-rest-run-class
  • PR-723 - Refactor requests to fix go sdk generation bug and add parameterized servers to openapi-spec
  • PR-726 - Add [Consumer|Producer]Performance tools back
  • PR-722 - Add reassignment API to openapi spec.
  • PR-721 - Add ACL APIs to openapi spec.
  • PR-719 - Add consumers (admin) APIs to openapi spec.
  • PR-718 - CNKAF-1017: Update v3 error response format
  • PR-715 - Add config endpoints to openapi spec.
  • PR-714 - OpenAPI Spec for a small subset of the V3 APIs.
  • PR-712 - Move MetricsReporter Config extraction to rest-utils
  • PR-711 - CLIENTS-1613: Add configuration to enable/disable V2 and V3 APIs.
  • PR-710 - Remove cluster-wide topic configs.
  • PR-709 - KC-613: Use slf4j version from common pom.xml.
  • PR-707 - Change resource label resource.kafka.cluster.id to resource.cluster.id
  • PR-703 - Enable setting cluster_id on KafkaRestMetricsContext
  • PR-706 - CNKAF-951: Add catchingCompose utility to handle exceptions.
  • PR-704 - CLIENTS-1599: Add constructor accepting path to KafkaRestApplication.
  • PR-697 - CLIENTS-1597: Add relationship reassignment and refactor reassignment resource
  • PR-701 - Rename alter config UPDATE to SET.
  • PR-700 - Add Batch Alter (UPDATE/DELETE) Broker/Cluster/Topic configs.
  • PR-696 - Add LIST/GET Consumer Group/Consumer/Consumer Assignment.
  • PR-693 - Add search/create/delete ACLs APIs.
  • PR-687 - CNKAF-739: Reassignment endpoint for a given topic name
  • PR-685 - CNKAF-739 - Reassignments endpoint for given topic name and partition id
  • PR-692 - Remove AdminClientWrapper.
  • PR-691 - Migrate off JSON:API cleanup.
  • PR-682 - CNKAF-739: Add Rest endpoint for list all partition reassignments
  • PR-690 - Switch from application/vnd.api+json to application/json.
  • PR-689 - KIP-606 MetricsContext
  • PR-686 - Migrate off JSON:API.
  • PR-683 - KIP-226: Add cluster-wide dynamic default configs.
  • PR-678 - ST-1323: Add license file so sources get attached
  • PR-679 - Migrate internal entities to Google’s Auto-Value.
  • PR-680 - Add static factory to Cluster.
  • PR-677 - KIP-226: Add support for configs source and synonyms.
  • PR-675 - Give more unique names to the testing containers.
  • PR-674 - Open visibility of configurePre/PostResourceHandling.
  • PR-672 - Fix REST Proxy injection-based authentication.
  • PR-671 - Make ParititonData public
  • PR-670 - Add JsonCreator to v3 entities
  • PR-667 - Copy admin client config from REST Proxy. Need for authentication tests.
  • PR-658 - Remove all methods from AdminClientWrapper.
  • PR-665 - CPKAFKA-4979: Remove non-determinism from searchReplicasByBroker.
  • PR-662 - Add find partition replicas by broker id endpoint.
  • PR-660 - CLIENTS-1505: Add broker config delete endpoint
  • PR-659 - CLIENTS-1504: Add update endpoint for broker config
  • PR-654 - Remove AdminClientWrapper.getBrokerIds() and add JSON:API errors.
  • PR-656 - Fix default Content-Type, when none or any (/) is provided.
  • PR-655 - Remove spurious topic/partitions existence test when producing.
  • PR-647 - Remove V1 APIs and all their dependencies.
  • PR-652 - CLIENTS-1501: Add broker configs endpoints
  • PR-617 - Fix compile error due to KIP-515 (ZK TLS)
  • PR-191 - SEC-1034: log4j migration to confluent repackaged version
  • PR-199 - Rest CORS option skip will now default to true
  • PR-195 - Add a config flag to skip OPTION when CORS is enabled
  • PR-193 - Removed OPTION omission for CORS
  • PR-192 - All original configs must be passed to MetricReporters
  • PR-190 - Use prefix to propagate metrics reporter configs
  • PR-186 - Extensible Application Metrics instance configuration
  • PR-180 - KIP-606 MetricsContext
  • PR-183 - ST-1323: Add license file so sources get attached
  • PR-182 - Add getter for Application’s Metrics object
  • PR-178 - Migrate rest-utils to use AK library instead of common-metrics
  • PR-177 - SEC-929-Customized HTTP response headers.
  • PR-172 - SEC-592: Customize Jetty request queue and threadpool size
  • PR-174 - Catch invalid format exception: #501

Schema Registry

  • PR-1624 - DG-857 Fix Avro converter for top-level bytes object
  • PR-1616 - DG-845 Have typeName override Object.class type; fixes #1615
  • PR-1611 - Correct License in Client library
  • PR-1598 - CP-644: Include CP jars required for Security and monitoring
  • PR-1601 - DG-820: Fix telemetry reporter handling of leadership changes
  • PR-1590 - DG-810: Explicitly add kafkastore.topic config to MetricsReporter instances
  • PR-1594 - DG-813 If use.latest.version is true, use on Protobuf dependencies
  • PR-1588 - DG-811 Allow JSON schema spec version to be specified
  • PR-1589 - DG-812 Derive file extension during download
  • PR-1587 - Upgrade to Kotlin 1.4.0
  • PR-1586 - DG-800 Cache ProtobufSchema using name in Converter
  • PR-1528 - SEC-1034: log4j migration to confluent repackaged version
  • PR-1580 - DG-789 Fix for CVE-2020-15824
  • PR-1579 - DG-788 Allow anyOf to be backward compat with oneOf/allOf
  • PR-1578 - DG-767 Calculate proper defaults for client ssl configs
  • PR-1576 - DG-767 Use passed in defaults for client ssl configs
  • PR-1577 - DG-774 restore ssl client configs
  • PR-1575 - DG-780 Add support for number with format for JSON Schema
  • PR-1569 - METRICS-1988: add confluent-telemetry directory to schema-registry-run-class
  • PR-1574 - DG-774 Avoid regenerating default names for Protobuf data
  • PR-1567 - DG-740 Convert connect defaults for JSON Schema
  • PR-1563 - DG-713: Disable automatic setup of telemetry reporting
  • PR-1561 - DG-723 Enable defaults for JSON Schema
  • PR-1556 - DG-715: Change telemetry cluster label names
  • PR-1550 - DG-695 Normalize JSON ref schemas before comparing
  • PR-1548 - ST-3988: update swagger-core to 1.6.2
  • PR-1547 - ST-3988: update swagger-core to 1.6.2
  • PR-1538 - DG-670 Treat anyOf same as oneOf in JSON Schema converter
  • PR-1537 - DG-652 Fix missing negation in conditional
  • PR-1535 - DG-652 Handle imports during protobuf compat checks
  • PR-1531 - DG-628 Allow changes to id during json schema compat checks
  • PR-1530 - DG-607 Add lenient processing for numbers in oneof
  • PR-1523 - DG-607 Ensure fields for union are optional
  • PR-1520 - Handle null error stream in RestService
  • PR-1522 - DG-606 Add option to not generate oneof for nullable fields
  • PR-1517 - Upgrade maven-plugin dependencies to 3.6 in 5.4.x
  • PR-1510 - Add metric name test
  • PR-1509 - align naming conventions
  • PR-1508 - Add telemetry reporter SR config option
  • PR-1507 - Use config originals for retrieving TR enablement
  • PR-1493 - Use MetricsContext to set JMX prefix
  • PR-1502 - Split telemetry cluster label
  • PR-1500 - Only add TelemetryReporter if explicitly enabled in config
  • PR-1483 - Telemetry reporter integration
  • PR-1497 - DG-554 Pass refs when forwarding register requests
  • PR-1496 - DG-535 Handle recursive Protobuf schemas
  • PR-1495 - DG-402 Missed some places where dotted names can occur
  • PR-1492 - Fix NPE during tombstone metric update
  • PR-1485 - Add node information to SchemaRegistryProtocol
  • PR-1487 - DG-527 Omit defaults for proto 3 schemas (fixed in wire)
  • PR-1482 - DG-499 Fix JSON Schema compat checks for partial models
  • PR-1484 - DG-524 Catch NPE during Avro compat check
  • PR-1480 - DG-27 Use leader/follower terminology
  • PR-1477 - Allow custom update handlers; fixes #382
  • PR-1476 - Add unit test for mvn validate action
  • PR-1468 - Implement new Schema Registry Metrics
  • PR-1474 - Add validate action for SR maven plugin
  • PR-1473 - Add ability to use latest version during serialization
  • PR-1471 - DG-436 Pass existing Avro schema instead of regenerating
  • PR-1470 - Allow unknown properties while deserializing SchemaRegistryProtocol.Assignment
  • PR-1469 - Fix message name on schema when version is included; fixes #1467
  • PR-1466 - DG-450 Perform proper matching of subschemas
  • PR-1461 - DG-268 Deprecate ZK-based coordination
  • PR-1455 - Fix Scala 2.13 deprecations
  • PR-1450 - DG-400 Support enums using allOf in JSONSchema converter
  • PR-1451 - Upgrade swagger deps
  • PR-1449 - DG-402 Add support for packages in ProtobufConverter
  • PR-1440 - Fix null schema type
  • PR-1433 - use property for jackson-jsonschema
  • PR-1436 - feat: adds subjectName in the exception for incompatible schema
  • PR-1437 - Fix cyclic schema references
  • PR-1431 - Cherry-pick for permanent delete
  • PR-1426 - DG-377 Allow generated classes to use refs during serialization
  • PR-1427 - DG-399 Use envelope in JsonSchemaUtils.toObject
  • PR-1417 - Allow id to be passed to console prod/cons; fixes #160
  • PR-1422 - Fix cycles in Avro schemaEquals
  • PR-1420 - SEC-929–Added one testing for verifying a feature added to rest-utils repo.
  • PR-1416 - DG-399 Use envelope in JsonSchemaUtils.toObject
  • PR-1397 - DG-91 Schema permanent delete.
  • PR-1389 - DG-91 Ability to permanently delete schemas
  • PR-1369 - DG-267 - Fix Protobuf converter enum handling
  • PR-1368 - Performance: Use singleton ObjectMapper
  • PR-1344 - Enhanced serialization exception
  • PR-1334 - DG-237: Migrate SchemaRegistry CP KafkaMetrics to AK KafkaMetrics
  • PR-1326 - Add API to retrieve supported schema types

Commercial Features

Confluent Server

  • CPKAFKA-6013: Skip tier state presence validation for deleted partitions
  • KSTORAGE-1082: tier replication causes incorrect HWM after restore (#2507)
  • KSTORAGE-1086: Better validations to check if tiering is disabled
  • [CNKAF-1144] Confluent.operator.managed property (#2484)
  • [Backport]METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
  • KSTORAGE-1040: Better checks to prevent tiering from being disabled dynamically
  • METRICS-2159: use a loading cache to avoid expensive pattern matching calls
  • CPKAFKA-5882. Add copyDependantLibs dependency to ce-sbk and ce-broker-plugins module
  • KC-845: Ensure LogConfig includes broker defaults to pass schema config validation
  • CPKAFKA-5873 Don’t load Netty classes when using default SSL engine
  • CONFLUENT: Remove shading from ce-broker-plugins and ce-sbk
  • CPKAFKA-5851: Align joda-time version with common
  • CNKAF-863: Only showed broker removal status for given broker.
  • KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
  • CPKAFKA-5849: Don’t log password configs in request log filter
  • CONFLUENT: Remove unused jacksonDatabind version definition
  • CPKAFKA-5795: Change guava dependency to 28.1-jre
  • CPKAFKA-5796: Remove ZK dependency from ce-licensing
  • CPKAFKA-5835: Fix regression in cluster link compatibility test
  • METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
  • [CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub. (#2408)
  • backport METRICS-2098 exclude zstd/lz4 from telemetry-clients to avoid conflicting versions
  • METRICS-2090: Remove duplicate labels on metrics
  • METRICS-2097: Don’t capture ‘total_size` and ‘tier_size` metrics
  • KC-637 cherry pick to 6.0.x
  • KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms. (#2353)
  • SEC-866-6.0.x again
  • CONFLUENT: Remove the soak clients status reporter cronjob
  • METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
  • METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
  • [CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces (#2326)
  • CNKAF-1058; Ensure that Control Request Batching does not send LeaderAndIsrRequest with yet unknown broker(s)
  • KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA (#2309)
  • backport METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics (#2282)
  • CPKAFKA-5762: Fix topic deletion in system tests
  • CNKAF-1039: Update error for insufficient metrics to compute plan
  • KC-671: SBK system test with CL
  • METRICS-1988: add confluent-telemetry directory to kafka classpath
  • CONFLUENT: Bump cc-trogdor chart docker image to cc-trogdor:v0.207.0-5.3.0-ce-SNAPSHOT
  • [Cherry 2367] [SEC-1345] Read HTTP Reason phrase when connection closed
  • Revert “SEC-866 - Migrate audit log “other” category into “management” category. (#2330)”
  • backport METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar (#2332)
  • [CLIENTS-1633] Schedule MDS url refresher asynchronously
  • Increase root FS to 20GB
  • METRICS-1763, METRICS-1767 Configure telemetry metrics whitelist for Connect and MQTT Proxy Proactive Support (#2241)
  • SEC-866 - Migrate audit log “other” category into “management” category.
  • KC-391: Source topic deletion system test for CL
  • KAFKA-10282; Remove Log metrics immediately when deleting log (#2342)
  • CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only - cherry pick to 6.0
  • Rebalance Cluster System Test
  • CNKAF-1008: Add Broker/Remove Broker proposal logging
  • KC-633: Auth failures system test for Cluster Linking
  • [CHERRY-2308] Move kafka-http-server project into clients project
  • KC-638: System tests with tier storage enabled on destination
  • KC-754: Cluster link compatibility system tests
  • KC-631 Good path cluster linking test using SASL_SSL
  • backport METRICS-1992: Fix schema registry cluster id resource label (#2285)
  • CONFLUENT: Fix broken cluster link system tests in 6.0.x
  • CNKAF-1029: Ensure Executor state is reset after unexpected exceptions
  • KC-735: Improve cluster link addition.
  • CNKAF-1035: Roll cluster while a new broker is getting added
  • METRICS-1998: add rbac metrics to default metrics list (#2269)
  • backport METRICS-1993: add connect.cluster.id label (#2264)
  • Updated default topic replicas factor to 3 for audit log.
  • KC-741: Don’t include passwords in cluster link request logs
  • KC-736: Allow deletion of cluster links if mirrored topics are marked for deletion
  • KSTORAGE-896: fix tier deletion manager system test (#2256)
  • CNKAF-822: Cherry pick add broker system test to 6.0.x
  • KC-725: Backport of KC-725 to 6.0.x
  • backport METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist. (#2266)
  • backport METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
  • KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup (#2253)
  • KSTORAGE-864 Fetch requests to return NotLeaderForPartition during ULE recovery
  • CNKAF-819: Test SBK broker removal with roll
  • METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
  • Backporting fix for METRICS-1764 and METRICS-1766
  • update kafkatest template for balancer replication factor (for < 3-node clusters)
  • KC-638: Cluster linking system test with tiered storage enabled in source cluster
  • CNKAF-825: Introduce a system test for replica placement constraints
  • CNKAF-1014: Rename usages of SBK in logs to DataBalancer
  • CNKAF-613: Introduce a broker removal system test
  • METRICS-1894:Telemetry Reporter is missing labels configured in prope…
  • METRICS-1817: add include regex default value to Provider interface
  • METRICS-1938: upgrade jose4j version (#2219)–backport
  • METRICS-1929: use advertised listeners/host-port when deriving local producer config
  • KSTORAGE-892 Re add couple of lines of ULE code that got deleted duri…
  • Backporting config/events Telemetry Reporter changes along with all shading issues.
  • Added one more toJson method in protoserde for webhook alert
  • (backport) MMA-8432: fix wrong package name
  • CNKAF-1004: Don’t describe removed broker configs
  • METRICS-1895: Update telemetry client version
  • KC-705: Retry metadata topic consumer.position() until authorizer startup times out
  • METRICS-1918: bump gson from 2.8.5 -> 2.8.6
  • (backport) prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
  • KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
  • (Backport) include UnderMinIsrPartitionCount in the default broker metrics list
  • CPKAFKA-2852: CCloud metadata listener thread should not die on unexpected exception
  • SEC-1276 Allow empty resourceNames in CRNs
  • SEC-384: Use the correct log level for debug output
  • KC-634: Cluster link system tests with broker failures
  • CNKAF-964: Shade opencensus protobufs into SBK jar (#2155)
  • Add KafkaRest Metrics Provider
  • KSTORAGE-845 Check if tier topic exists already before calling AdminZkClient#createTopic
  • CNKAF-952: Update javadoc for removeBroker ConfluentAdmin API with InvalidBrokerRemovalException
  • CNKAF-851: log goals skipped during detection in one line
  • KC-630: Cluster link system test support with good path mirror test
  • CONFLUENT: Prevent log start offset overrun during retention
  • KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order (#2133)
  • Back port [METRICS-1815] Connect client metrics missing some resource level tag…
  • CNKAF-936: Remove dot-handling logic in metrics processor (#2121)
  • METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
  • Deprecate KafkaHttpServlet in favour of rest-utils Application. (#2135)
  • [METRICS-1844, METRICS-1839, METRICS-1756,METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
  • [CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle, handle ab…
  • METRICS-1801: align telemetry domain metric names
  • METRICS-1835 Shade confluent-serializers dependencies in confluent-metrics (#2111)
  • METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter (#2096)
  • METRICS-1810 use SBK topic replicas config if present (#2109)
  • fix: fixes exception in server shutdown due to metrics registration
  • METRICS-1809: add telemetry settings to default server.properties file
  • [CNKAF-927] Fix typo in server.properties. (#2090)
  • [CNKAF-924, CNKAF-926] Fix Databalancer Topic RF config, remove Teleme…
  • CONFLUENT: Added check for CL metrics for 6.0.x
  • CPKAFKA-4955: Fix failing testTopicDeletion
  • Removed two testing files accidentally added.
  • SEC-1130-Inject ConfluentAdmin from ce-kafka
  • backport METRICS-1820: use password object for telemetry api secret
  • CNKAF-893: Update SBK to work with secure Zookeeper
  • METRICS-1816, METRICS-1705 local telemetry exporter metrics whitelist configuration for SBK (#2031)
  • CONFLUENT: deleting extra topic config change in cluster linking test
  • CNKAF-909: Add direct dependency on opencensus for SBK (#2057)
  • KAFKA-10149: Allow auto preferred leader election when partitions are…
  • CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
  • SEC-1034: use confluent-log4j instead of log4j
  • Add RackAwareGoal to goal list
  • CNKAF-884: Remove unnecessary metrics (#2059)
  • Migrate MDS to run on KafkaHttpServer implementation
  • KC-505: integration tests for CL metrics for 6.0.x
  • METRICS-1717 use shared domain for jvm level metrics (#1949)
  • CNKAF-854: Disabling SBK should persist the broker removal cancellation status
  • [CNKAF-873] Don’t require network capacity configs. (#1996)
  • SEC-1183 - Treat empty authority in CRN pattern as a wildcard
  • CNKAF-882: Add proposal logging for add/remove
  • CNKAF-837: Add broker should be able to add removed brokers.
  • KC-393: Integration tests for mirroring compacted topics
  • Add inter broker listener endpoint information to HTTP server binder.
  • CNKAF-773: Do not throw an exception when the broker is not present in the cluster model
  • CNKAF-847: Complete removal callback when there are no proposals to execute
  • KSTORAGE-802: Allow specifying AWS credentials through configuration file
  • [CNKAF-848] Set preferred user SBK config in server.properties. (#1999)
  • [CNKAF-719] Add option to disable dynamic throttle setting when stati…
  • KSTORAGE-819: Force roll segments only for partitions eligible for tiering
  • KSTORAGE-799: reduce Log.lock contention in ArchiveTask segment selection logic
  • KSTORAGE-774: Fix active segment roll logic when roll.min.bytes setting is enabled
  • KSTORAGE-766: Ensure delete segment throttler does not block active segment rolls
  • KSTORAGE-769: Fix to remove the costly toSet conversions in disk throttler
  • KSTORAGE-758: Include ERROR status in TierPartitionStatus.isOpen()
  • KSTORAGE-595 NPE in TierDeletedPartitionsCoordinator after tier topic is reassigned / deleted from broker
  • KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util (#1906)
  • KC-611: Fix mirror topic create processing incorrect config.
  • KSTORAGE-547: state recovery injection tooling
  • CONFLUENT: Fix Confluent provider start future completion, add unit tests
  • [CNKAF-881] SBK: Don’t treat all new brokers as empty. (#2005)
  • SBK/CruiseControl: low utilization fix
  • CNAKF-833, CNKAF-842: Fix missing request rate metric and enable tele…
  • RackAwareGoal should allow for uneven replica distribution based on t…
  • CNKAF-788: Introduce proper API exceptions in the broker removal API
  • METRICS-1708: add telemetry version header for forward-compatibility (#1883)
  • CNKAF-791: Change data format of failedBrokers in ZK
  • CNKAF-807: Remove zkclient from gradle dependencies.
  • CNKAF-575: Remove IOtec:ZkClient dependency.
  • [CNKAF-776] Switch SBK Maven publish group to “io.confluent”
  • [METRICS-1609] dynamically add telemetry reporter at runtime
  • KC-389: Retry segment deletion in cluster link test to fix timing issue
  • KC-389: Cluster link integration tests with tier storage enabled
  • CPKAFKA-5207: Complete authorizer start future only when sufficient nodes are ready
  • CNKAF-843: Fix remove broker validation check
  • Persist the broker removal state and handle failover
  • CNKAF-773: Do not throw exception when there are no replicas to move on broker removal
  • CNKAF-428: Integration test for replica placement self-healing
  • Add retries to telemetry sampler integration test (#1922)
  • [CNKAF-803] Add a metric to track the number of pending add-broker requests
  • KC-72: Add topic listing for cluster links.
  • CPKAFKA-5207: Fail fast during broker start up if multiple brokers use same MDS URL
  • KC-536 bug fix to make offsets/acls configs dynamic (#1893)
  • CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
  • fix telemetry reporter reconfiguration
  • update ksql and streams provider
  • CNKAF-649: Handle broker restart during broker removal
  • KC-390: Cluster link authorization tests
  • backport [METRICS-1564] telemetry - derive interbroker client configs (#1816)
  • (Backport) make commit-id optional, update whitelist to include C3 metrics
  • SEC-1083 - AuditAdmin role’s scopeType = Cluster
  • KC-392: Integration tests for cluster link quotas on source and destination
  • KC-582: Add multi-tenant support for alterMirrors.
  • CNKAF-795: Ensure SBK clients have bootstrap.servers configured
  • Backport PR 1897 to 6.0.x
  • SEC-1083 - Add AuditAdmin role (#1861)
  • SEC-1039 - TopicSpec builder sets partitions and RF.
  • CNKAF-794: Properly wrap broker removal exceptions in listener and improve remove broker test
  • Removed disk throttling dependency on produce backpressure config
  • Jvm Collector for Telemetry Reporter
  • METRICS-1698 Setting default telemetry.enabled and exporter.enabled values to false.
  • KC-224: make leader epoch cache flush asynchronous when stamping new leader epoch
  • KC-537: apply topic-create-policy when auto-creating topics while fetching metadata
  • CNKAF-782: MetricsReporterSamplers are configured in an odd manner.
  • CONFLUENT: Fix cipher name for ChaCha20
  • CNKAF-428: Replica placement
  • METRICS-1152 document how to enable the telemetry-reporter in CPD
  • KC-534: Adds forceful cluster link deletion.
  • KCORE-54: Controller should accumulate metadata into fewer requests to mitigate handling delays
  • CNAKF-645: Add metrics sampler to consume from telemetry reporter
  • Generic telemetry reporter
  • KSTORAGE-649: Remove duplicate library dependencies accidentally included in merge.
  • [cnkaf-730] part 2: Completion callbacks for add and remove brokers
  • [CNKAF-730] Minimal addBrokers support for SBK
  • KC-207: Destination CL Metrics
  • METRICS-1362 Updating telemetry dependency version from 1.87 to 1.98 to set new content-type
  • KSTORAGE-649 and KSTORAGE-660: Support sts::AssumeRole and SSE-KMS
  • CNKAF-697 - High bandwidth throttles during cluster roll
  • Make ClusterLinkIntegrationTest run as integration test
  • CNKAF-760: Integrate api state persistence with remove broker
  • Upgrade cloudevents from v0.3 to v1.0
  • CC-8138: Add Connect feature to inject Confluent license properties into connector properties before startup
  • KC-571: Move tenant prefix to cluster link metadata instead of configs
  • CIAM-69 Use authorizer setting to configure CC RBAC roles
  • [CNKAF-759] Deal with non-Anomaly executions.
  • SEC-994: Update Cluster Registry doc
  • KC-387: Multi-tenant integration tests for security updates, offset migration
  • KSTORAGE-752: removed disk throttler dependency on logDir existence at server startup
  • KC-518: Use cluster link ID as primary key for cluster link metadata.
  • Refactor Event Logger to make it generic
  • KSTORAGE-699 Rationalize ArchiveTask/DeletionTask Exceptions
  • CNKAF-649: Introduce integration test for SBK’s AddBroker, RemoveBroker and DescribeBroker APIs
  • Add confluent metrics labels for connect metrics
  • CNKAF-649 - Compose the broker removal operation into multiple interruptible tasks
  • KSTORAGE-740: Add log closing time metrics
  • SEC-951: Upgraded bc-fips to 1.0.2
  • Add Confluent metrics tag Kafka server and client
  • KSTORAGE-730: Allow max.partition.fetch.bytes to be configurable for tier fetches
  • CNKAF-752: Store and fetch remove broker api state
  • KSTORAGE-746; Make timing in TierTopicListenersTest deterministic
  • CPKAFKA-4744; Fix log cleaner test failure caused by concurrent cleaning
  • KC-407: Micro-benchmark for ClusterLinkFetcherThread based on ReplicaFetcherThreadBenchmark
  • Introducing disk-usage based throttling for producers and replication followers
  • cherry-pick upstream KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter (#8691)
  • KCORE-142 new tests and bug fix for offset command config
  • [CNKAF-741] Handle duplicate metrics in the DataBalancerMetricsRegistry
  • Add domain to Context class
  • CNKAF-702: At SBK startup check for api state persistence topic
  • KC-382: Allow alter cluster configs for subset of configs in CCloud
  • KC-533: Allow headers to be injected into RBAC records
  • CIAM-64 Allow running other Providers with the MultiTenantAuthorizer
  • KSTORAGE-741: Deserialize errorOffsetAndEpoch from Header during scan and init
  • CNKAF-649 - Broker Removal operation with in-memory state
  • CNKAF-617: Complete the DescribeBrokerRemovals API handler
  • KSTORAGE-743: prevent tier.feature use with IBP < 2.4
  • METRICS-1555 Exposing proxy configs as top level telemetry configs and making them dynamically configurable
  • KCORE-142 - Offset Migration for Cluster Linking
  • [CNKAF-637] Config variable for auto-healing
  • KSTORAGE-425 Add tiered storage metrics to MetricsReporter
  • CONFLUENT: Bump major CCloud version #major
  • KC-370: Support Authentication events in ConfluentAuditLogProvider
  • CPKAFKA-3743; Consumer group tool does not work on internal endpoint
  • KSTORAGE-720: Always return tiered storage related configurations in DescribeConfigResponse
  • [CNKAF-742] Detect new, empty brokers in controller; call into SBK
  • KC-120: Integration Tests for ZK-Based ACLs for Periodic ACL Sync
  • CONFLUENT: Add PowerMockIgnore for Scram API in RBAC test to fix classloader issues in subsequent tests
  • SEC-959 Cluster Registry - Full Cluster Info CRUD Gatekeeping
  • KC-517: Update cluster link integration test to use admin client.
  • SEC-1015: Added kafka Metrics support to Confluent Authorizer, RBAC and AuditLog.
  • CNKAF-649: Revise SBK Executor’s locking strategy
  • KCORE-154: add tiered storage and CP to downgrade system test
  • KC-128: Validate source topic read access on mirror topic creation.
  • KCORE-329 (2/2): Adds ability to alter a topic’s mirror in TopicCommand.
  • Created Broker removal CLI
  • KSTORAGE-726: handle offset gaps in tier fetches
  • CPKAFKA-2789; Add support-metrics-fullcollector to ce-kafka
  • KC-506: Use cluster link tags for cluster link fetcher metrics
  • KSTORAGE-627: TierMetadataComparator tool for tier-topic recovery pipeline
  • KCORE 506: migrate secrets to Vault
  • Spelling fix in LdapGroupManager
  • KCORE-95: Encrypt sensitive cluster link configs stored in ZooKeeper
  • KCORE-437: Refactor ClusterLink managers to cleanly disable/enable
  • CONFLUENT: Improve performance of matchingAcls and AclSeqs.find
  • KCORE-49: Disables mirror topic partition and restricted config updates.
  • KCORE-485: Process mirror state during replication only on active destination leader
  • KCORE-487: Add AuthenticationEvent and update Audit log event schema
  • CONFLUENT: Introduce separate methods in KafkaApis for consumer and follower fetch handling
  • Update Netty to 4.1.50 and clean up logging of NettySslEngineFactory
  • CIAM-45 Allow RoleBindings at path scopes
  • CNKAF-636: SBK broker failure detection
  • KSTORAGE-627: TierPartitionForceRestore, TierPartitionState hotswap and tier task recovery
  • CNKAF-701: Refactor SampleStore topic existence check
  • Added tests for KafkaCruiseControl.
  • CNKAF-649: Add BrokerRemoval state machine
  • KSTORAGE-573: epoch state cache is flushed multiple times in recovery
  • [METRICS-1234] telemetry - add support for multiple exporters of the same type
  • KCORE-389: Add initial version of Audit API and update existing code
  • CNKAF-677: Remove ZK fallback for SBK and move methods from ExecutorUtils to Executor
  • CNKAF-684: Honor low utilization threshold settings for self-healing …
  • CIAM-28 Add ConfluentCloudCrnAuthority with special treatment of path
  • KSTORAGE-717: Filter all Confluent-specific topic configs from DescribeConfigsResponse
  • KSTORAGE-652: FileTierPartitionState base path is not updated correctly
  • CNKAF-699: Skip num cores or disk capacity update if node missing
  • KCORE-340: Propagate new partitions in source cluster to controller
  • KSTORAGE-653: re-add check_cluster_state check to tier upgrade test
  • KCORE-141: Tenant prefixed cluster links and topic mirroring for Cloud
  • KCORE-331: _confluent-license topic exponential backoff
  • fix some downstream validation issue
  • Add shutdown hook to KafkaHttpServlet interface
  • SEC-1008 Do not close null AdminClient
  • CONFLUENT: Remove ConfluentLeaderAndIsr and TierListOffset requests/responses
  • CNKAF-704: Remove tenant prefix from CreatePartition responses
  • CNKAF-314: Add DescribeBrokerRemovals{Request,Response} RPC and Admin Client API
  • CONFLUENT: Set Scala 2.13 as the default Scala version
  • KCORE-436: Client interceptor for destination tenant prefixing in cluster link clients
  • CIAM-25 Allow CRNs with Paths but no Clusters
  • emit deltas as cumulative metrics
  • CNKAF-649: (part 1/2) Introduce ShutdownManager for SBK broker removal
  • CNKAF-697 (part1): Increase min broker quota constants
  • CONFLUENT: Improve error message when cluster linking is disabled
  • KSTORAGE-412 Server side handling for the unclean leader flag from LeaderAndIsr re…
  • METRICS-1514 collect one-minute rates from yammer meters
  • Upgrade telemetry api to latest version
  • CONFLUENT: Enable fatal warnings with Scala 2.13 and fix warnings
  • CONFLUENT: Make createSslEngine(String, String) public again
  • KCORE-329: Alter mirrors API.
  • KCORE-426: Server should not resolve DNS on establishment of SASL connection
  • CPKAFKA-4923: Disable audit logging in RbacTest::test_ldap_to_rbac_authorizer_upgrade
  • CNKAF-632: RemoveBrokers api plumbing
  • switch downstream validation
  • [MINOR] Remove scoverage plugin from SBK build.
  • Bump Chart versions
  • Merge changes from Apache Kafka
  • CONFLUENT: Always close consumer in ConsumeBenchWorker
  • KCORE-438 (1/2): Remove topic read test on cluster link creation.
  • CNKAF-638: Introduce SBK internal RPC client
  • KCORE-41: Disable local produce requests for mirror topics.
  • CONFLUENT: Fix build warnings in new ACL sync code
  • CLIENTS-1476: Create Kafka HTTP Server interfaces and loading mechanism.
  • KCORE-99: Periodic ACL Migration
  • CNKAF-622: Properly handle dynamic config changes.
  • KCORE-435: Propagate cluster link create/delete using ZK change notification
  • KCORE-427: Add broker config to enable cluster linking feature
  • CONFLUENT: Don’t reach into java.lang.reflect.Field internals
  • KSTORAGE-683: Alter fencing logic for recovery work
  • CPKAFKA-4982 -Change Trogdor’s Helm chart to deploy in parallel.
  • KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
  • KCORE-167: Process replication failures with retry timeout to move partition to failed state
  • CONFLUENT: Fix locking during metadata restart for link reconfiguration
  • KSTORAGE-677: FIX: Reset archiver max lag whenever there is no lag
  • remove unnecessary labels from kafka server resource
  • CNKAF-629: Add InitiateShutdown Request
  • KCORE-55: Adds mirrorTopic and clusterLink to topic creation API.
  • KCORE-53: Dynamic security config update for inter-cluster connections
  • CNKAF-483: Added static and dynamic configs for request backpressure tuning
  • KSTORAGE-26: Fix non-deterministic TierEpochStateReplicationTest
  • CNKAF-593: Consolidate all thread creation in KafkaCruiseControlThreadFactory
  • KCORE-104: Fix truncation during unclean leader election for linked partition
  • CONFLUENT: Simplify Gauge definitions in Partition
  • CNKAF-610: Rename StartRebalance RPC and DrainBroker Admin API to RemoveBrokers
  • [CNKAF-606] Stop proposal execution on Cruise Control shutdown/SBK deactivation
  • [CNKAF-558] Handle SSL interbroker listeners for SBK clients.
  • CONFLUENT: Ensure license validator is closed in test to avoid impacting subsequent test
  • KSTORAGE-648: remove S3TierObjectStore multi-part log segment upload support
  • CNKAF-596: Add a test to verify SBK API access with RBAC
  • KSTORAGE-628: Implement TierPartitionFence injector tool for recovery
  • KSTORAGE-63: pt2, add TierTopicListeners, add getAndRemoveAll and tests
  • KSTORAGE-63: pt1, add TierPartitionForceRestore metadata
  • SEC-912 - Allow MDS to make changes to dynamic configs
  • KCORE-341: Fix permission type in authorizer debug log entry
  • SOAK: Revise soak workloads
  • [SEC-912] Allow MDS to make changes to dynamic configs
  • KSTORAGE-631: ducker up should not require GCS credentials to be activated in Dockerfile
  • CNKAF-577: Handle null config values when describing static throttle configs
  • CONFLUENT: Add support for Scala 2.13
  • WFLOW-51: Implement a beginShutdown hook which allows for triggering ControlledShutdown
  • KCORE-42 (4/4): Adds mirror SyncTopicsConfigs task.
  • KCORE-48: Broker implementation for cluster linking APIs.
  • KCORE-48 (partial): Adds ClusterLinkCommand.
  • KCORE-330: Allow brokers to complete start up with configured license
  • [CNKAF-594] Support broker shutdown for SBK
  • [CNKAF-492] Set some more config parameters for SBK
  • Update cloud Dockerfile to use JDK 14.0.1
  • CNKAF-313: Migrate SBK to use DataBalancerMetricsRegistry
  • CNKAF-549: Add metric for self healing enabled
  • Instructions for exporting metrics to Sandbox env
  • KSTORAGE-644: Separate fencing error metrics for primary & catchup consumers
  • CONFLUENT: Update JDK to 11.0.7 by bumping cc-base to 5.1.0
  • KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
  • KCORE-137 (partial): Expands per-topic cluster link state.
  • [SEC-895] ConfluentAuditLogProvider should handle reconfiguration wit…
  • CONFLUENT: Reduce allocations in requests via buffer caching
  • KAFKA-9864: Avoid expensive QuotaViolationException usage (#8477)
  • CNKAF-50: Add metrics for replication throttling
  • KCORE-51: Adds cluster link metadata object.
  • CNKAF-585; Multi tenant interceptor rejects create topic requests with NO_REPLICATION_FACTOR or NO_NUM_PARTITIONS when quotas are enabled
  • KCORE-316: Don’t load configs from ZK when the log has already been loaded
  • Fix MultiTenantTest
  • CPKAFKA-1836: Simplify forwarding scripts by just adding .sh suffix
  • CONFLUENT: add utility method to write public keys to pem files (#19)
  • CAAS-1960: Record authenticated tenant connection/disconnection metrics
  • KAFKA-7304: Limit connection accept rate to prioritize other channels (#29)
  • CONFLUENT: Add Aegis Proxy
  • CONFLUENT: Update cc-base to v2.3.0
  • Merge changes from apache kafka
  • CPKAFKA-2164: Ensure that we specify a charset when writing out and reading string bytes
  • CONFLUENT: Fix unused compiler warnings
  • [SEC-901] Speed up SameClusterTest
  • KCORE-42 (partial): Adds functions for merging source + mirror topics…
  • SEC-728 - MDS Cluster Registry - Enable Dynamic Configuration
  • KCORE-48 (partial): Adds ClusterLinkClientManager for managing admin …
  • Towards KCORE-165 (partial): Adds ClusterLinkScheduler for handling cluster l…
  • Towards KCORE-136: Cluster linking APIs client+RPC wiring.
  • [CNKAF-582] Handle topic creation during checkStartupComponents
  • CNKAF-372: Disk capacity sizing
  • CONFLUENT: Remove unnecessary allocations in hot matchingAclExists
  • METRICS-1415 fix collection of integer gauges
  • CNKAF-313: Adds a MetricsRegistry wrapper class for SBK
  • [CNKAF-370] Support dynamic enable/disable of SBK
  • [CNKAF-492] Adjust the default parameters of the KafkaCruiseControlConfig.
  • KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
  • KSTORAGE-628: Introduce RecoveryUtils class with recovery tooling related functions
  • CPKAFKA-4647: Disable audit logging in RbacTest::test_broker_failure test
  • KSTORAGE-628: Introduce TierPartitionFence event type + fencing logic
  • add AppInfo version+commitId to metrics messages
  • KSTORAGE-577: System test for fencing logic, along with CLI tool to trigger fencing
  • KSTORAGE-632: Better logging for replica fetcher state transitions for tiered storage
  • KSTORAGE-381 Tooling around TierMetadataValidator to use tier-validator as a cc-service
  • CNKAF-536: Reduce number of warnings when SBK creating clients
  • [METRICS-1361] telemetry-api -> opencensus-proto
  • CNKAF-537: Add drainbrokers adminClientAPI
  • CNKAF-562: Change log level for self-healing being triggered from WARN to INFO.
  • KSTORAGE-613: fix log start offset initialization on full truncation
  • KSTORAGE-626: Remove metrics reporter configuration when configuring internal tier topic clients
  • [CNKAF-528] Use the inter-broker listener for SBK clients.
  • Prevent warning message on every client metadata update
  • KSTORAGE-580: Return a generic error message when compaction is enabled on a tiered topic
  • KCORE-40: Inter-cluster fetcher for cluster linking
  • Update role, Owner tags for cost reporting
  • KCORE-297: Fix wildcard principals for centralized ACLs and no.acl.found for ConfluentServerAuthorizer
  • CNKAF-541: Reduce the number of tests run in RandomClusterTest
  • METRICS-1232 - Whitelist check for individual and derived metrics
  • [METRICS-1233] make telemetry metrics whitelist a dynamic config
  • CNKAF-548: Skip checking quotas when recording replication traffic
  • Set TopicExclusion string in SBK-generated CruiseControl config.
  • KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422) (#1477)
  • KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422)
  • KSTORAGE-331, KSTORAGE-392: add and improve tier component listener metrics and heartbeats
  • CPKAFKA-4753: Add metrics tracking total txn time, txn timeouts, and illegal state errors
  • CPKAFKA-4649: Ensure LDAP group manager is recreated on writer failover
  • SBK MINOR: Drain permits in startUp
  • CNKAF-319: Add Start rebalance protocol
  • modify kafka_path to only set project for confluentplatform
  • CNKAF-520, CNKAF-525: Wait for KafkaSampleStore check to complete at startup
  • KSTORAGE-411: Pass a hint down from controller to the leader in case of an unclean leader election
  • KSTORAGE-453: Persist leader epoch in tier checkpoint file
  • re-schedule pr scans to weekly
  • [CNKAF-492] SBK: CruiseControl Property defaults
  • Publish preview artifacts with preview release settings
  • CNKAF-50: Add static broker-level replication throttles
  • SBK: Implement KafkaDataBalancer startup/shutdown methods
  • CPKAFKA-4772: Fix PhysicalClusterMetadataTest’s tests
  • METRICS-1292 Dynamic Update of API key and secret in Kafka.
  • [CNKAF-216] Derive SBK capacity values from config parameters directly.
  • CONFLUENT: Make Rest Client Http Authenticator configurable
  • CNKAF 362: Topic exclusion logic for SBK in KafkaDataBalancer
  • CNKAF-305: Dynamic configs for sbk - enable/disable, throttle
  • CONFLUENT: Update cc-trogdor-service and cc-soak-clients to 0.803.0-6.0.0-ce-SNAPSHOT
  • KSTORAGE-545: Handle failed deletions from GcsTierObjectStore
  • CONFLUENT :HttpBearerCredentialProvider must implement configurable
  • ST-3292: migrate to semaphore2
  • CNKAF-19: ADB: Soak test self-healing
  • KSTORAGE-579: FIX fencing advancement issues
  • KSTORAGE-574: producer and epoch state restore is not resilient to failures
  • KSTORAGE-399 : Crash broker if FileTierPartitionState encounters corrupted flushed state file during initialization.
  • SEC-162: RestClient refactor regression
  • [CNKAF-479] Support executing ce-sbk objects with the rest of Kafka
  • [CNKAF-491] Remove clusterConfigs functionality from the TopicConfigProvider.
  • Revert “KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319) (#1401)”
  • KSTORAGE-482 Enhancing TierMetadataValidator tool to verify objects in S3 TierObjectStore
  • KSTORAGE-559: add maximum eligible segments deleted on every cleanup run
  • CNKAF-486: Add filter to exclude internal topics
  • Put an initial implementation of the DataBalancer interface in ce-sbk.
  • KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319)
  • KSTORAGE-568: bound hotset retention by LSO
  • KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions)
  • CPKAFKA-4670 Fix for system tests running 0.8.2.2 version producers
  • CNKAF-324: Allow configuring a delay before broker registration with ZK.
  • CNKAF 447: Update SBK mode config to a flag
  • [CNKAF-477] Restore ConfluentMetricsReporterSampler
  • KSTORAGE-572: FIX: Access fenced partitions metric under lock in TierTopicConsumer
  • CPKAFKA-4676: TierRoundtripTest.test_tier_roundtrip timeout increase
  • KSTORAGE-548: Check for missing init leader messages
  • KCORE-234: Upgrade netty to 4.1.48.Final and netty-tcnative to 2.0.29.Final
  • MCM-965: Reduced physical cluster metadata reload period to 2 mins
  • CC-7979: Add new topic tracking operations of KIP-558 to default RBAC roles
  • KAFKA-9685: Solve Set concatenation perf issue in AclAuthorizer (cherry-pick)
  • Bump up rollingupgrade deb package from 0.5.0 to 0.9.0
  • [CPKAFKA-4658] Restore removed SchemaRegistryUrl Config.
  • KCORE-225: Schedule periodic license validation task at fixed rate
  • DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client (#1374)
  • DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client
  • KSTORAGE-465: GCS tier deletion manager system test support
  • v5.5.x KSTORAGE-524: Fix potential NPE in DelayedFetch
  • KCORE-199: Set default values for schema configs on topics to avoid NPE
  • CNKAF 289: Increase partition count in soak cluster
  • [CNKAF-283] Put heavily trimmed CruiseControl into ce-kafka/ce-sbk
  • KSTORAGE-550 Narrow scope of KafkaException to avoid handling of any unwanted bug.
  • CONFLUENT MINOR: Revert #799, remove batch verifier from eos topic consumers
  • KCORE-111: Default replica placement constraints for external topics
  • CONFLUENT: fix exhaustive match warnings in TierArchiver
  • KSTORAGE-354: avoid roundtrip and connection in GCS fetch path
  • KCORE-175: Extend TopicPlacement to include assignment validation logic
  • SEC-159: RestRequest pluggable HTTP auth
  • KSTORAGE-543: Fix AutoAbortingS3InputStream: account for # read bytes being -1
  • KSTORAGE-214, KSTORAGE-407: Bump FileTierPartitionState version
  • Tier Storage system test for Consumer.offsetForTimes API
  • Refactoring FIPS implementation
  • KSTORAGE-389: Stronger detection of duplicate message processing for tiered storage
  • CPKAFKA-2336: Log layer metrics for tiered storage
  • CNKAF-305: Remove topic config from DynamicBalancer config
  • METRICS-1138 Removing shading of transitive dependencies due to telemetry-client jar
  • KCORE-190: Add max LSO lag metric
  • KSTORAGE-468 : Metadata states validation insync with log start offset.
  • CNKAF-304: Default value for rack aware flag
  • KCORE-104: Re-instate PartitionLockTest.testAppendReplicaFetchWithUpdateIsr
  • KSTORAGE-524: Fix potential NPE in DelayedFetch
  • SEC-804: Wait for config update before verification in audit config change test
  • CPKAFKA-1953: resume materialization from TierTopicManager commit positions
  • Log TierPartitionStatus to differentiate online vs catchup consumer
  • KSTORAGE-450: default confluent.tier.fetcher.num.threads = 4
  • KSTORAGE-457 ListOffsetRequest v0 does not account for tiered section…
  • KSTORAGE-330, KSTORAGE-328, KSTORAGE-325 TierStorageMaterialization recovery and validation tool.
  • KSTORAGE-452: Improve README for system tests w/ Tiered Storage
  • KSTORAGE-400: Handle duplicate PartitionDeleteInitiate messages
  • KSTORAGE-420: missing toString on MergedLog
  • KSTORAGE-460: Disable preferred tier reads for replica fetch
  • CPKAFKA-4369. Cherry pick of #1228 into 5.5.x
  • KSTORAGE-214, KSTORAGE-407: Fence user partition failures when materializing Tier Topic events
  • KSTORAGE-510: Ensure that PendingFetch::finish() stops the PendingFetch
  • Reduced test strength to ignore consumer data coverage caused by catc…
  • CPKAFKA-4126: Add prefix to system test config to match MDS changes
  • SEC-147: Refactor RestClient
  • KSTORAGE-519: Fix: Ensure unknown exceptions from DeletionTask are isolated to user partition
  • KSTORAGE-511: Add size asserts for TierPartitionStateEntry
  • KCORE-183: Avoid new allocations in MultiTenantRequestContext.filteredMetadataResponse call
  • KSTORAGE-344: TierMaterializationValidation as service for System Tests
  • CONFLUENT: Fix bug in result handling from readFromLocalLog
  • CNKAF-323: Add metric with per-tenant partition count
  • KCORE-151: Allow some reassignment of offline brokers
  • KSTORAGE-508: System test s3 prefix random generation incorrectly app…
  • KSTORAGE-505: Tier deletion manager system test failing in vagrant runs
  • KSTORAGE-424: Allow use of SSL for internal tier topic clients
  • KSTORAGE-384 Adding system test for tiered state file corruption
  • Merge branch ‘ccs-master’
  • KSTORAGE-502: Improve documentation for tiered storage configurations
  • CNKAF-305: Dynamic configs for SBK - kafka-side changes
  • KCORE-93: Removing ZK utils from ce-kafka
  • KCORE-110: Introducing new flag to display partitions violating topic replica constraints
  • CNKAF-304: Update network config name
  • CNKAF-304: Add static configs for self-balancing kafka
  • KCORE-104: Re-introduce PartitionLockTest
  • CONFLUENT: Update protobuf to 3.11.4 (#1237)
  • Update protobuf to 3.11.4
  • CPKAFKA-2700; Disable license validator in MultiTenantAuthorizer
  • Define a DataBalancer object in the Kafka Controller.
  • SEC-744 Make Audit Log Configs visible to AdminClient
  • METRICS-582: Adding tracking for kafka.broker.rack in telemetry
  • SEC-752 AuditLog FetchFollower
  • KSTORAGE-163: Initial implementation of TierFetcher memory bounding.
  • CPKAFKA-2705: Added retention to soak topics
  • KSTORAGE-479: Revert “KSTORAGE-464: Fix: ensure a queued TierTopic ev…
  • Add manual testing sections to ce-metrics README
  • SEC-746 Ignore unknown JSON properties for forwards compatibility
  • KSTORAGE-464: Fix: ensure a queued TierTopic event future throws when replaced
  • KSTORAGE-337/KSTORAGE-348: tier deletion manager system test
  • CONFLUENT: Add check in consume test util to verify records were consumed
  • METRICS-541 increase metric reporter default publishing time from 15 se…
  • CPKAFKA-3878: Re-enable scala inlining
  • CPKAFKA-4369. Copy Kafka configs when creating LogConfig- Part 1 of 2
  • CONFLUENT: Log better message on soak clients task creation 409 error response
  • TierArchiver: sleep outside of synchronized block in main loop

Security

  • Validate KSQL/RBAC cannot initialize on non-SASL protocols
  • Check authentication.roles in KSQL bearer token requests
  • CLIENTS-1658: Remove kafka core dependency from kafka-rest-security-plugin
  • SEC-1034: log4j migration to confluent repackaged version
  • remove test scope from connect-runtime dependency
  • DG-706: fix bug in schema registry client tests, as well as some refactor
  • DG-708: Skip Schema Validation query range check for first lookup
  • Include Kafka dependencies to KSQL packaging
  • Bump Vert.x version in confluent-ksql-security-plugin to 3.9.1
  • fix: Make rest-utils compile scope as ksqlDB no longer pulls this in
  • SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
  • Remove AdminClientWrapper.
  • Switch from application/vnd.api+json to application/json.
  • Migrate off JSON:API.
  • SEC-951: Upgraded bc-fips jars to 1.0.2
  • Add authorizations for mode endpoints
  • Add test that reproduces rest-proxy authorization bug.
  • Fix authentication for non-producer POST requests in kafka-rest.
  • Fix issue SSL class could not be found
  • Remove obsolete kafka-rest dependencies.
  • Fix bugs in Vertx auth plugin loginservice initialisation and unimplemented response method
  • Fix Content-Type for produce request.
  • Fix test due to ksql refactoring.
  • Remove kafka-rest V1 APIs and all their dependencies.
  • DG-403 Add missing method to retrieve schema refs
  • Vertx auth adaptors
  • SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
  • Add confluent-security-plugins-common module back
  • Remove support-metrics-common dependency
  • KCORE-82: Removing ZK licensing code and introducing LicenseManager
  • Use kafka.metrics.KafkaYammerMetrics instead of com.yammer.metrics.Metrics
  • Fixing compiling error due to KIP-515

MQTT Proxy

  • CC-11294: Bumped up Netty version to be consistent with Kafka and rest of CP.
  • METRICS-1989: confluent-kafka-mqtt depends on confluent-telemetry
  • CC-10900: Add confluent-telemetry to production jars.
  • CC-9739: Pass producer-striped configs and bootstrap.servers to AdminClient when getting Kafka cluster id.
  • CC-9786: Depend on confluent-metrics in tests only.
  • fix for METRICS-1733 change telemetry config constant names
  • CC-8363: Configure telemetry reporter.

Metadata Service

  • SEC-1421 - Set default replicator factor when creating audit log topic
  • SEC-1369 - Update MDS to non-broken version of Maven Assembly plugin
  • SEC-1034: log4j migration to confluent repackaged version
  • SEC-866 - move audit log “other” to “management”
  • [SEC-1345] Read HTTP Reason phrase when connection closed
  • Move kafka-http-server project into clients project
  • SEC-1323 - Make MDS Api doc not have everything be Bold
  • Fixed code which hard code security protocl to SASL_PLAINTEXT, should…
  • SEC-1268 - Backout confluent.security.event.logger.destination.log.* properties
  • SEC-1259 - Clarify audit log config API spec
  • SEC-777 - Cluster Registry & MDS should be lenient with it’s JSON Parsing
  • SEC-777 Cluster Registry & MDS should be lenient with it’s JSON Parsing
  • SEC-1222 Role Resources For Principal API throw a null pointer exception if ClusterName is passed in ResourceRequest
  • SEC-1089 CR C3 - Return Nice Names for all other Private UI endpoints
  • Fixed defect that RbacApiAppConfig missed all audit config with prefix confluent.security.
  • CLIENTS-1599: Convert KafkaHttpServlet to rest-utils Application.
  • SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
  • SEC-1212 - Do not create a new admin client if admin config not changed
  • SEC-1097: Disallow reassignment of name <=> scope.
  • SEC-1088 CR C3 - Return Nice Names for C3 visibility endpoint
  • MMA-7805: Fix for correctly packaging cluster-information-migration-script
  • MMA-7805 Script to import and export cluster information data to and from Registry
  • SEC-1191 - Assign default topics to produce, consume and other category
  • SEC-1021: updated validation + make error messages useful
  • SEC-1130 - Inject ConfluentAdmin from ce-kafka
  • Migrate MDS to KafkaHttpServlet
  • Added more testing cases for verifying audit log backend API.
  • SEC-1127 - Fixed a defect that AuditLogConfigSpec didn’t get updated in DynamicC…
  • SEC-988 Cluster Registry Fix to make CLI backwards compatible
  • SEC-1122 - Fix 6.0.x build break
  • SEC-986: Lockdown cluster name
  • SEC-940 - Make clusters optional in MdsScope in swagger spec
  • SEC-1037: Return sorted ‘full’ clusters followed by sorted ‘redacted’ clusters for cluster registry
  • SEC-1003 Cluster Registry : Migrate from “name” to “clusterName”
  • SEC-1020 - Cluster Info Protocol Validation
  • SEC-982: Nice name gatekeeping
  • CIAM-12 - Try loading token.key.path as a public key for cloud
  • SEC-731: Single shot overwrite of multiple clusters
  • SEC-959: Full Cluster Info CRUD Gatekeeping for Cluster Registry
  • SEC-1019 Cluster Registry : Introduce security protocol field to ClusterInfo
  • SEC-985: MDS CR NiceNames : Update basic RBAC endpoints
  • SEC-1015: Fixed compile error due to interface changed by SEC-1015 in ce-kafka.
  • For backward compatibility we name the json scope object as ‘scope’
  • Add support for Cluster Scope object with clusters or paths to MdsSco…
  • CIAM-49 Spec file for MDS API v2alpha1 - fix whitespace
  • CIAM-49 Spec file for MDS API v2alpha1
  • SEC-1001: Require at least one host in a Cluster
  • SEC-926: Using dynamic config for cluster registry backend
  • SEC-941 Add Http api that returns an array of Mds Scope object.
  • SEC-942: MDS CR NiceNames : Introduce MdsScope Object
  • SEC-965: Sorted-ness of ClusterRegistry
  • SEC-943 - MDS CR : Investigate creating ResourceType for CR CRUD
  • Fix OpenAPI specification format
  • SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
  • SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
  • SEC-949 - MDS CR : Update CR given Audit logs fix
  • SEC-728 - Cluster Registry - Make the backend be a broker Dynamic Config
  • SEC-891 - MDS : Short term fix flakey Token tests by isolating them
  • [SEC-844] restore /security/1.0 prefix
  • SEC-871 - MDS Test : Test / document OOTB Ldap and RBAC case sensitivity
  • SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
  • SEC-800 - MDS : Remove MdsLoginService.roles
  • SEC-751 MDS - Clean up the openapi yaml spec for CLI
  • SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
  • SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
  • SEC-727 - Cluster Registry - Lock down the basic list of clusters
  • SEC-726 - Service Reg - Basic list of clusters

Replicator

  • REP-1176 Log error when connector submission fails for Replicator executable
  • REP-1131: Add support for Replicator with regex router
  • REP-1055 Enable converter configs for Replicator executable
  • REP-787: Synchronize fetching end offsets from source cluster to calculate lag
  • REP-824: Changed JMX initialization to use method from KIP-606
  • REP-721: Delete old license validation code from codebase
  • REP-795 moved translated records log to DEBUG and only print if records were translated
  • REP-761: ensure offset.topic.commit is false if topic.preserve.partitions is false
  • REP-755: retrieve port from –connect-url if specified
  • REP-743: Fix issue where we don’t properly log the topic that got an UnknownTopicPartitionException
  • REP-754 fix for offset commit failure resulting in loss of offsets
  • REP-689: Reduce log level of offset commit exception from ERROR to WARN
  • REP-665: Deleting all ZK dependencies from code base
  • REP-673: Port ZK dependencies for replicator-verifier into the verifier module
  • REP-664: Remove ZK dependencies from ReplicatorSourceConnector and ReplicatorSourceTask