Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 6.0.0¶

Community Features¶

Common¶

PR-294 - SEC-1307: Backport “log4j replacement with confluent repackaged version”
PR-278 - SEC-1334: update confluent-log4j version
PR-277 - KC-613: Update slf4j version to match AK’s.
PR-274 - ST-3988: Update jackson version to 2.10.5
PR-275 - SEC-1305: update jackson-bom version
PR-272 - METRICS-1918: bump gson from 2.8.5 -> 2.8.6
PR-270 - SEC-1034: log4j migration to confluent repackaged version
PR-269 - Upgrade Jackson to 2.10.2 and Avro to 1.9.2
PR-265 - downgrade maven-deploy-plugin to 2.8.2
PR-258 - update various maven plugins
PR-257 - ST-1323: Enable source jars if Apache or Confluent license files are found
PR-262 - fix: Pin the Scala version
PR-261 - Update ZooKeeper to 3.5.8
PR-260 - Update Scala version to 2.13
PR-259 - add property for jackson-jsonschema version
PR-248 - Upgrade Avro to 1.9.2

Kafka¶

PR-9197 - Revert KAFKA-9309: Add the ability to translate Message to JSON
PR-9172 - KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect
PR-392 - KAFKA-2715: Removed previous system_test folder
PR-384 - KAFKA-2714: Added integration tests for exceptional cases in fetching
PR-9089 - KAFKA-10224: Update jersey license from CDDL to EPLv2
PR-9066 - KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets
PR-9086 - FIX: Remove staticmethod tag to be able to use logger of instance
PR-9075 - KAFKA-10306: GlobalThread should fail on InvalidOffsetException
PR-9065 - KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates
PR-9051 - KAFKA-10268: dynamic config like “–delete-config log.retention.ms” doesn’t work
PR-9026 - KAFKA-10274; Consistent timeouts in transactions_test
PR-8979 - KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers
PR-8239 - KAFKA-9666: Don’t increase transactional epoch when trying to fence if the log append fails
PR-9009 - KAFKA-6453: Document how timestamps are computed for aggregations and joins
PR-8994 - KAFKA-10247: Correctly reset state when task is corrupted
PR-9010 - KAFKA-10262: Ensure that creating task directory is thread safe
PR-9005 - KAFKA-10263: Do not assign standby for revoking stateless tasks
PR-8996 - KAFKA-10249: don’t try to read un-checkpointed offsets of in-memory stores
PR-8934 - KAFKA-10134: Use long poll if we do not have fetchable partitions
PR-8989 - KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups
PR-8962 - KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory
PR-8983 - KAFKA-8398: Prevent NPE in forceUnmap
PR-8858 - KAFKA-10153: Error Reporting in Connect Documentation
PR-8917 - KAFKA-10180: Fix security_config caching.
PR-8905 - KAFKA-10173: Fix suppress changelog binary schema compatibility
PR-8926 - KAFKA-10166: always write checkpoint before closing an (initialized) task
PR-8924 - KAFKA-10198: guard against recycling dirty state
PR-8900 - KAFKA-10169: swallow non-fatal KafkaException and don’t abort transaction during clean close
PR-8887 - KAFKA-10135: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager…
PR-8896 - KAFKA-10185: Restoration info logging
PR-8890 - KAFKA-9891: add integration tests for EOS and StandbyTask
PR-8822 - KAFKA-10113; Specify fetch offsets correctly in LogTruncationException
PR-8876 - KAFKA-10167: use the admin client to read end-offset
PR-8841 - KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition
PR-8853 - KAFKA-10147 MockAdminClient#describeConfigs(Collection<ConfigResource…
PR-8882 - KAFKA-10165: Remove Percentiles from e2e metrics
PR-8856 - KAFKA-10150: task state transitions/management and committing cleanup
PR-8880 - KAFKA-10169: Error message when transit to Aborting / AbortableError / FatalError
PR-8872 - Fix log message for transition from standby to active
PR-8873 - Avoid WARN log message when re-init from checkpoint skipped
PR-8849 - KAFKA-10144: clean up corrupted standby tasks before attempting a commit
PR-8764 - KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes
PR-8787 - KAFKA-10085: correctly compute lag for optimized source changelogs
PR-8818 - KAFKA-10086: Integration test for ensuring warmups are effective
PR-8069 - KAFKA-9374: Make connector interactions asynchronous
PR-8828 - KAFKA-9216: Enforce that Connect’s internal topics use compact cleanup policy
PR-8829 - KAFKA-10115: Incorporate errors.tolerance with the Errant Record Reporter
PR-8455 - KAFKA-9845: Warn users about using config providers with plugin.path property
PR-8502 - KAFKA-9066: Retain metrics for failed tasks
PR-8847 - KAFKA-7833: Add missing test
PR-8453 - KAFKA-9841: Revoke duplicate connectors and tasks when zombie workers return with an outdated assignment
PR-8663 - KAFKA-9985: Sink connector may exhaust broker when writing in DLQ
PR-8833 - KAFKA-9441: remove prepareClose() to simplify task management
PR-8825 - KAFKA-7833: Add StateStore name conflict check in InternalTopologyBuilder
PR-8630 - KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation
PR-8775 - KAFKA-10079: improve thread-level stickiness
PR-7384 - KAFKA-8938: Improve allocations during Struct validation in ConnectSchema
PR-8827 - KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used
PR-8805 - KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group
PR-8783 - KAFKA-10063 UnsupportedOperation when querying cleaner metrics after shutdown
PR-8803 - KAFKA-10102: update ProcessorTopology instead of rebuilding it
PR-8676 - KAFKA-10005: Decouple RestoreListener from RestoreCallback
PR-8684 - KAFKA-10012 Reducing memory overhead associated with strings in SelectorMetrics
PR-8270 - KAFKA-9216: Enforce internal config topic settings for Connect workers during startup
PR-8820 - KAFKA-10097: Internalize checkpoint data
PR-8804 - KAFKA-9851: Revoking Connect tasks due to connectivity issues should also clear the running assignment
PR-8486 - KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable
PR-8135 - KAFKA-9570: Define SSL configs in all worker config classes, not just distributed
PR-8814 - KAFKA-10111: Make SinkTaskContext.errantRecordReporter() a default method
PR-8811 - KAFKA-10110: Corrected potential NPE when null label value added to KafkaMetricsContext
PR-8776 - KAFKA-9441: Improve Kafka Streams task management
PR-8759 - KAFKA-10066: TestOutputTopic should pass record headers into deserializers
PR-8755 - KAFKA-10069: Correctly remove user-defined “predicate” and “negate” configs from transformation properties
PR-8644 - KAFKA-9313: Set use_all_dns_ips as the new default for client.dns.lookup (KIP-602)
PR-8784 - KAFKA-9788: Use distinct names for transaction and group load time sensors
PR-8792 - KAFKA-10089 The stale ssl engine factory is not closed after reconfigure
PR-8782 - KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends
PR-8785 - KAFKA-10084: Fix EosTestDriver end offset
PR-8786 - KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests
PR-8737 - KAFKA-9945: TopicCommand should support –if-exists and –if-not-exists when –bootstrap-server is used
PR-8695 - KAFKA-9320: Enable TLSv1.3 by default (KIP-573)
PR-8777 - KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment
PR-8706 - KAFKA-10030 allow fetching a key from a single partition
PR-8760 - Kafka-10064 Add documentation for KIP-571
PR-8668 - KAFKA-9987: optimize sticky assignment algorithm for same-subscription case
PR-7956 - KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing
PR-8769 - KAFKA-10074: Improve performance of matchingAcls
PR-8723 - KIP-569-KAFKA-9494: DescribeConfigsResponse - include additional metadata information
PR-8739 - KAFKA-10056; Ensure consumer metadata contains new topics on subscription change
PR-8705 - KAFKA-10029; Don’t update completedReceives when channels are closed to avoid ConcurrentModificationException
PR-8238 - KAFKA-9130: KIP-518 Allow listing consumer groups per state
PR-8248 - KAFKA-9501: convert between active and standby without closing stores
PR-8699 - KAFKA-9673: Filter and Conditional SMTs
PR-8720 - KAFKA-9971: Error Reporting in Sink Connectors (KIP-610)
PR-8736 - KAFKA-9802; Increase transaction timeout in system tests to reduce flakiness
PR-8731 - KAFKA-10050: kafka_log4j_appender.py fixed for JDK11
PR-8589 - KAFKA-9146: KIP-571 Add option to force delete active members in StreamsResetter
PR-8691 - KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter
PR-8709 - KAFKA-9952; Remove immediate fetch completion logic on high watermark updates
PR-8692 - KAFKA-10018: Change command line tools from /bin/sh to /bin/bash
PR-8697 - KAFKA-9983: KIP-613, add INFO level e2e latency metrics
PR-8376 - KAFKA-9724 Newer clients not always sending fetch request to older brokers
PR-8504 - KAFKA-9298: reuse mapped stream error in joins
PR-8716 - KAFKA-6145: KIP-441: Fix assignor config passthough
PR-6824 - KAFKA-7523: Add ConnectedStoreProvider to Processor API
PR-8221 - KAFKA-9561: update task input partitions after rebalance
PR-7886 - KAFKA-9353: Add groupInstanceId to DescribeGroup for better visibility
PR-8722 - KAFKA-5295: Allow source connectors to specify topic-specific settings for new topics (KIP-158)
PR-8395 - Added doc for KIP-535 and updated it for KIP-562
PR-8118 - KAFKA-9472: Remove deleted Connect tasks from status store
PR-2604 - KAFKA-4794: Add access to OffsetStorageReader from SourceConnector
PR-8698 - KAFKA-10022:console-producer supports the setting of client.id
PR-6284 - KAFKA-6755: Allow literal value for MaskField SMT
PR-8357 - KAFKA-9767: Add logging to basic auth rest extension
PR-8620 - KAFKA-9944: Added supporting customized HTTP response headers for Kafka Connect.
PR-8511 - KAFKA-9888: Copy connector configs before passing to REST extensions
PR-8654 - KAFKA-9931: Implement KIP-605 to expand support for Connect worker internal topic configurations
PR-8713 - KAFKA-6145: Add unit tests for assignments of only stateless tasks
PR-8696 - KAFKA-6145: KIP-441: Enforce Standby Task Stickiness
PR-8679 - KAFKA-10003: Mark KStream.through() as deprecated
PR-8658 - KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas
PR-8379 - KAFKA-9780: Deprecate commit records without record metadata
PR-8651 - kafkatest: Deploy VerifiableClient in constructor to avoid test timeouts
PR-8472 - KAFKA-9855 - return cached Structs for Schemas with no fields
PR-8608 - KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties
PR-8444 - KAFKA-8869: Remove task configs for deleted connectors from config snapshot
PR-7942 - KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect
PR-8661 - KAFKA-9603: Do not turn on bulk loading for segmented stores on stand-by tasks
PR-8671 - KAFKA-9859 / Add topics generated by KTable FK join to internal topic matching logic
PR-8681 - KAFKA-10010: Should make state store registration idempotent
PR-8682 - KAFKA-10011: Remove task id from lockedTaskDirectories during handleLostAll
PR-8673 - KAFKA-9992: EmbeddedKafkaCluster.deleteTopicAndWait not working with kafka_2.13
PR-8689 - KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173
PR-8675 - KAFKA-10004: ConfigCommand fails to find default broker configs without ZK
PR-8569 - KIP-551: Expose disk read and write metrics
PR-8678 - Update Gradle to 6.4.1
PR-8659 - KAFKA-9617 Replica Fetcher can mark partition as failed when max.message.bytes is changed
PR-8618 - KAFKA-9955: Prevent SinkTask::close from shadowing other exceptions
PR-8558 - KAFKA-8611 / KIP-221 documentation
PR-8670 - KAFKA-10001: Should trigger store specific callback if it is also a listener
PR-8090 - KAFKA-9537 - Cleanup error messages for abstract transformations
PR-8674 - KAFKA-9996: Upgrade zookeeper to 3.5.8
PR-8667 - KAFKA-9994: Handle task migrated inside corruption path
PR-8665 - KAFKA-9984 Should fail the subscription when pattern is empty
PR-8222 - KAFKA-9650: include human readable units in ms and bytes configs
PR-8588 - KAFKA-6145: KIP-441: Improve assignment balance
PR-313 - KAFKA-2641: Upgrade path for ZK authentication
PR-8550 - KAFKA-9850 Move KStream#repartition operator validation during Topology build process
PR-8648 - KAFKA-9966: Add internal assignment listener to stabilize EOS version 2 upgrade test
PR-8596 - KAFKA-9821: persist followup rebalance in assignment and consolidate rebalance triggering mechanisms
PR-8647 - KAFKA-9669; Loosen validation of inner offsets for older message formats
PR-8254 - KIP-557: Add Emit On Change Support
PR-8626 - KAFKA-9921: explicit handling of null values with retainDuplicates
PR-8643 - KAFKA-9956: Authorizer APIs may be invoked more than once for a given request
PR-8632 - KAFKA-9972: Only commit tasks with valid states
PR-8628 - KAFKA-9942: Fixes ConfigCommand client quotas w/ default users.
PR-8613 - KAFKA-6145: Set HighAvailabilityTaskAssignor as default in streams_upgrade_test.py
PR-8230 - KAFKA-9667: Connect JSON serde strip trailing zeros
PR-8483 - KAFKA-9865: Expose output topic names from TopologyTestDriver
PR-8360 - KAFKA-9768: Fix handling of rest.advertised.listener config
PR-7950 - KAFKA-9419: Integer Overflow Possible with CircularIterator
PR-8602 - KAFKA-9947; Ensure proper shutdown of components in TransactionsBounceTest
PR-8591 - KAFKA-6342: Move workaround for JSON parsing of non-escaped strings
PR-8609 - KAFKA-9946; StopReplicaRequest deletePartition changes may cause premature topic deletion handling in controller
PR-8496 - KAFKA-9748: Add Streams EOS version 2 integration test
PR-8607 - KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
PR-8527 - Remove dependencies on deprecated –zookeeper command flags in junit tests
PR-8551 - KAFKA-9918 SslEngineFactory is NOT closed when channel is closing
PR-8565 - KAFKA-9798: Send one round synchronously before starting the async producer
PR-8533 - KAFKA-9589: Fixed bug in V2 log validator tests
PR-8577 - use appropriate fn for readability. (maybe)
PR-8586 - KAFKA-9939; Fix overcounting delayed fetches in request rate metrics
PR-8595 - KAFKA-8410: Revert Part 1: processor context bounds (#8414)
PR-8597 - KAFKA-6145: KIP 441 remove balance factor
PR-8590 - KAFKA-6145: Remove check to reuse previous assignment
PR-8204 - KAFKA-9633: Ensure ConfigProviders are closed
PR-8554 - KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd
PR-8567 - KAFKA-9652: Fix throttle metric in RequestChannel and request log due to KIP-219
PR-8559 - KAFKA-9922:Update examples README
PR-8442 - KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses
PR-8578 - KAFKA-9875: Make integration tests more resilient
PR-8582 - KAFKA-9932: Don’t load configs from ZK when the log has already been loaded
PR-8574 - KAFKA-9925: decorate pseudo-topics with app id
PR-8580 - KAFKA-9832: fix attempt to commit non-running tasks
PR-8540 - KAFKA-9127: don’t create StreamThreads for global-only topology
PR-8568 - KAFKA-9176: Retry on getting local stores from KafkaStreams
PR-8548 - KAFKA-9823: Follow-up, check state for handling commit error response
PR-8541 - KAFKA-6145: KIP-441: Add TaskAssignor class config
PR-8564 - KAFKA-9921: disable caching on stores configured to retain duplicates
PR-8566 - Fix minor code issue
PR-8525 - KAFKA-9885; Evict last members of a group when the maximum allowed is reached
PR-8524 - KAFKA-9866: Avoid election for topics where preferred leader is not in ISR
PR-8509 - KAFKA-9839: Broker should accept control requests with newer broker epoch
PR-8224 - KAFKA-9704: Fix the issue z/OS won’t let us resize file when mmap.
PR-8326 - KAFKA-8639: Replace AddPartitionsToTxn with Automated Protocol
PR-8445 - KAFKA-9823: Remember the sent generation for the coordinator request
PR-8536 - KAFKA-9883: Add better error message when REST API forwards a request and leader is not known
PR-8537 - KAFKA-9907: Switch default build to Scala 2.13
PR-8464 - KAFKA-9852: Change the max duration that calls to the buffer pool can block from 2000ms to 10ms
PR-8417 - KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy
PR-8530 - KAFKA-9388: Refactor integration tests to always use different application ids
PR-8482 - KAFKA-9863: update the deprecated –zookeeper option in the documentation into –bootstrap-server
PR-8497 - KAFKA-6145: KIP-441 Build state constrained assignment from balanced one
PR-8520 - Add explicit grace period to tumbling window example
PR-8522 - KAFKA-9868: Reduce transaction log partitions for embed broker
PR-8523 - Adding github whitelist
PR-8475 - KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence
PR-8501 - KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test
PR-8448 - KAFKA-9796; Broker shutdown could be stuck forever under certain conditions
PR-8471 - KAFKA-9854 Re-authenticating causes mismatched parse of response
PR-8476 - KAFKA-9838; Add log concurrency test and fix minor race condition
PR-8378 - KAFKA-9779: Add Stream System Test 2.5
PR-6210 - KAFKA-7885: TopologyDescription violates equals-hashCode contract.
PR-8092 - KAFKA-9540: Move “Could not find the standby task while closing it” log to debug level
PR-8256 - KAFKA-9675: Fix bug that prevents RocksDB metrics to be updated
PR-8440 - KAFKA-9832: extend Kafka Streams EOS system test
PR-8492 - KAFKA-9739: Fix for 2.5 branch
PR-8403 - KAFKA-9797; Fix TestSecurityRollingUpgrade.test_enable_separate_interbroker_listener
PR-8477 - KAFKA-9864: avoid expensive QuotaViolationException usage
PR-8469 - [KAFKA-9826] Handle an unaligned first dirty offset during log cleaning.
PR-8470 - KAFKA-8611 / Refactor KStreamRepartitionIntegrationTest
PR-8257 - KAFKA-9539; Add leader epoch in StopReplicaRequest (KIP-570)
PR-8353 - KAFKA-9764: Remove stream simple benchmark suite
PR-8457 - KAFKA-9842; Add test case for OffsetsForLeaderEpoch grouping in Fetcher
PR-8077 - KAFKA-9583: use topic-partitions grouped by node to send OffsetsForLeaderEpoch requests
PR-8474 - KAFKA-9853: Improve performance of Log.fetchOffsetByTimestamp
PR-7015 - KAFKA-8436: use automated protocol for AddOffsetsToTxn
PR-8399 - KAFKA-3720: Change TimeoutException to BufferExhaustedException when no memory can be allocated for a record within max.block.ms
PR-8443 - KAFKA-9832: Extend Streams system tests for EOS version 2
PR-8436 - KAFKA-6145: KIP-441 avoid unnecessary movement of standbys
PR-7170 - KIP-221 / Add KStream#repartition operation
PR-7844 - KAFKA-9309: Add the ability to translate Message classes to and from JSON
PR-8441 - KAFKA-9748: Extend Streams integration tests for EOS version 2
PR-8212 - KAFKA-9642: Change “BigDecimal(double)” constructor to “BigDecimal.valueOf(double)”
PR-8317 - KAFKA-9691: Fix NPE by waiting for reassignment request
PR-8315 - KAFKA-9433: Use automated protocol for AlterConfigs request and response
PR-8425 - KAFKA-6145: KIP-441 Move tasks with caught-up destination clients right away
PR-8451 - KAFKA-9835; Protect FileRecords.slice from concurrent write
PR-8409 - KAFKA-6145: KIP-441 Pt. 6 Trigger probing rebalances until group is stable
PR-8338 - KAFKA-8890: KIP-519- Make SSL context/engine configuration extensible
PR-8405 - KAFKA-9801: Still trigger rebalance when static member joins in CompletingRebalance phase
PR-8414 - KAFKA-8410: Part 1: processor context bounds
PR-8423 - KAFKA-9818: improve error message to debug test
PR-8420 - KAFKA-9815; Ensure consumer always re-joins if JoinGroup fails
PR-8371 - KAFKA-9753: A few more metrics to add
PR-8402 - KAFKA-9793: Expand the try-catch for task commit in HandleAssignment
PR-8383 - KAFKA-9784: Add OffsetFetch to group concurrency test
PR-8422 - KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator
PR-8418 - KAFKA-9807; Protect LSO reads from concurrent high-watermark updates
PR-8408 - KAFKA-9810: Document Connect Root REST API on /
PR-8412 - KAFKA-9750; Fix race condition with log dir reassign completion
PR-8411 - KAFKA-9812: fix infinite loop in test code
PR-8400 - KAFKA-9739: Fixes null key changing child node
PR-8375 - KAFKA-9776: Downgrade TxnCommit API v3 when broker doesn’t support
PR-8407 - KAFKA-9809: Shrink transaction timeout for streams
PR-8359 - KAFKA-9778: Add validateConnector functionality to the EmbeddedConnectCluster
PR-8337 - KAFKA-6145: Pt. 5 Implement high availability assignment
PR-8331 - KAFKA-9748: extend EosIntegrationTest for EOS version 2
PR-7755 - KAFKA-9233: Fix IllegalStateException in Fetcher retrieval of beginni…
PR-8269 - KAFKA-9659: Add more log4j when updating static member mappings
PR-8356 - KAFKA-9441: Cleanup Streams metrics for removed task commit latency metrics
PR-8370 - KAFKA-9753: Add active tasks process ratio
PR-8389 - KAFKA-9777; Remove txn purgatory to fix race condition on txn completion
PR-8279 - KAFKA-9706: Handle null keys/values in Flatten transformation
PR-8367 - KAFKA-9719: Streams with EOS version 2 should fail fast for older brokers
PR-8388 - KAFKA-9759: Add doc change for KIP-562
PR-8350 - KAFKA-9760: Add KIP-447 protocol change to upgrade notes
PR-8382 - KAFKA-9685: PT2, avoid unnecessary set creation in ACL matching
PR-8164 - KAFKA-9600: EndTxn should enforce strict epoch checking if from client
PR-8368 - KAFKA-9770: Close underlying state store also when flush throws
PR-8369 - KAFKA-9771: Port patch for inter-worker Connect SSL from Jetty 9.4.25
PR-8358 - KAFKA-9756: Process more than one record of one task at a time
PR-8320 - KAFKA-8470: State change logs should not be in TRACE level
PR-8280 - KAFKA-9707: Fix InsertField.Key should apply to keys of tombstone records
PR-8332 - KAFKA-9688: kafka-topic.sh should show KIP-455 adding and removing replicas
PR-8346 - KAFKA-9373: Reduce shutdown time by avoiding unnecessary loading of indexes
PR-6741 - Improve performance of checkpointHighWatermarks, patch 1/2
PR-8138 - KAFKA-9573: Fix JVM options to run early versions of Kafka on the latest JVMs
PR-8343 - KAFKA-9758: Doc changes for KIP-523 and KIP-527
PR-8290 - KAFKA-9677: Fix consumer fetch with small consume bandwidth quotas
PR-8339 - KAFKA-9752; New member timeout can leave group rebalance stuck
PR-8285 - KAFKA-9700:Fix negative estimatedCompressionRatio issue
PR-8287 - KAFKA-9711 The authentication failure caused by SSLEngine#beginHandsh…
PR-8334 - KAFKA-6145: Add balanced assignment algorithm
PR-8336 - KAFKA-9749: TransactionMarkerRequestCompletionHandler should treat KAFKA_STORAGE
PR-8327 - KAFKA-9743: Catch commit offset exception to eventually close dirty tasks
PR-7939 - KAFKA-9407: Return Immutable List from SchemaSourceTask
PR-8330 - KAFKA-9742: Fix broken StandbyTaskEOSIntegrationTest
PR-8226 - KAFKA-9651: Fix ArithmeticException (÷ by 0) in Partitioner impls
PR-8318 - KAFKA-9451: Enable producer per thread for Streams EOS
PR-8205 - KAFKA-9634: Add note about thread safety of ConfigProvider
PR-8252 - KAFKA-6145: Pt 2.5 Compute overall task lag per client
PR-8325 - KAFKA-9741: Update ConsumerGroupMetadata before calling onPartitionsRevoked()
PR-8324 - KAFKA-9701 (fix): Only check protocol name when generation is valid
PR-8307 - KAFKA-9727: cleanup the state store for standby task dirty close and check null for changelogs
PR-8262 - KAFKA-6145: Add constrained balanced assignment algorithm
PR-8319 - KAFKA-9734: Fix IllegalState in Streams transit to standby
PR-8155 - KAFKA-9553: Improve measurement for loading groups and transactions
PR-8244 - KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API
PR-8223 - KAFKA-9654 ReplicaAlterLogDirsThread can’t be created again if the pr…
PR-8284 - KAFKA-9225: Bump rocksdb 5.18.3 to 5.18.4
PR-8218 - KAFKA-9441: Unify committing within TaskManager
PR-7972 - KAFKA-9435: DescribeLogDirs automated protocol
PR-7039 - KAFKA-8618: Replace Txn marker with automated protocol
PR-8299 - KAFKA-9568: enforce rebalance if client endpoint has changed
PR-8313 - KAFKA-5604: Remove the redundant TODO marker on the Streams side
PR-8253 - KAFKA-9656: Return COORDINATOR_NOT_AVAILABLE for older producer clients
PR-8161 - KAFKA-9047: Making AdminClient group operations respect retries and backoff
PR-7936 - KAFKA-9404: Use ArrayList instead of LinkedList in Sensor Class
PR-8260 - KAFKA-9625: Fixing IncrementalAlterConfigs with respect to Broker Configs
PR-8278 - KAFKA-8803: Remove timestamp check in completeTransitionTo
PR-8289 - KAFKA-9712: Catch and handle exception thrown by reflections scanner
PR-8236 - KAFKA-9670: Reduce allocations in Metadata Response preparation
PR-8083 - KIP-546 (1/2): Implements describeClientQuotas() and alterClientQuotas().
PR-8267 - KAFKA-6647: Do note delete the lock file while holding the lock
PR-8246 - KAFKA-6145: Pt 2. Include offset sums in subscription
PR-8293 - KAFKA-9715: Eliminate reference to interBrokerProtocolVersion in TransactionStateManager
PR-8294 - KAFKA-9718; Don’t log passwords for AlterConfigs in request logs
PR-7957 - KAFKA-8768: DeleteRecords request/response automated protocol
PR-8265 - KAFKA-9657: Throw upon offset fetch unsupported stable flag protocol
PR-8266 - KAFKA-9695; Handle null config values for createTopics, alterConfigs
PR-8272 - KAFKA-9701: Add more debug log on client to reproduce the issue
PR-8177 - KAFKA-9605: Do not attempt to abort batches when txn manager is in fatal error
PR-7876 - KAFKA-9344:Logged consumer config does not always match actual config…
PR-8232 - KAFKA-9658: Fix removing user quotas
PR-8255 - KAFKA-9686 MockConsumer#endOffsets should be idempotent
PR-8235 - KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode
PR-283 - KAFKA-2391: add timeout to KafkaConsumer blocking calls
PR-282 - KAFKA-2428
PR-8220 - KAFKA-9645: Fallback to unsubscribe during Task Migrated
PR-8228 - KAFKA-9663: Doc some null returns in KafkaStreams
PR-284 - KAFKA-2620: Introduce Scalariform
PR-8233 - KAFKA-9668: Iterating over KafkaStreams.getAllMetadata() results in ConcurrentModificationException
PR-279 - KAFKA-2479: Transient exception
PR-8227 - KAFKA-9662: Wait for consumer offset reset in throttle test to avoid losing early messages
PR-8121 - KAFKA-6145: Pt 1. Bump protocol version and encode task lag map
PR-8213 - KAFKA-9615: Clean up task/producer create and close
PR-8229 - KAFKA-9661: Propagate includeSynonyms option to AdminClient in ConfigCommand
PR-254 - KAFKA-2588 ReplicaManager partitionCount metric should actually be re…
PR-277 - Kafka-2587: Only notification handler will update the cache and all verifications will use waitUntilTrue.
PR-8209 - KAFKA-9632; Fix MockScheduler synchronization for safe use in Log/Partition tests
PR-8186 - KAFKA-9618: Directory deletion failure leading to error task RocksDB open
PR-275 - KAFKA-2527; System Test for Quotas in Ducktape
PR-8208 - KAFKA-8895: delete all topics before recreating
PR-8197 - KAFKA-9573: Fix VerifiableProducer and VerifiableConsumer to work with older Kafka versions
PR-1286 - KAFKA-3618: Handle ApiVersionsRequest before SASL authentication
PR-8087 - KAFKA-9525: add enforceRebalance method to Consumer API
PR-8190 - KAFKA-9623: Keep polling until the task manager is no longer rebalancing in progress
PR-8187 - KAFKA-9620: Do not throw in the middle of consumer user callbacks
PR-273 - KAFKA-2517; Performance Regression post SSL implementation (zero copy)
PR-8168 - KAFKA-9607: Do not clear partition queues during close
PR-8169 - KAFKA-9610: Do not throw illegal state when remaining partitions are not empty
PR-8173 - KAFKA-9614: Not initialize topology twice in StreamTask
PR-8165 - KAFKA-9601: Stop logging raw connector config values
PR-8166 - KAFKA-9602: Close the stream internal producer only in EOS
PR-8062 - KAFKA-9498; Topic validation during the topic creation triggers unnecessary TopicChange events
PR-8029 - KAFKA-8147: Add changelog topic configuration to KTable suppress
PR-8167 - KAFKA-9533: Revert ValueTransform forwards null
PR-8009 - KAFKA-9308: Reworded the ssl part of the security documentation
PR-8159 - KAFKA-9599 create unique sensor to record group rebalance
PR-8145 - KAFKA-9581: Remove rebalance exception withholding
PR-8142 - KAFKA-9577: SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version
PR-8105 - KAFKA-9441: Add internal StreamsProducer
PR-263 - KAFKA 2578 Client Metadata internal state should be synchronized
PR-8143 - KAFKA-9582: Do not abort transaction in unclean close
PR-8111 - KAFKA-9206: throw a KafkaException when encountering CORRUPT_MESSAGE
PR-8058 - KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted
PR-8116 - KAFKA-9562: part 1: ignore exceptions while flushing stores in close(dirty)
PR-7337 - KAFKA-8910: Incorrect javadoc at KafkaProducer.InterceptorCallback#onCompletion
PR-8108 - KAFKA-9533: ValueTransform forwards null values
PR-7913 - KAFKA-9343: Add ps command for Kafka and zookeeper process on z/OS.
PR-8119 - KAFKA-9558: Fixing retry logic for getListOffsetsCalls
PR-8089 - KAFKA-6266: Repeated occurrence of WARN Resetting first dirty offset
PR-8125 - KAFKA-9515: Upgrade ZooKeeper to 3.5.7
PR-8106 - KAFKA-9319: Fix generation of CA certificate for system tests.
PR-8109 - KAFKA-9545: Fix subscription bugs from Stream refactoring
PR-8088 - KAFKA-9535: Update metadata upon retrying partitions for ListOffset
PR-7389 - KAFKA-8805: Bump producer epoch on recoverable errors
PR-8060 - KAFKA-9274: Gracefully handle timeout exception
PR-7781 - KAFKA-8904: Improve producer’s topic metadata fetching.
PR-8085 - KAFKA-9556: Fix two issues with KIP-558 and expand testing coverage
PR-8112 - KAFKA-9557: correct thread process-rate sensor to measure throughput
PR-8023 - KAFKA-8507 kip 499 Unify connection name flag for command line tool
PR-7674 - KAFKA-9106 make metrics exposed via jmx configurable
PR-7731 - KAFKA-9204: allow ReplaceField SMT to handle tombstone records
PR-7733 - KAFKA-9192: fix NPE when for converting optional json schema in structs
PR-8000 - KAFKA-9417: New Integration Test for KIP-447
PR-8031 - KAFKA-9447: Add new customized EOS model example
PR-8091 - KAFKA-6607: Commit correct offsets for transactional input data
PR-8053 - KAFKA-9499; Improve deletion process by batching more aggressively
PR-7955 - KAFKA-9423: Refine layout of configuration options on website and make individual settings directly linkable
PR-8015 - KAFKA-9500: Fix topology bug in foreign key joins
PR-8065 - KAKFA-9503: Fix TopologyTestDriver output order
PR-7996 - KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS
PR-8054 - KAFKA-9390: Make serde pseudo-topics unique
PR-8024 - KAFKA-9483; Add Scala KStream#toTable to the Streams DSL
PR-8040 - KAFKA-6607: Commit correct offsets for transactional input data
PR-8059 - KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent fields
PR-8061 - KAFKA-9517: Fix default serdes with FK join
PR-8033 - KAFKA-9487: Follow-up PR of Kafka-9445
PR-8081 - KAFKA-9523: Migrate BranchedMultiLevelRepartitionConnectedTopologyTest into a unit test
PR-8018 - KAFKA-9480: Fix bug that prevented to measure task-level process-rate
PR-8039 - KAFKA-9505: Only loop over topics-to-validate in retries
PR-8027 - KAFKA-7658: Follow up to original PR
PR-8057 - KAFKA-9507 AdminClient should check for missing committed offsets
PR-8056 - KAFKA-9519: Deprecating ZK for ConfigCommand
PR-8048 - KAFKA-9509: Fixing flakiness of MirrorConnectorsIntegrationTest.testReplication
PR-252 - KAFKA-2586; Enable SSL for inter-broker communication when SSL is enabled in tests
PR-255 - KAFKA-2593 Key value stores can use specified serializers and deserializers
PR-7770 - KAFKA-9261; Client should handle unavailable leader metadata
PR-7997 - KAFKA-9113: Clean up task management and state management
PR-8037 - KAFKA-9491; Increment high watermark after full log truncation
PR-8007 - KAFKA-9477 Document RoundRobinAssignor as an option for partition.assignment.strategy
PR-7568 - KAFKA-9074: Correct Connect’s Values.parseString to properly parse a time and timestamp literal
PR-8030 - KAFKA-9492; Ignore record errors in ProduceResponse for older versions
PR-8028 - KAFKA-9490: Fix generics for Grouped
PR-8026 - Bump trunk to 2.6.0-SNAPSHOT
PR-7725 - KAFKA-9027, KAFKA-9028: Convert create/delete acls requests/response to use generated protocol

ksqlDB¶

PR-6126 - fix: create /var/lib/kafka-streams directory on RPM installation
PR-6165 - fix: allow expressions in flat map (6.0.x)
PR-5952 - METRICS-1988: add confluent-telemetry directory to ksql-run-class
PR-5937 - fix: avoid losing cause of processing errors
PR-5895 - fix: adds a handler to gracefully shutdown
PR-5885 - fix: close query on invalid use of HTTP/2 with /query endpoint (6.0.x)
PR-5871 - fix: remove schema compat check if schema exists
PR-5844 - fix: Reuse KsqlClient instance for inter node requests (#5742)
PR-5837 - fix: always use the changelog subject in table state stores
PR-5786 - fix: better error on DECIMAL type with PROTOBUF format
PR-5818 - fix: Allow backported ALPN to be used and warn if no OpenSSL
PR-5783 - fix: SEC-1034: log4j migration to confluent repackaged version
PR-5788 - fix: circumvent KAFKA-10179 by forcing changelog topics for tables
PR-5763 - fix: ksqlDB should not truncate decimals
PR-5761 - support UDTFs on streams without keys
PR-5756 - fix: do not overwrite schemas from a CREATE STREAM/TABLE
PR-5730 - fix: make sure old query stream doesn’t block on close
PR-5684 - fix: join queries with SELECT * and WHERE fail
PR-5703 - fix: support CASE statements returning NULL
PR-5656 - fix: allow empty structs in schema inference
PR-5645 - fix: ensure only deserializable cmds are written to command topic
PR-5644 - fix: support GROUP BY with no source columns used
PR-5638 - feat: implements ARRAY_JOIN as requested in (#5028) (#5474)
PR-5617 - fix: don’t fail if broker does not support AuthorizedOperations
PR-5621 - fix: /inserts-stream endpoint now supports nested types
PR-5563 - feat: new split_to_map udf
PR-5559 - feat: add CHR UDF
PR-5597 - fix: fail AVRO/Protobuf/JSON Schema statements if SR is missing
PR-5591 - fix: make stream and column names case-insensitive in /inserts-stream
PR-5592 - fix: disallow requests to /inserts-stream if insert values disabled
PR-5576 - fix: Stream with no key should not fail if key has data
PR-5568 - fail aggregations that don’t select any value columns
PR-5516 - feat: reload TLS certificate without restarting server
PR-5560 - fix: set retention.ms to -1 instead of Long.MAX_VALUE
PR-5548 - feat: new UDFs for set-like operations on Arrays
PR-5552 - fix: improve print topic format detection
PR-5528 - feat: create MetricsContext for ksql metrics reporters
PR-5534 - feat: new CONCAT_WS UDF
PR-5531 - fix: inserts now support zero-valued decimals
PR-5536 - feat: new UDFs for working with Maps
PR-5546 - feat: new string UDFs LPad, RPad
PR-5523 - feat: Add encode udf
PR-5542 - feat: support TIMESTAMP being a key column
PR-5535 - feat: new UUID UDF
PR-5532 - fix: Prevent memory leaks caused by pull query logging
PR-5533 - feat: keys can now be named using the KEY syntax
PR-5529 - fix: deals with issue #5521 by adding more descriptive error message
PR-5513 - fix: update Concat UDF to new framework and make variadic
PR-5504 - feat: Adds UDF regexp_replace
PR-5501 - feat: Adds udf regexp_split_to_array
PR-5495 - feat: turn on snappy compression for produced data
PR-5515 - fix: Retry on connection closed
PR-5505 - feat: new UDFs for array max/min/sort
PR-5507 - feat: Adds UDF regexp_extract_all
PR-4909 - feat: add extra log messages for pull queries
PR-5506 - fix: allow dynamic construction of an ARRAY of STRUCTS with duplicate values #5436
PR-5461 - fix: KSQL does not accept queries when running QueryLimit - 1 queries
PR-5497 - build: reference plugin URL as property (MINOR)
PR-5478 - fix: Remove stacktrace from error message
PR-5435 - fix(client): fail queryID future if executeQuery() request fails (MINOR)
PR-5469 - fix: /inserts-stream endpoint now accepts complex types
PR-5477 - fix: Improve pull query error logging
PR-5470 - ST-1323: Add license file so sources get attached
PR-5459 - fix: avoid unnecessary warnings from PRINT
PR-5448 - feat(client): support (non-streaming) insert into in Java client
PR-5455 - fix: allow setting auto.offset.reset=latest on /query-stream endpoint
PR-5425 - fix: exclude window bounds from persistent query value & schema match
PR-5431 - fix: fail on WINDOW clause without matching GROUP BY
PR-5340 - feat: Expose Vert.x metrics
PR-5444 - Fix markdown for function references
PR-5385 - feat: INSTR function #881
PR-5412 - feat: introduce RegexClassifier for classifying errors via cfg
PR-5392 - feat: Pull Queries: QPS check utilizes internal API flag to determine if forwarded
PR-5355 - How-to guides: first set
PR-5391 - fix(client): expose query ID from executeQuery() return type
PR-5386 - fix: Block writer thread if response output buffer is full
PR-5374 - feat: expose JMX metric that classifies an error state
PR-5363 - feat: drop WITH(KEY) syntax
PR-5371 - feat(client): support push query termination in Java client
PR-5382 - MINOR: Use kafka.scala.version variable in pom file
PR-5380 - MINOR: Update mbknor-jackson-jsonschema_2.12 artifact to mbknor-jackson-jsonschema_2.13
PR-5381 - fix: Override Scala version in our build
PR-5360 - fix(client): Java client error handling
PR-5270 - fix: remove leading zeros when casting decimal to string
PR-5273 - feat: implement earliest_by_offset() UDAF
PR-4975 - fix: fix NPE in latest_by_offset if first element being processed has null…
PR-4782 - feat: Implement latest_by_offset() UDAF
PR-5212 - feat: Adds the ability have internal endpoints listen on ksql.internal.listener
PR-5332 - fix: Allow value delimiter to be specified for datagen
PR-5093 - Any key name
PR-5200 - feat(client): Java client with push + pull query support
PR-5287 - fix: allow structs in schema provider return types
PR-5205 - fix: /query-stream endpoint should serialize Struct (MINOR)
PR-5239 - fix: Move Cors handler in front of /chc handlers
PR-5124 - Backport array len
PR-5213 - fix: use sr’s jackson-jsonschema version
PR-5158 - feat: Remove dependency to ksqldb-common
PR-5198 - fix: use schema in annotation as schema provider if present
PR-5193 - feat: Do not allow access to new streaming endpoints using HTTP1.x
PR-5189 - build: pin streams to 5.5.0 instead of 5.5.0-SNAPSHOT
PR-5162 - bug: stop multi-way joins failing on ambiguous column errors
PR-5150 - feat: klip-14 - rowtime as pseduocolumn
PR-5145 - fix: do not allow implicit casting in UDAF function lookup
PR-5122 - Event-driven microservice tutorial
PR-5104 - feat: fail startup if command contains incompatible version
PR-5105 - feat: transient queries added to show queries output
PR-5137 - BREAKING CHANGE: minor: allow more precise configuration of rocks bounds
PR-5142 - fix: support quotes in explain statements (MINOR)
PR-5126 - feat: Implement cluster health check endpoints
PR-5123 - refactor: Migrate old websocket query endpoint
PR-5139 - fix: include lower-case identifiers among those that need quotes (#3723)
PR-5081 - feat: add multi-join expression support
PR-5075 - fix: replace ‘null’ in explain plan with correct op type (MINOR)
PR-5023 - fix: Improve error message for where/having type errors
PR-5064 - feat: support multi-way joins
PR-5054 - fix: add content type header to ported endpoints response (MINOR)
PR-5069 - fix: make endpoints available while waiting for precondition
PR-5048 - fix: reject requests to new API server if server is not ready
PR-5047 - fix: make endpoints available while waiting for precondition
PR-5059 - fix: Make sure internal client is configured for TLS
PR-4985 - fix: Don’t set chunked transfer encoding for HTTP2 proxy
PR-4974 - fix: Catch server startup exceptions
PR-4945 - feat: Support wildcards for unauthed paths
PR-4944 - feat: CORS support for Vert.x based ksqlDB
PR-4851 - feat: Migrate /ksql and /ksql/terminate endpoints to Vert.x
PR-5019 - fix: improve handling of NULLs
PR-5052 - fix: json creator for QueryStatusCount should match getter (MINOR)
PR-5029 - feat: add KsqlQueryStatus to decouple from KafkaStreams.State
PR-4921 - Two new tutorials: materialized views + streaming ETL pipelines
PR-5013 - feat: support more advanced suite of LIKE expressions
PR-4952 - Support aliases in PARTITION BY and GROUP BY
PR-5002 - feat: speed up restarts by not building topologies for terminated queries
PR-4986 - Distinguish keys for tables by naming them PRIMARY KEY
PR-4875 - feat: scatter gather query status from all servers in cluster for ‘SHOW QUERIES [EXTENDED]’ statement
PR-4993 - fix: Allows unclosed quote characters to exist in comments
PR-5012 - fix: allow trailing slash in listeners config (MINOR)
PR-5005 - fix: query id for TERMINATE should be case insensitive
PR-4977 - fix: generated aliases for struct field access no longer require quoting
PR-5006 - Remove unused / duplicate methods (MINOR)
PR-4951 - feat: Adds rate limiting to pull queries
PR-4972 - fix: When pull query metrics enabled, throws NPE
PR-4956 - fix: output valid multiline queries when running SHOW QUERIES
PR-4966 - fix: Pass correct request configs when making pull query request
PR-4942 - fix: do not allow grouping sets
PR-4926 - fix: throw error when column does not exist during INSERT INTO
PR-4940 - fix: do not allow GROUP BY and PARTITION BY on boolean expressions
PR-4908 - fix: Don’t wait for streams thread to be in running state
PR-4878 - fix: handle leap days correctly during timestamp extraction
PR-4893 - fix: Infer TLS based on scheme of server string
PR-4889 - fix: If startup hangs (esp on preconditions), shutdown server correctly
PR-4872 - fix: ‘drop (stream|table) if exists’ fails if source does not exist
PR-4827 - fix: avoid duplicate column name errors from auto-generated aliases
PR-4822 - fix: Stop PARTITION BY and UDTFs that fail from terminating the query
PR-4850 - feat: expand IFNULL to support all types
PR-4859 - fix: Filter out hosts with no lag info by default
PR-4868 - test: add test coverage to ensure that historical commands from previous versions always work in the current version
PR-4837 - fix: Returns empty lag info for a dead host rather than last received lags
PR-4867 - fix: avoid long possible format lists on PRINT TOPIC for nulls
PR-4829 - feat: add COALESCE function
PR-4653 - fix: CommandRunner metric has correct metric displayed when thread dies
PR-4823 - fix: do not filter out rows where PARTITION BY resolves to null
PR-4814 - fix: use describeTopics() for Kafka healtcheck probe
PR-4771 - feat: add internal request to KsqlRequestConfig and SessionProperties
PR-4816 - fix: do not lose old key values on repartition
PR-4830 - fix: add deserializer for SqlType
PR-4809 - fix: Removes unnecessary error logging for heartbeat since this is ex…
PR-4754 - fix: better error code on shutdown
PR-4805 - fix: only create processing log stream if it doesn’t exist
PR-4743 - fix: make CLI unit-tests platform independent
PR-4785 - fix: docker image directories in light of renames (MINOR)
PR-4608 - feat: Add metrics for pull queries endpoint
PR-4778 - feat: Security plumbing
PR-4777 - fix: add transfer-encoding:chunked header if Jetty omits this
PR-4733 - feat: support for tunable retention, grace period for windowed tables
PR-4728 - feat: add REGEXP_EXTRACT UDF
PR-4717 - fix: create schemas at topic creation
PR-4709 - fix: stop logging about command topic creation on startup if exists (MINOR)
PR-4725 - feat: add ARRAY_LENGTH UDF
PR-4713 - feat: Integrate new API with existing security framework
PR-4714 - test: avoid NPE if user puts input topic name in output
PR-4685 - feat: basic auth for new API server
PR-4681 - fix: fix the error message for failing to wait for CommandRunner to catch up in DistributingExecutor
PR-4632 - fix: change default exception handling for timestamp extractors to log-and-continue
PR-4675 - fix: change configOverrides back to streamsProperties
PR-4597 - fix: added special handling for forwarded pull query request
PR-4650 - fix: Return immutable list of streams metadata
PR-4643 - fix: idempotent terminate that can handle hung streams
PR-4489 - fix: csas/ctas with timestamp column is used for output rowtime
PR-4625 - fix: support partial schemas
PR-4509 - feat: display errors in CLI in red text
PR-4621 - fix: patch KafkaStreamsInternalTopicsAccessor as KS internals changed
PR-4606 - fix: StoreQueryParameters name change
PR-4573 - feat: add the topic name to deserialization errors
PR-4575 - feat: log groupby errors to processing logger
PR-4590 - feat: Simple perf tool for new API
PR-4581 - feat: Provide upper limit on number of push queries
PR-4511 - [SECURITY] Use HTTPS to resolve dependencies in Maven Build
PR-4512 - feat: Plug insert streams into backend
PR-4551 - feat: enhance PRINT TOPIC’s format detection

REST Proxy¶

PR-738 - KC-613: Remove share/java/kafka-rest dependencySet.
PR-737 - KC-613: Split share/java/kafka-rest in kafka-rest-bin and kafka-rest-lib.
PR-734 - CLIENTS-1659: Remove V1 API Kafka core dependency and associated Zookeeper Client configs
PR-731 - CLIENTS-1657: Explicitly depend on Guava.
PR-729 - METRICS-1988: add confluent-telemetry directory to kafka-rest-run-class
PR-723 - Refactor requests to fix go sdk generation bug and add parameterized servers to openapi-spec
PR-726 - Add [Consumer|Producer]Performance tools back
PR-722 - Add reassignment API to openapi spec.
PR-721 - Add ACL APIs to openapi spec.
PR-719 - Add consumers (admin) APIs to openapi spec.
PR-718 - CNKAF-1017: Update v3 error response format
PR-715 - Add config endpoints to openapi spec.
PR-714 - OpenAPI Spec for a small subset of the V3 APIs.
PR-712 - Move MetricsReporter Config extraction to rest-utils
PR-711 - CLIENTS-1613: Add configuration to enable/disable V2 and V3 APIs.
PR-710 - Remove cluster-wide topic configs.
PR-709 - KC-613: Use slf4j version from common pom.xml.
PR-707 - Change resource label resource.kafka.cluster.id to resource.cluster.id
PR-703 - Enable setting cluster_id on KafkaRestMetricsContext
PR-706 - CNKAF-951: Add catchingCompose utility to handle exceptions.
PR-704 - CLIENTS-1599: Add constructor accepting path to KafkaRestApplication.
PR-697 - CLIENTS-1597: Add relationship reassignment and refactor reassignment resource
PR-701 - Rename alter config UPDATE to SET.
PR-700 - Add Batch Alter (UPDATE/DELETE) Broker/Cluster/Topic configs.
PR-696 - Add LIST/GET Consumer Group/Consumer/Consumer Assignment.
PR-693 - Add search/create/delete ACLs APIs.
PR-687 - CNKAF-739: Reassignment endpoint for a given topic name
PR-685 - CNKAF-739 - Reassignments endpoint for given topic name and partition id
PR-692 - Remove AdminClientWrapper.
PR-691 - Migrate off JSON:API cleanup.
PR-682 - CNKAF-739: Add Rest endpoint for list all partition reassignments
PR-690 - Switch from application/vnd.api+json to application/json.
PR-689 - KIP-606 MetricsContext
PR-686 - Migrate off JSON:API.
PR-683 - KIP-226: Add cluster-wide dynamic default configs.
PR-678 - ST-1323: Add license file so sources get attached
PR-679 - Migrate internal entities to Google’s Auto-Value.
PR-680 - Add static factory to Cluster.
PR-677 - KIP-226: Add support for configs source and synonyms.
PR-675 - Give more unique names to the testing containers.
PR-674 - Open visibility of configurePre/PostResourceHandling.
PR-672 - Fix REST Proxy injection-based authentication.
PR-671 - Make ParititonData public
PR-670 - Add JsonCreator to v3 entities
PR-667 - Copy admin client config from REST Proxy. Need for authentication tests.
PR-658 - Remove all methods from AdminClientWrapper.
PR-665 - CPKAFKA-4979: Remove non-determinism from searchReplicasByBroker.
PR-662 - Add find partition replicas by broker id endpoint.
PR-660 - CLIENTS-1505: Add broker config delete endpoint
PR-659 - CLIENTS-1504: Add update endpoint for broker config
PR-654 - Remove AdminClientWrapper.getBrokerIds() and add JSON:API errors.
PR-656 - Fix default Content-Type, when none or any (/) is provided.
PR-655 - Remove spurious topic/partitions existence test when producing.
PR-647 - Remove V1 APIs and all their dependencies.
PR-652 - CLIENTS-1501: Add broker configs endpoints
PR-617 - Fix compile error due to KIP-515 (ZK TLS)
PR-191 - SEC-1034: log4j migration to confluent repackaged version
PR-199 - Rest CORS option skip will now default to true
PR-195 - Add a config flag to skip OPTION when CORS is enabled
PR-193 - Removed OPTION omission for CORS
PR-192 - All original configs must be passed to MetricReporters
PR-190 - Use prefix to propagate metrics reporter configs
PR-186 - Extensible Application Metrics instance configuration
PR-180 - KIP-606 MetricsContext
PR-183 - ST-1323: Add license file so sources get attached
PR-182 - Add getter for Application’s Metrics object
PR-178 - Migrate rest-utils to use AK library instead of common-metrics
PR-177 - SEC-929-Customized HTTP response headers.
PR-172 - SEC-592: Customize Jetty request queue and threadpool size
PR-174 - Catch invalid format exception: #501

Schema Registry¶

PR-1624 - DG-857 Fix Avro converter for top-level bytes object
PR-1616 - DG-845 Have typeName override Object.class type; fixes #1615
PR-1611 - Correct License in Client library
PR-1598 - CP-644: Include CP jars required for Security and monitoring
PR-1601 - DG-820: Fix telemetry reporter handling of leadership changes
PR-1590 - DG-810: Explicitly add kafkastore.topic config to MetricsReporter instances
PR-1594 - DG-813 If use.latest.version is true, use on Protobuf dependencies
PR-1588 - DG-811 Allow JSON schema spec version to be specified
PR-1589 - DG-812 Derive file extension during download
PR-1587 - Upgrade to Kotlin 1.4.0
PR-1586 - DG-800 Cache ProtobufSchema using name in Converter
PR-1528 - SEC-1034: log4j migration to confluent repackaged version
PR-1580 - DG-789 Fix for CVE-2020-15824
PR-1579 - DG-788 Allow anyOf to be backward compat with oneOf/allOf
PR-1578 - DG-767 Calculate proper defaults for client ssl configs
PR-1576 - DG-767 Use passed in defaults for client ssl configs
PR-1577 - DG-774 restore ssl client configs
PR-1575 - DG-780 Add support for number with format for JSON Schema
PR-1569 - METRICS-1988: add confluent-telemetry directory to schema-registry-run-class
PR-1574 - DG-774 Avoid regenerating default names for Protobuf data
PR-1567 - DG-740 Convert connect defaults for JSON Schema
PR-1563 - DG-713: Disable automatic setup of telemetry reporting
PR-1561 - DG-723 Enable defaults for JSON Schema
PR-1556 - DG-715: Change telemetry cluster label names
PR-1550 - DG-695 Normalize JSON ref schemas before comparing
PR-1548 - ST-3988: update swagger-core to 1.6.2
PR-1547 - ST-3988: update swagger-core to 1.6.2
PR-1538 - DG-670 Treat anyOf same as oneOf in JSON Schema converter
PR-1537 - DG-652 Fix missing negation in conditional
PR-1535 - DG-652 Handle imports during protobuf compat checks
PR-1531 - DG-628 Allow changes to id during json schema compat checks
PR-1530 - DG-607 Add lenient processing for numbers in oneof
PR-1523 - DG-607 Ensure fields for union are optional
PR-1520 - Handle null error stream in RestService
PR-1522 - DG-606 Add option to not generate oneof for nullable fields
PR-1517 - Upgrade maven-plugin dependencies to 3.6 in 5.4.x
PR-1510 - Add metric name test
PR-1509 - align naming conventions
PR-1508 - Add telemetry reporter SR config option
PR-1507 - Use config originals for retrieving TR enablement
PR-1493 - Use MetricsContext to set JMX prefix
PR-1502 - Split telemetry cluster label
PR-1500 - Only add TelemetryReporter if explicitly enabled in config
PR-1483 - Telemetry reporter integration
PR-1497 - DG-554 Pass refs when forwarding register requests
PR-1496 - DG-535 Handle recursive Protobuf schemas
PR-1495 - DG-402 Missed some places where dotted names can occur
PR-1492 - Fix NPE during tombstone metric update
PR-1485 - Add node information to SchemaRegistryProtocol
PR-1487 - DG-527 Omit defaults for proto 3 schemas (fixed in wire)
PR-1482 - DG-499 Fix JSON Schema compat checks for partial models
PR-1484 - DG-524 Catch NPE during Avro compat check
PR-1480 - DG-27 Use leader/follower terminology
PR-1477 - Allow custom update handlers; fixes #382
PR-1476 - Add unit test for mvn validate action
PR-1468 - Implement new Schema Registry Metrics
PR-1474 - Add validate action for SR maven plugin
PR-1473 - Add ability to use latest version during serialization
PR-1471 - DG-436 Pass existing Avro schema instead of regenerating
PR-1470 - Allow unknown properties while deserializing SchemaRegistryProtocol.Assignment
PR-1469 - Fix message name on schema when version is included; fixes #1467
PR-1466 - DG-450 Perform proper matching of subschemas
PR-1461 - DG-268 Deprecate ZK-based coordination
PR-1455 - Fix Scala 2.13 deprecations
PR-1450 - DG-400 Support enums using allOf in JSONSchema converter
PR-1451 - Upgrade swagger deps
PR-1449 - DG-402 Add support for packages in ProtobufConverter
PR-1440 - Fix null schema type
PR-1433 - use property for jackson-jsonschema
PR-1436 - feat: adds subjectName in the exception for incompatible schema
PR-1437 - Fix cyclic schema references
PR-1431 - Cherry-pick for permanent delete
PR-1426 - DG-377 Allow generated classes to use refs during serialization
PR-1427 - DG-399 Use envelope in JsonSchemaUtils.toObject
PR-1417 - Allow id to be passed to console prod/cons; fixes #160
PR-1422 - Fix cycles in Avro schemaEquals
PR-1420 - SEC-929–Added one testing for verifying a feature added to rest-utils repo.
PR-1416 - DG-399 Use envelope in JsonSchemaUtils.toObject
PR-1397 - DG-91 Schema permanent delete.
PR-1389 - DG-91 Ability to permanently delete schemas
PR-1369 - DG-267 - Fix Protobuf converter enum handling
PR-1368 - Performance: Use singleton ObjectMapper
PR-1344 - Enhanced serialization exception
PR-1334 - DG-237: Migrate SchemaRegistry CP KafkaMetrics to AK KafkaMetrics
PR-1326 - Add API to retrieve supported schema types

Commercial Features¶

Control Center¶

Introduced self-balancing cluster options on the UI, enabling users to easily balance workloads and dynamically expand and shrink clusters.
Introduced Tiered Storage options on the UI, giving users the option to add and scale storage.
Enabled auto-updates to the user interface by default.

Confluent Server¶

CPKAFKA-6013: Skip tier state presence validation for deleted partitions
KSTORAGE-1082: tier replication causes incorrect HWM after restore (#2507)
KSTORAGE-1086: Better validations to check if tiering is disabled
[CNKAF-1144] Confluent.operator.managed property (#2484)
[Backport]METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
KSTORAGE-1040: Better checks to prevent tiering from being disabled dynamically
METRICS-2159: use a loading cache to avoid expensive pattern matching calls
CPKAFKA-5882. Add copyDependantLibs dependency to ce-sbk and ce-broker-plugins module
KC-845: Ensure LogConfig includes broker defaults to pass schema config validation
CPKAFKA-5873 Don’t load Netty classes when using default SSL engine
CONFLUENT: Remove shading from ce-broker-plugins and ce-sbk
CPKAFKA-5851: Align joda-time version with common
CNKAF-863: Only showed broker removal status for given broker.
KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
CPKAFKA-5849: Don’t log password configs in request log filter
CONFLUENT: Remove unused jacksonDatabind version definition
CPKAFKA-5795: Change guava dependency to 28.1-jre
CPKAFKA-5796: Remove ZK dependency from ce-licensing
CPKAFKA-5835: Fix regression in cluster link compatibility test
METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
[CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub. (#2408)
backport METRICS-2098 exclude zstd/lz4 from telemetry-clients to avoid conflicting versions
METRICS-2090: Remove duplicate labels on metrics
METRICS-2097: Don’t capture ‘total_size` and ‘tier_size` metrics
KC-637 cherry pick to 6.0.x
KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms. (#2353)
SEC-866-6.0.x again
CONFLUENT: Remove the soak clients status reporter cronjob
METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
[CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces (#2326)
CNKAF-1058; Ensure that Control Request Batching does not send LeaderAndIsrRequest with yet unknown broker(s)
KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA (#2309)
backport METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics (#2282)
CPKAFKA-5762: Fix topic deletion in system tests
CNKAF-1039: Update error for insufficient metrics to compute plan
KC-671: SBK system test with CL
METRICS-1988: add confluent-telemetry directory to kafka classpath
CONFLUENT: Bump cc-trogdor chart docker image to cc-trogdor:v0.207.0-5.3.0-ce-SNAPSHOT
[Cherry 2367] [SEC-1345] Read HTTP Reason phrase when connection closed
Revert “SEC-866 - Migrate audit log “other” category into “management” category. (#2330)”
backport METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar (#2332)
[CLIENTS-1633] Schedule MDS url refresher asynchronously
Increase root FS to 20GB
METRICS-1763, METRICS-1767 Configure telemetry metrics whitelist for Connect and MQTT Proxy Proactive Support (#2241)
SEC-866 - Migrate audit log “other” category into “management” category.
KC-391: Source topic deletion system test for CL
KAFKA-10282; Remove Log metrics immediately when deleting log (#2342)
CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only - cherry pick to 6.0
Rebalance Cluster System Test
CNKAF-1008: Add Broker/Remove Broker proposal logging
KC-633: Auth failures system test for Cluster Linking
[CHERRY-2308] Move kafka-http-server project into clients project
KC-638: System tests with tier storage enabled on destination
KC-754: Cluster link compatibility system tests
KC-631 Good path cluster linking test using SASL_SSL
backport METRICS-1992: Fix schema registry cluster id resource label (#2285)
CONFLUENT: Fix broken cluster link system tests in 6.0.x
CNKAF-1029: Ensure Executor state is reset after unexpected exceptions
KC-735: Improve cluster link addition.
CNKAF-1035: Roll cluster while a new broker is getting added
METRICS-1998: add rbac metrics to default metrics list (#2269)
backport METRICS-1993: add connect.cluster.id label (#2264)
Updated default topic replicas factor to 3 for audit log.
KC-741: Don’t include passwords in cluster link request logs
KC-736: Allow deletion of cluster links if mirrored topics are marked for deletion
KSTORAGE-896: fix tier deletion manager system test (#2256)
CNKAF-822: Cherry pick add broker system test to 6.0.x
KC-725: Backport of KC-725 to 6.0.x
backport METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist. (#2266)
backport METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup (#2253)
KSTORAGE-864 Fetch requests to return NotLeaderForPartition during ULE recovery
CNKAF-819: Test SBK broker removal with roll
METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
Backporting fix for METRICS-1764 and METRICS-1766
update kafkatest template for balancer replication factor (for < 3-node clusters)
KC-638: Cluster linking system test with tiered storage enabled in source cluster
CNKAF-825: Introduce a system test for replica placement constraints
CNKAF-1014: Rename usages of SBK in logs to DataBalancer
CNKAF-613: Introduce a broker removal system test
METRICS-1894:Telemetry Reporter is missing labels configured in prope…
METRICS-1817: add include regex default value to Provider interface
METRICS-1938: upgrade jose4j version (#2219)–backport
METRICS-1929: use advertised listeners/host-port when deriving local producer config
KSTORAGE-892 Re add couple of lines of ULE code that got deleted duri…
Backporting config/events Telemetry Reporter changes along with all shading issues.
Added one more toJson method in protoserde for webhook alert
(backport) MMA-8432: fix wrong package name
CNKAF-1004: Don’t describe removed broker configs
METRICS-1895: Update telemetry client version
KC-705: Retry metadata topic consumer.position() until authorizer startup times out
METRICS-1918: bump gson from 2.8.5 -> 2.8.6
(backport) prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
(Backport) include UnderMinIsrPartitionCount in the default broker metrics list
CPKAFKA-2852: CCloud metadata listener thread should not die on unexpected exception
SEC-1276 Allow empty resourceNames in CRNs
SEC-384: Use the correct log level for debug output
KC-634: Cluster link system tests with broker failures
CNKAF-964: Shade opencensus protobufs into SBK jar (#2155)
Add KafkaRest Metrics Provider
KSTORAGE-845 Check if tier topic exists already before calling AdminZkClient#createTopic
CNKAF-952: Update javadoc for removeBroker ConfluentAdmin API with InvalidBrokerRemovalException
CNKAF-851: log goals skipped during detection in one line
KC-630: Cluster link system test support with good path mirror test
CONFLUENT: Prevent log start offset overrun during retention
KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order (#2133)
Back port [METRICS-1815] Connect client metrics missing some resource level tag…
CNKAF-936: Remove dot-handling logic in metrics processor (#2121)
METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
Deprecate KafkaHttpServlet in favour of rest-utils Application. (#2135)
[METRICS-1844, METRICS-1839, METRICS-1756,METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
[CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle, handle ab…
METRICS-1801: align telemetry domain metric names
METRICS-1835 Shade confluent-serializers dependencies in confluent-metrics (#2111)
METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter (#2096)
METRICS-1810 use SBK topic replicas config if present (#2109)
fix: fixes exception in server shutdown due to metrics registration
METRICS-1809: add telemetry settings to default server.properties file
[CNKAF-927] Fix typo in server.properties. (#2090)
[CNKAF-924, CNKAF-926] Fix Databalancer Topic RF config, remove Teleme…
CONFLUENT: Added check for CL metrics for 6.0.x
CPKAFKA-4955: Fix failing testTopicDeletion
Removed two testing files accidentally added.
SEC-1130-Inject ConfluentAdmin from ce-kafka
backport METRICS-1820: use password object for telemetry api secret
CNKAF-893: Update SBK to work with secure Zookeeper
METRICS-1816, METRICS-1705 local telemetry exporter metrics whitelist configuration for SBK (#2031)
CONFLUENT: deleting extra topic config change in cluster linking test
CNKAF-909: Add direct dependency on opencensus for SBK (#2057)
KAFKA-10149: Allow auto preferred leader election when partitions are…
CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
SEC-1034: use confluent-log4j instead of log4j
Add RackAwareGoal to goal list
CNKAF-884: Remove unnecessary metrics (#2059)
Migrate MDS to run on KafkaHttpServer implementation
KC-505: integration tests for CL metrics for 6.0.x
METRICS-1717 use shared domain for jvm level metrics (#1949)
CNKAF-854: Disabling SBK should persist the broker removal cancellation status
[CNKAF-873] Don’t require network capacity configs. (#1996)
SEC-1183 - Treat empty authority in CRN pattern as a wildcard
CNKAF-882: Add proposal logging for add/remove
CNKAF-837: Add broker should be able to add removed brokers.
KC-393: Integration tests for mirroring compacted topics
Add inter broker listener endpoint information to HTTP server binder.
CNKAF-773: Do not throw an exception when the broker is not present in the cluster model
CNKAF-847: Complete removal callback when there are no proposals to execute
KSTORAGE-802: Allow specifying AWS credentials through configuration file
[CNKAF-848] Set preferred user SBK config in server.properties. (#1999)
[CNKAF-719] Add option to disable dynamic throttle setting when stati…
KSTORAGE-819: Force roll segments only for partitions eligible for tiering
KSTORAGE-799: reduce Log.lock contention in ArchiveTask segment selection logic
KSTORAGE-774: Fix active segment roll logic when roll.min.bytes setting is enabled
KSTORAGE-766: Ensure delete segment throttler does not block active segment rolls
KSTORAGE-769: Fix to remove the costly toSet conversions in disk throttler
KSTORAGE-758: Include ERROR status in TierPartitionStatus.isOpen()
KSTORAGE-595 NPE in TierDeletedPartitionsCoordinator after tier topic is reassigned / deleted from broker
KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util (#1906)
KC-611: Fix mirror topic create processing incorrect config.
KSTORAGE-547: state recovery injection tooling
CONFLUENT: Fix Confluent provider start future completion, add unit tests
[CNKAF-881] SBK: Don’t treat all new brokers as empty. (#2005)
SBK/CruiseControl: low utilization fix
CNAKF-833, CNKAF-842: Fix missing request rate metric and enable tele…
RackAwareGoal should allow for uneven replica distribution based on t…
CNKAF-788: Introduce proper API exceptions in the broker removal API
METRICS-1708: add telemetry version header for forward-compatibility (#1883)
CNKAF-791: Change data format of failedBrokers in ZK
CNKAF-807: Remove zkclient from gradle dependencies.
CNKAF-575: Remove IOtec:ZkClient dependency.
[CNKAF-776] Switch SBK Maven publish group to “io.confluent”
[METRICS-1609] dynamically add telemetry reporter at runtime
KC-389: Retry segment deletion in cluster link test to fix timing issue
KC-389: Cluster link integration tests with tier storage enabled
CPKAFKA-5207: Complete authorizer start future only when sufficient nodes are ready
CNKAF-843: Fix remove broker validation check
Persist the broker removal state and handle failover
CNKAF-773: Do not throw exception when there are no replicas to move on broker removal
CNKAF-428: Integration test for replica placement self-healing
Add retries to telemetry sampler integration test (#1922)
[CNKAF-803] Add a metric to track the number of pending add-broker requests
KC-72: Add topic listing for cluster links.
CPKAFKA-5207: Fail fast during broker start up if multiple brokers use same MDS URL
KC-536 bug fix to make offsets/acls configs dynamic (#1893)
CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
fix telemetry reporter reconfiguration
update ksql and streams provider
CNKAF-649: Handle broker restart during broker removal
KC-390: Cluster link authorization tests
backport [METRICS-1564] telemetry - derive interbroker client configs (#1816)
(Backport) make commit-id optional, update whitelist to include C3 metrics
SEC-1083 - AuditAdmin role’s scopeType = Cluster
KC-392: Integration tests for cluster link quotas on source and destination
KC-582: Add multi-tenant support for alterMirrors.
CNKAF-795: Ensure SBK clients have bootstrap.servers configured
Backport PR 1897 to 6.0.x
SEC-1083 - Add AuditAdmin role (#1861)
SEC-1039 - TopicSpec builder sets partitions and RF.
CNKAF-794: Properly wrap broker removal exceptions in listener and improve remove broker test
Removed disk throttling dependency on produce backpressure config
Jvm Collector for Telemetry Reporter
METRICS-1698 Setting default telemetry.enabled and exporter.enabled values to false.
KC-224: make leader epoch cache flush asynchronous when stamping new leader epoch
KC-537: apply topic-create-policy when auto-creating topics while fetching metadata
CNKAF-782: MetricsReporterSamplers are configured in an odd manner.
CONFLUENT: Fix cipher name for ChaCha20
CNKAF-428: Replica placement
METRICS-1152 document how to enable the telemetry-reporter in CPD
KC-534: Adds forceful cluster link deletion.
KCORE-54: Controller should accumulate metadata into fewer requests to mitigate handling delays
CNAKF-645: Add metrics sampler to consume from telemetry reporter
Generic telemetry reporter
KSTORAGE-649: Remove duplicate library dependencies accidentally included in merge.
[cnkaf-730] part 2: Completion callbacks for add and remove brokers
[CNKAF-730] Minimal addBrokers support for SBK
KC-207: Destination CL Metrics
METRICS-1362 Updating telemetry dependency version from 1.87 to 1.98 to set new content-type
KSTORAGE-649 and KSTORAGE-660: Support sts::AssumeRole and SSE-KMS
CNKAF-697 - High bandwidth throttles during cluster roll
Make ClusterLinkIntegrationTest run as integration test
CNKAF-760: Integrate api state persistence with remove broker
Upgrade cloudevents from v0.3 to v1.0
CC-8138: Add Connect feature to inject Confluent license properties into connector properties before startup
KC-571: Move tenant prefix to cluster link metadata instead of configs
CIAM-69 Use authorizer setting to configure CC RBAC roles
[CNKAF-759] Deal with non-Anomaly executions.
SEC-994: Update Cluster Registry doc
KC-387: Multi-tenant integration tests for security updates, offset migration
KSTORAGE-752: removed disk throttler dependency on logDir existence at server startup
KC-518: Use cluster link ID as primary key for cluster link metadata.
Refactor Event Logger to make it generic
KSTORAGE-699 Rationalize ArchiveTask/DeletionTask Exceptions
CNKAF-649: Introduce integration test for SBK’s AddBroker, RemoveBroker and DescribeBroker APIs
Add confluent metrics labels for connect metrics
CNKAF-649 - Compose the broker removal operation into multiple interruptible tasks
KSTORAGE-740: Add log closing time metrics
SEC-951: Upgraded bc-fips to 1.0.2
Add Confluent metrics tag Kafka server and client
KSTORAGE-730: Allow max.partition.fetch.bytes to be configurable for tier fetches
CNKAF-752: Store and fetch remove broker api state
KSTORAGE-746; Make timing in TierTopicListenersTest deterministic
CPKAFKA-4744; Fix log cleaner test failure caused by concurrent cleaning
KC-407: Micro-benchmark for ClusterLinkFetcherThread based on ReplicaFetcherThreadBenchmark
Introducing disk-usage based throttling for producers and replication followers
cherry-pick upstream KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter (#8691)
KCORE-142 new tests and bug fix for offset command config
[CNKAF-741] Handle duplicate metrics in the DataBalancerMetricsRegistry
Add domain to Context class
CNKAF-702: At SBK startup check for api state persistence topic
KC-382: Allow alter cluster configs for subset of configs in CCloud
KC-533: Allow headers to be injected into RBAC records
CIAM-64 Allow running other Providers with the MultiTenantAuthorizer
KSTORAGE-741: Deserialize errorOffsetAndEpoch from Header during scan and init
CNKAF-649 - Broker Removal operation with in-memory state
CNKAF-617: Complete the DescribeBrokerRemovals API handler
KSTORAGE-743: prevent tier.feature use with IBP < 2.4
METRICS-1555 Exposing proxy configs as top level telemetry configs and making them dynamically configurable
KCORE-142 - Offset Migration for Cluster Linking
[CNKAF-637] Config variable for auto-healing
KSTORAGE-425 Add tiered storage metrics to MetricsReporter
CONFLUENT: Bump major CCloud version #major
KC-370: Support Authentication events in ConfluentAuditLogProvider
CPKAFKA-3743; Consumer group tool does not work on internal endpoint
KSTORAGE-720: Always return tiered storage related configurations in DescribeConfigResponse
[CNKAF-742] Detect new, empty brokers in controller; call into SBK
KC-120: Integration Tests for ZK-Based ACLs for Periodic ACL Sync
CONFLUENT: Add PowerMockIgnore for Scram API in RBAC test to fix classloader issues in subsequent tests
SEC-959 Cluster Registry - Full Cluster Info CRUD Gatekeeping
KC-517: Update cluster link integration test to use admin client.
SEC-1015: Added kafka Metrics support to Confluent Authorizer, RBAC and AuditLog.
CNKAF-649: Revise SBK Executor’s locking strategy
KCORE-154: add tiered storage and CP to downgrade system test
KC-128: Validate source topic read access on mirror topic creation.
KCORE-329 (2/2): Adds ability to alter a topic’s mirror in TopicCommand.
Created Broker removal CLI
KSTORAGE-726: handle offset gaps in tier fetches
CPKAFKA-2789; Add support-metrics-fullcollector to ce-kafka
KC-506: Use cluster link tags for cluster link fetcher metrics
KSTORAGE-627: TierMetadataComparator tool for tier-topic recovery pipeline
KCORE 506: migrate secrets to Vault
Spelling fix in LdapGroupManager
KCORE-95: Encrypt sensitive cluster link configs stored in ZooKeeper
KCORE-437: Refactor ClusterLink managers to cleanly disable/enable
CONFLUENT: Improve performance of matchingAcls and AclSeqs.find
KCORE-49: Disables mirror topic partition and restricted config updates.
KCORE-485: Process mirror state during replication only on active destination leader
KCORE-487: Add AuthenticationEvent and update Audit log event schema
CONFLUENT: Introduce separate methods in KafkaApis for consumer and follower fetch handling
Update Netty to 4.1.50 and clean up logging of NettySslEngineFactory
CIAM-45 Allow RoleBindings at path scopes
CNKAF-636: SBK broker failure detection
KSTORAGE-627: TierPartitionForceRestore, TierPartitionState hotswap and tier task recovery
CNKAF-701: Refactor SampleStore topic existence check
Added tests for KafkaCruiseControl.
CNKAF-649: Add BrokerRemoval state machine
KSTORAGE-573: epoch state cache is flushed multiple times in recovery
[METRICS-1234] telemetry - add support for multiple exporters of the same type
KCORE-389: Add initial version of Audit API and update existing code
CNKAF-677: Remove ZK fallback for SBK and move methods from ExecutorUtils to Executor
CNKAF-684: Honor low utilization threshold settings for self-healing …
CIAM-28 Add ConfluentCloudCrnAuthority with special treatment of path
KSTORAGE-717: Filter all Confluent-specific topic configs from DescribeConfigsResponse
KSTORAGE-652: FileTierPartitionState base path is not updated correctly
CNKAF-699: Skip num cores or disk capacity update if node missing
KCORE-340: Propagate new partitions in source cluster to controller
KSTORAGE-653: re-add check_cluster_state check to tier upgrade test
KCORE-141: Tenant prefixed cluster links and topic mirroring for Cloud
KCORE-331: _confluent-license topic exponential backoff
fix some downstream validation issue
Add shutdown hook to KafkaHttpServlet interface
SEC-1008 Do not close null AdminClient
CONFLUENT: Remove ConfluentLeaderAndIsr and TierListOffset requests/responses
CNKAF-704: Remove tenant prefix from CreatePartition responses
CNKAF-314: Add DescribeBrokerRemovals{Request,Response} RPC and Admin Client API
CONFLUENT: Set Scala 2.13 as the default Scala version
KCORE-436: Client interceptor for destination tenant prefixing in cluster link clients
CIAM-25 Allow CRNs with Paths but no Clusters
emit deltas as cumulative metrics
CNKAF-649: (part 1/2) Introduce ShutdownManager for SBK broker removal
CNKAF-697 (part1): Increase min broker quota constants
CONFLUENT: Improve error message when cluster linking is disabled
KSTORAGE-412 Server side handling for the unclean leader flag from LeaderAndIsr re…
METRICS-1514 collect one-minute rates from yammer meters
Upgrade telemetry api to latest version
CONFLUENT: Enable fatal warnings with Scala 2.13 and fix warnings
CONFLUENT: Make createSslEngine(String, String) public again
KCORE-329: Alter mirrors API.
KCORE-426: Server should not resolve DNS on establishment of SASL connection
CPKAFKA-4923: Disable audit logging in RbacTest::test_ldap_to_rbac_authorizer_upgrade
CNKAF-632: RemoveBrokers api plumbing
switch downstream validation
[MINOR] Remove scoverage plugin from SBK build.
Bump Chart versions
Merge changes from Apache Kafka
CONFLUENT: Always close consumer in ConsumeBenchWorker
KCORE-438 (1/2): Remove topic read test on cluster link creation.
CNKAF-638: Introduce SBK internal RPC client
KCORE-41: Disable local produce requests for mirror topics.
CONFLUENT: Fix build warnings in new ACL sync code
CLIENTS-1476: Create Kafka HTTP Server interfaces and loading mechanism.
KCORE-99: Periodic ACL Migration
CNKAF-622: Properly handle dynamic config changes.
KCORE-435: Propagate cluster link create/delete using ZK change notification
KCORE-427: Add broker config to enable cluster linking feature
CONFLUENT: Don’t reach into java.lang.reflect.Field internals
KSTORAGE-683: Alter fencing logic for recovery work
CPKAFKA-4982 -Change Trogdor’s Helm chart to deploy in parallel.
KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
KCORE-167: Process replication failures with retry timeout to move partition to failed state
CONFLUENT: Fix locking during metadata restart for link reconfiguration
KSTORAGE-677: FIX: Reset archiver max lag whenever there is no lag
remove unnecessary labels from kafka server resource
CNKAF-629: Add InitiateShutdown Request
KCORE-55: Adds mirrorTopic and clusterLink to topic creation API.
KCORE-53: Dynamic security config update for inter-cluster connections
CNKAF-483: Added static and dynamic configs for request backpressure tuning
KSTORAGE-26: Fix non-deterministic TierEpochStateReplicationTest
CNKAF-593: Consolidate all thread creation in KafkaCruiseControlThreadFactory
KCORE-104: Fix truncation during unclean leader election for linked partition
CONFLUENT: Simplify Gauge definitions in Partition
CNKAF-610: Rename StartRebalance RPC and DrainBroker Admin API to RemoveBrokers
[CNKAF-606] Stop proposal execution on Cruise Control shutdown/SBK deactivation
[CNKAF-558] Handle SSL interbroker listeners for SBK clients.
CONFLUENT: Ensure license validator is closed in test to avoid impacting subsequent test
KSTORAGE-648: remove S3TierObjectStore multi-part log segment upload support
CNKAF-596: Add a test to verify SBK API access with RBAC
KSTORAGE-628: Implement TierPartitionFence injector tool for recovery
KSTORAGE-63: pt2, add TierTopicListeners, add getAndRemoveAll and tests
KSTORAGE-63: pt1, add TierPartitionForceRestore metadata
SEC-912 - Allow MDS to make changes to dynamic configs
KCORE-341: Fix permission type in authorizer debug log entry
SOAK: Revise soak workloads
[SEC-912] Allow MDS to make changes to dynamic configs
KSTORAGE-631: ducker up should not require GCS credentials to be activated in Dockerfile
CNKAF-577: Handle null config values when describing static throttle configs
CONFLUENT: Add support for Scala 2.13
WFLOW-51: Implement a beginShutdown hook which allows for triggering ControlledShutdown
KCORE-42 (4/4): Adds mirror SyncTopicsConfigs task.
KCORE-48: Broker implementation for cluster linking APIs.
KCORE-48 (partial): Adds ClusterLinkCommand.
KCORE-330: Allow brokers to complete start up with configured license
[CNKAF-594] Support broker shutdown for SBK
[CNKAF-492] Set some more config parameters for SBK
Update cloud Dockerfile to use JDK 14.0.1
CNKAF-313: Migrate SBK to use DataBalancerMetricsRegistry
CNKAF-549: Add metric for self healing enabled
Instructions for exporting metrics to Sandbox env
KSTORAGE-644: Separate fencing error metrics for primary & catchup consumers
CONFLUENT: Update JDK to 11.0.7 by bumping cc-base to 5.1.0
KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
KCORE-137 (partial): Expands per-topic cluster link state.
[SEC-895] ConfluentAuditLogProvider should handle reconfiguration wit…
CONFLUENT: Reduce allocations in requests via buffer caching
KAFKA-9864: Avoid expensive QuotaViolationException usage (#8477)
CNKAF-50: Add metrics for replication throttling
KCORE-51: Adds cluster link metadata object.
CNKAF-585; Multi tenant interceptor rejects create topic requests with NO_REPLICATION_FACTOR or NO_NUM_PARTITIONS when quotas are enabled
KCORE-316: Don’t load configs from ZK when the log has already been loaded
Fix MultiTenantTest
CPKAFKA-1836: Simplify forwarding scripts by just adding .sh suffix
CONFLUENT: add utility method to write public keys to pem files (#19)
CAAS-1960: Record authenticated tenant connection/disconnection metrics
KAFKA-7304: Limit connection accept rate to prioritize other channels (#29)
CONFLUENT: Add Aegis Proxy
CONFLUENT: Update cc-base to v2.3.0
Merge changes from apache kafka
CPKAFKA-2164: Ensure that we specify a charset when writing out and reading string bytes
CONFLUENT: Fix unused compiler warnings
[SEC-901] Speed up SameClusterTest
KCORE-42 (partial): Adds functions for merging source + mirror topics…
SEC-728 - MDS Cluster Registry - Enable Dynamic Configuration
KCORE-48 (partial): Adds ClusterLinkClientManager for managing admin …
Towards KCORE-165 (partial): Adds ClusterLinkScheduler for handling cluster l…
Towards KCORE-136: Cluster linking APIs client+RPC wiring.
[CNKAF-582] Handle topic creation during checkStartupComponents
CNKAF-372: Disk capacity sizing
CONFLUENT: Remove unnecessary allocations in hot matchingAclExists
METRICS-1415 fix collection of integer gauges
CNKAF-313: Adds a MetricsRegistry wrapper class for SBK
[CNKAF-370] Support dynamic enable/disable of SBK
[CNKAF-492] Adjust the default parameters of the KafkaCruiseControlConfig.
KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
KSTORAGE-628: Introduce RecoveryUtils class with recovery tooling related functions
CPKAFKA-4647: Disable audit logging in RbacTest::test_broker_failure test
KSTORAGE-628: Introduce TierPartitionFence event type + fencing logic
add AppInfo version+commitId to metrics messages
KSTORAGE-577: System test for fencing logic, along with CLI tool to trigger fencing
KSTORAGE-632: Better logging for replica fetcher state transitions for tiered storage
KSTORAGE-381 Tooling around TierMetadataValidator to use tier-validator as a cc-service
CNKAF-536: Reduce number of warnings when SBK creating clients
[METRICS-1361] telemetry-api -> opencensus-proto
CNKAF-537: Add drainbrokers adminClientAPI
CNKAF-562: Change log level for self-healing being triggered from WARN to INFO.
KSTORAGE-613: fix log start offset initialization on full truncation
KSTORAGE-626: Remove metrics reporter configuration when configuring internal tier topic clients
[CNKAF-528] Use the inter-broker listener for SBK clients.
Prevent warning message on every client metadata update
KSTORAGE-580: Return a generic error message when compaction is enabled on a tiered topic
KCORE-40: Inter-cluster fetcher for cluster linking
Update role, Owner tags for cost reporting
KCORE-297: Fix wildcard principals for centralized ACLs and no.acl.found for ConfluentServerAuthorizer
CNKAF-541: Reduce the number of tests run in RandomClusterTest
METRICS-1232 - Whitelist check for individual and derived metrics
[METRICS-1233] make telemetry metrics whitelist a dynamic config
CNKAF-548: Skip checking quotas when recording replication traffic
Set TopicExclusion string in SBK-generated CruiseControl config.
KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422) (#1477)
KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422)
KSTORAGE-331, KSTORAGE-392: add and improve tier component listener metrics and heartbeats
CPKAFKA-4753: Add metrics tracking total txn time, txn timeouts, and illegal state errors
CPKAFKA-4649: Ensure LDAP group manager is recreated on writer failover
SBK MINOR: Drain permits in startUp
CNKAF-319: Add Start rebalance protocol
modify kafka_path to only set project for confluentplatform
CNKAF-520, CNKAF-525: Wait for KafkaSampleStore check to complete at startup
KSTORAGE-411: Pass a hint down from controller to the leader in case of an unclean leader election
KSTORAGE-453: Persist leader epoch in tier checkpoint file
re-schedule pr scans to weekly
[CNKAF-492] SBK: CruiseControl Property defaults
Publish preview artifacts with preview release settings
CNKAF-50: Add static broker-level replication throttles
SBK: Implement KafkaDataBalancer startup/shutdown methods
CPKAFKA-4772: Fix PhysicalClusterMetadataTest’s tests
METRICS-1292 Dynamic Update of API key and secret in Kafka.
[CNKAF-216] Derive SBK capacity values from config parameters directly.
CONFLUENT: Make Rest Client Http Authenticator configurable
CNKAF 362: Topic exclusion logic for SBK in KafkaDataBalancer
CNKAF-305: Dynamic configs for sbk - enable/disable, throttle
CONFLUENT: Update cc-trogdor-service and cc-soak-clients to 0.803.0-6.0.0-ce-SNAPSHOT
KSTORAGE-545: Handle failed deletions from GcsTierObjectStore
CONFLUENT :HttpBearerCredentialProvider must implement configurable
ST-3292: migrate to semaphore2
CNKAF-19: ADB: Soak test self-healing
KSTORAGE-579: FIX fencing advancement issues
KSTORAGE-574: producer and epoch state restore is not resilient to failures
KSTORAGE-399 : Crash broker if FileTierPartitionState encounters corrupted flushed state file during initialization.
SEC-162: RestClient refactor regression
[CNKAF-479] Support executing ce-sbk objects with the rest of Kafka
[CNKAF-491] Remove clusterConfigs functionality from the TopicConfigProvider.
Revert “KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319) (#1401)”
KSTORAGE-482 Enhancing TierMetadataValidator tool to verify objects in S3 TierObjectStore
KSTORAGE-559: add maximum eligible segments deleted on every cleanup run
CNKAF-486: Add filter to exclude internal topics
Put an initial implementation of the DataBalancer interface in ce-sbk.
KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319)
KSTORAGE-568: bound hotset retention by LSO
KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions)
CPKAFKA-4670 Fix for system tests running 0.8.2.2 version producers
CNKAF-324: Allow configuring a delay before broker registration with ZK.
CNKAF 447: Update SBK mode config to a flag
[CNKAF-477] Restore ConfluentMetricsReporterSampler
KSTORAGE-572: FIX: Access fenced partitions metric under lock in TierTopicConsumer
CPKAFKA-4676: TierRoundtripTest.test_tier_roundtrip timeout increase
KSTORAGE-548: Check for missing init leader messages
KCORE-234: Upgrade netty to 4.1.48.Final and netty-tcnative to 2.0.29.Final
MCM-965: Reduced physical cluster metadata reload period to 2 mins
CC-7979: Add new topic tracking operations of KIP-558 to default RBAC roles
KAFKA-9685: Solve Set concatenation perf issue in AclAuthorizer (cherry-pick)
Bump up rollingupgrade deb package from 0.5.0 to 0.9.0
[CPKAFKA-4658] Restore removed SchemaRegistryUrl Config.
KCORE-225: Schedule periodic license validation task at fixed rate
DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client (#1374)
DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client
KSTORAGE-465: GCS tier deletion manager system test support
v5.5.x KSTORAGE-524: Fix potential NPE in DelayedFetch
KCORE-199: Set default values for schema configs on topics to avoid NPE
CNKAF 289: Increase partition count in soak cluster
[CNKAF-283] Put heavily trimmed CruiseControl into ce-kafka/ce-sbk
KSTORAGE-550 Narrow scope of KafkaException to avoid handling of any unwanted bug.
CONFLUENT MINOR: Revert #799, remove batch verifier from eos topic consumers
KCORE-111: Default replica placement constraints for external topics
CONFLUENT: fix exhaustive match warnings in TierArchiver
KSTORAGE-354: avoid roundtrip and connection in GCS fetch path
KCORE-175: Extend TopicPlacement to include assignment validation logic
SEC-159: RestRequest pluggable HTTP auth
KSTORAGE-543: Fix AutoAbortingS3InputStream: account for # read bytes being -1
KSTORAGE-214, KSTORAGE-407: Bump FileTierPartitionState version
Tier Storage system test for Consumer.offsetForTimes API
Refactoring FIPS implementation
KSTORAGE-389: Stronger detection of duplicate message processing for tiered storage
CPKAFKA-2336: Log layer metrics for tiered storage
CNKAF-305: Remove topic config from DynamicBalancer config
METRICS-1138 Removing shading of transitive dependencies due to telemetry-client jar
KCORE-190: Add max LSO lag metric
KSTORAGE-468 : Metadata states validation insync with log start offset.
CNKAF-304: Default value for rack aware flag
KCORE-104: Re-instate PartitionLockTest.testAppendReplicaFetchWithUpdateIsr
KSTORAGE-524: Fix potential NPE in DelayedFetch
SEC-804: Wait for config update before verification in audit config change test
CPKAFKA-1953: resume materialization from TierTopicManager commit positions
Log TierPartitionStatus to differentiate online vs catchup consumer
KSTORAGE-450: default confluent.tier.fetcher.num.threads = 4
KSTORAGE-457 ListOffsetRequest v0 does not account for tiered section…
KSTORAGE-330, KSTORAGE-328, KSTORAGE-325 TierStorageMaterialization recovery and validation tool.
KSTORAGE-452: Improve README for system tests w/ Tiered Storage
KSTORAGE-400: Handle duplicate PartitionDeleteInitiate messages
KSTORAGE-420: missing toString on MergedLog
KSTORAGE-460: Disable preferred tier reads for replica fetch
CPKAFKA-4369. Cherry pick of #1228 into 5.5.x
KSTORAGE-214, KSTORAGE-407: Fence user partition failures when materializing Tier Topic events
KSTORAGE-510: Ensure that PendingFetch::finish() stops the PendingFetch
Reduced test strength to ignore consumer data coverage caused by catc…
CPKAFKA-4126: Add prefix to system test config to match MDS changes
SEC-147: Refactor RestClient
KSTORAGE-519: Fix: Ensure unknown exceptions from DeletionTask are isolated to user partition
KSTORAGE-511: Add size asserts for TierPartitionStateEntry
KCORE-183: Avoid new allocations in MultiTenantRequestContext.filteredMetadataResponse call
KSTORAGE-344: TierMaterializationValidation as service for System Tests
CONFLUENT: Fix bug in result handling from readFromLocalLog
CNKAF-323: Add metric with per-tenant partition count
KCORE-151: Allow some reassignment of offline brokers
KSTORAGE-508: System test s3 prefix random generation incorrectly app…
KSTORAGE-505: Tier deletion manager system test failing in vagrant runs
KSTORAGE-424: Allow use of SSL for internal tier topic clients
KSTORAGE-384 Adding system test for tiered state file corruption
Merge branch ‘ccs-master’
KSTORAGE-502: Improve documentation for tiered storage configurations
CNKAF-305: Dynamic configs for SBK - kafka-side changes
KCORE-93: Removing ZK utils from ce-kafka
KCORE-110: Introducing new flag to display partitions violating topic replica constraints
CNKAF-304: Update network config name
CNKAF-304: Add static configs for self-balancing kafka
KCORE-104: Re-introduce PartitionLockTest
CONFLUENT: Update protobuf to 3.11.4 (#1237)
Update protobuf to 3.11.4
CPKAFKA-2700; Disable license validator in MultiTenantAuthorizer
Define a DataBalancer object in the Kafka Controller.
SEC-744 Make Audit Log Configs visible to AdminClient
METRICS-582: Adding tracking for kafka.broker.rack in telemetry
SEC-752 AuditLog FetchFollower
KSTORAGE-163: Initial implementation of TierFetcher memory bounding.
CPKAFKA-2705: Added retention to soak topics
KSTORAGE-479: Revert “KSTORAGE-464: Fix: ensure a queued TierTopic ev…
Add manual testing sections to ce-metrics README
SEC-746 Ignore unknown JSON properties for forwards compatibility
KSTORAGE-464: Fix: ensure a queued TierTopic event future throws when replaced
KSTORAGE-337/KSTORAGE-348: tier deletion manager system test
CONFLUENT: Add check in consume test util to verify records were consumed
METRICS-541 increase metric reporter default publishing time from 15 se…
CPKAFKA-3878: Re-enable scala inlining
CPKAFKA-4369. Copy Kafka configs when creating LogConfig- Part 1 of 2
CONFLUENT: Log better message on soak clients task creation 409 error response
TierArchiver: sleep outside of synchronized block in main loop

Security¶

Validate KSQL/RBAC cannot initialize on non-SASL protocols
Check authentication.roles in KSQL bearer token requests
CLIENTS-1658: Remove kafka core dependency from kafka-rest-security-plugin
SEC-1034: log4j migration to confluent repackaged version
remove test scope from connect-runtime dependency
DG-706: fix bug in schema registry client tests, as well as some refactor
DG-708: Skip Schema Validation query range check for first lookup
Include Kafka dependencies to KSQL packaging
Bump Vert.x version in confluent-ksql-security-plugin to 3.9.1
fix: Make rest-utils compile scope as ksqlDB no longer pulls this in
SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
Remove AdminClientWrapper.
Switch from application/vnd.api+json to application/json.
Migrate off JSON:API.
SEC-951: Upgraded bc-fips jars to 1.0.2
Add authorizations for mode endpoints
Add test that reproduces rest-proxy authorization bug.
Fix authentication for non-producer POST requests in kafka-rest.
Fix issue SSL class could not be found
Remove obsolete kafka-rest dependencies.
Fix bugs in Vertx auth plugin loginservice initialisation and unimplemented response method
Fix Content-Type for produce request.
Fix test due to ksql refactoring.
Remove kafka-rest V1 APIs and all their dependencies.
DG-403 Add missing method to retrieve schema refs
Vertx auth adaptors
SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
Add confluent-security-plugins-common module back
Remove support-metrics-common dependency
KCORE-82: Removing ZK licensing code and introducing LicenseManager
Use kafka.metrics.KafkaYammerMetrics instead of com.yammer.metrics.Metrics
Fixing compiling error due to KIP-515

MQTT Proxy¶

CC-11294: Bumped up Netty version to be consistent with Kafka and rest of CP.
METRICS-1989: confluent-kafka-mqtt depends on confluent-telemetry
CC-10900: Add confluent-telemetry to production jars.
CC-9739: Pass producer-striped configs and bootstrap.servers to AdminClient when getting Kafka cluster id.
CC-9786: Depend on confluent-metrics in tests only.
fix for METRICS-1733 change telemetry config constant names
CC-8363: Configure telemetry reporter.

Metadata Service¶

SEC-1421 - Set default replicator factor when creating audit log topic
SEC-1369 - Update MDS to non-broken version of Maven Assembly plugin
SEC-1034: log4j migration to confluent repackaged version
SEC-866 - move audit log “other” to “management”
[SEC-1345] Read HTTP Reason phrase when connection closed
Move kafka-http-server project into clients project
SEC-1323 - Make MDS Api doc not have everything be Bold
Fixed code which hard code security protocl to SASL_PLAINTEXT, should…
SEC-1268 - Backout confluent.security.event.logger.destination.log.* properties
SEC-1259 - Clarify audit log config API spec
SEC-777 - Cluster Registry & MDS should be lenient with it’s JSON Parsing
SEC-777 Cluster Registry & MDS should be lenient with it’s JSON Parsing
SEC-1222 Role Resources For Principal API throw a null pointer exception if ClusterName is passed in ResourceRequest
SEC-1089 CR C3 - Return Nice Names for all other Private UI endpoints
Fixed defect that RbacApiAppConfig missed all audit config with prefix confluent.security.
CLIENTS-1599: Convert KafkaHttpServlet to rest-utils Application.
SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
SEC-1212 - Do not create a new admin client if admin config not changed
SEC-1097: Disallow reassignment of name <=> scope.
SEC-1088 CR C3 - Return Nice Names for C3 visibility endpoint
MMA-7805: Fix for correctly packaging cluster-information-migration-script
MMA-7805 Script to import and export cluster information data to and from Registry
SEC-1191 - Assign default topics to produce, consume and other category
SEC-1021: updated validation + make error messages useful
SEC-1130 - Inject ConfluentAdmin from ce-kafka
Migrate MDS to KafkaHttpServlet
Added more testing cases for verifying audit log backend API.
SEC-1127 - Fixed a defect that AuditLogConfigSpec didn’t get updated in DynamicC…
SEC-988 Cluster Registry Fix to make CLI backwards compatible
SEC-1122 - Fix 6.0.x build break
SEC-986: Lockdown cluster name
SEC-940 - Make clusters optional in MdsScope in swagger spec
SEC-1037: Return sorted ‘full’ clusters followed by sorted ‘redacted’ clusters for cluster registry
SEC-1003 Cluster Registry : Migrate from “name” to “clusterName”
SEC-1020 - Cluster Info Protocol Validation
SEC-982: Nice name gatekeeping
CIAM-12 - Try loading token.key.path as a public key for cloud
SEC-731: Single shot overwrite of multiple clusters
SEC-959: Full Cluster Info CRUD Gatekeeping for Cluster Registry
SEC-1019 Cluster Registry : Introduce security protocol field to ClusterInfo
SEC-985: MDS CR NiceNames : Update basic RBAC endpoints
SEC-1015: Fixed compile error due to interface changed by SEC-1015 in ce-kafka.
For backward compatibility we name the json scope object as ‘scope’
Add support for Cluster Scope object with clusters or paths to MdsSco…
CIAM-49 Spec file for MDS API v2alpha1 - fix whitespace
CIAM-49 Spec file for MDS API v2alpha1
SEC-1001: Require at least one host in a Cluster
SEC-926: Using dynamic config for cluster registry backend
SEC-941 Add Http api that returns an array of Mds Scope object.
SEC-942: MDS CR NiceNames : Introduce MdsScope Object
SEC-965: Sorted-ness of ClusterRegistry
SEC-943 - MDS CR : Investigate creating ResourceType for CR CRUD
Fix OpenAPI specification format
SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
SEC-949 - MDS CR : Update CR given Audit logs fix
SEC-728 - Cluster Registry - Make the backend be a broker Dynamic Config
SEC-891 - MDS : Short term fix flakey Token tests by isolating them
[SEC-844] restore /security/1.0 prefix
SEC-871 - MDS Test : Test / document OOTB Ldap and RBAC case sensitivity
SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
SEC-800 - MDS : Remove MdsLoginService.roles
SEC-751 MDS - Clean up the openapi yaml spec for CLI
SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
SEC-727 - Cluster Registry - Lock down the basic list of clusters
SEC-726 - Service Reg - Basic list of clusters

Replicator¶

REP-1176 Log error when connector submission fails for Replicator executable
REP-1131: Add support for Replicator with regex router
REP-1055 Enable converter configs for Replicator connector
REP-787: Synchronize fetching end offsets from source cluster to calculate lag
REP-824: Changed JMX initialization to use method from KIP-606
REP-721: Delete old license validation code from codebase
REP-795 moved translated records log to DEBUG and only print if records were translated
REP-761: ensure offset.topic.commit is false if topic.preserve.partitions is false
REP-755: retrieve port from –connect-url if specified
REP-743: Fix issue where we don’t properly log the topic that got an UnknownTopicPartitionException
REP-754 fix for offset commit failure resulting in loss of offsets
REP-689: Reduce log level of offset commit exception from ERROR to WARN
REP-665: Deleting all ZK dependencies from code base
REP-673: Port ZK dependencies for replicator-verifier into the verifier module
REP-664: Remove ZK dependencies from ReplicatorSourceConnector and ReplicatorSourceTask