Confluent Platform Component Changelogs¶

This topic provides changelogs for the individual Confluent Platform components.

Version 6.0.0¶

Community Features¶

Common¶

PR-294 - SEC-1307: Backport “log4j replacement with confluent repackaged version”
PR-278 - SEC-1334: update confluent-log4j version
PR-277 - KC-613: Update slf4j version to match AK’s.
PR-274 - ST-3988: Update jackson version to 2.10.5
PR-275 - SEC-1305: update jackson-bom version
PR-272 - METRICS-1918: bump gson from 2.8.5 -> 2.8.6
PR-270 - SEC-1034: log4j migration to confluent repackaged version
PR-269 - Upgrade Jackson to 2.10.2 and Avro to 1.9.2
PR-265 - downgrade maven-deploy-plugin to 2.8.2
PR-258 - update various maven plugins
PR-257 - ST-1323: Enable source jars if Apache or Confluent license files are found
PR-262 - fix: Pin the Scala version
PR-261 - Update ZooKeeper to 3.5.8
PR-260 - Update Scala version to 2.13
PR-259 - add property for jackson-jsonschema version
PR-248 - Upgrade Avro to 1.9.2

Kafka¶

PR-9197 - Revert KAFKA-9309: Add the ability to translate Message to JSON
PR-9172 - KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect
PR-392 - KAFKA-2715: Removed previous system_test folder
PR-384 - KAFKA-2714: Added integration tests for exceptional cases in fetching
PR-9089 - KAFKA-10224: Update jersey license from CDDL to EPLv2
PR-9066 - KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets
PR-9086 - FIX: Remove staticmethod tag to be able to use logger of instance
PR-9075 - KAFKA-10306: GlobalThread should fail on InvalidOffsetException
PR-9065 - KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates
PR-9051 - KAFKA-10268: dynamic config like “–delete-config log.retention.ms” doesn’t work
PR-9026 - KAFKA-10274; Consistent timeouts in transactions_test
PR-8979 - KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers
PR-8239 - KAFKA-9666: Don’t increase transactional epoch when trying to fence if the log append fails
PR-9009 - KAFKA-6453: Document how timestamps are computed for aggregations and joins
PR-8994 - KAFKA-10247: Correctly reset state when task is corrupted
PR-9010 - KAFKA-10262: Ensure that creating task directory is thread safe
PR-9005 - KAFKA-10263: Do not assign standby for revoking stateless tasks
PR-8996 - KAFKA-10249: don’t try to read un-checkpointed offsets of in-memory stores
PR-8934 - KAFKA-10134: Use long poll if we do not have fetchable partitions
PR-8989 - KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups
PR-8962 - KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory
PR-8983 - KAFKA-8398: Prevent NPE in forceUnmap
PR-8858 - KAFKA-10153: Error Reporting in Connect Documentation
PR-8917 - KAFKA-10180: Fix security_config caching.
PR-8905 - KAFKA-10173: Fix suppress changelog binary schema compatibility
PR-8926 - KAFKA-10166: always write checkpoint before closing an (initialized) task
PR-8924 - KAFKA-10198: guard against recycling dirty state
PR-8900 - KAFKA-10169: swallow non-fatal KafkaException and don’t abort transaction during clean close
PR-8887 - KAFKA-10135: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager…
PR-8896 - KAFKA-10185: Restoration info logging
PR-8890 - KAFKA-9891: add integration tests for EOS and StandbyTask
PR-8822 - KAFKA-10113; Specify fetch offsets correctly in LogTruncationException
PR-8876 - KAFKA-10167: use the admin client to read end-offset
PR-8841 - KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition
PR-8853 - KAFKA-10147 MockAdminClient#describeConfigs(Collection<ConfigResource…
PR-8882 - KAFKA-10165: Remove Percentiles from e2e metrics
PR-8856 - KAFKA-10150: task state transitions/management and committing cleanup
PR-8880 - KAFKA-10169: Error message when transit to Aborting / AbortableError / FatalError
PR-8872 - Fix log message for transition from standby to active
PR-8873 - Avoid WARN log message when re-init from checkpoint skipped
PR-8849 - KAFKA-10144: clean up corrupted standby tasks before attempting a commit
PR-8764 - KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes
PR-8787 - KAFKA-10085: correctly compute lag for optimized source changelogs
PR-8818 - KAFKA-10086: Integration test for ensuring warmups are effective
PR-8069 - KAFKA-9374: Make connector interactions asynchronous
PR-8828 - KAFKA-9216: Enforce that Connect’s internal topics use compact cleanup policy
PR-8829 - KAFKA-10115: Incorporate errors.tolerance with the Errant Record Reporter
PR-8455 - KAFKA-9845: Warn users about using config providers with plugin.path property
PR-8502 - KAFKA-9066: Retain metrics for failed tasks
PR-8847 - KAFKA-7833: Add missing test
PR-8453 - KAFKA-9841: Revoke duplicate connectors and tasks when zombie workers return with an outdated assignment
PR-8663 - KAFKA-9985: Sink connector may exhaust broker when writing in DLQ
PR-8833 - KAFKA-9441: remove prepareClose() to simplify task management
PR-8825 - KAFKA-7833: Add StateStore name conflict check in InternalTopologyBuilder
PR-8630 - KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation
PR-8775 - KAFKA-10079: improve thread-level stickiness
PR-7384 - KAFKA-8938: Improve allocations during Struct validation in ConnectSchema
PR-8827 - KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used
PR-8805 - KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group
PR-8783 - KAFKA-10063 UnsupportedOperation when querying cleaner metrics after shutdown
PR-8803 - KAFKA-10102: update ProcessorTopology instead of rebuilding it
PR-8676 - KAFKA-10005: Decouple RestoreListener from RestoreCallback
PR-8684 - KAFKA-10012 Reducing memory overhead associated with strings in SelectorMetrics
PR-8270 - KAFKA-9216: Enforce internal config topic settings for Connect workers during startup
PR-8820 - KAFKA-10097: Internalize checkpoint data
PR-8804 - KAFKA-9851: Revoking Connect tasks due to connectivity issues should also clear the running assignment
PR-8486 - KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable
PR-8135 - KAFKA-9570: Define SSL configs in all worker config classes, not just distributed
PR-8814 - KAFKA-10111: Make SinkTaskContext.errantRecordReporter() a default method
PR-8811 - KAFKA-10110: Corrected potential NPE when null label value added to KafkaMetricsContext
PR-8776 - KAFKA-9441: Improve Kafka Streams task management
PR-8759 - KAFKA-10066: TestOutputTopic should pass record headers into deserializers
PR-8755 - KAFKA-10069: Correctly remove user-defined “predicate” and “negate” configs from transformation properties
PR-8644 - KAFKA-9313: Set use_all_dns_ips as the new default for client.dns.lookup (KIP-602)
PR-8784 - KAFKA-9788: Use distinct names for transaction and group load time sensors
PR-8792 - KAFKA-10089 The stale ssl engine factory is not closed after reconfigure
PR-8782 - KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends
PR-8785 - KAFKA-10084: Fix EosTestDriver end offset
PR-8786 - KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests
PR-8737 - KAFKA-9945: TopicCommand should support –if-exists and –if-not-exists when –bootstrap-server is used
PR-8695 - KAFKA-9320: Enable TLSv1.3 by default (KIP-573)
PR-8777 - KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment
PR-8706 - KAFKA-10030 allow fetching a key from a single partition
PR-8760 - Kafka-10064 Add documentation for KIP-571
PR-8668 - KAFKA-9987: optimize sticky assignment algorithm for same-subscription case
PR-7956 - KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing
PR-8769 - KAFKA-10074: Improve performance of matchingAcls
PR-8723 - KIP-569-KAFKA-9494: DescribeConfigsResponse - include additional metadata information
PR-8739 - KAFKA-10056; Ensure consumer metadata contains new topics on subscription change
PR-8705 - KAFKA-10029; Don’t update completedReceives when channels are closed to avoid ConcurrentModificationException
PR-8238 - KAFKA-9130: KIP-518 Allow listing consumer groups per state
PR-8248 - KAFKA-9501: convert between active and standby without closing stores
PR-8699 - KAFKA-9673: Filter and Conditional SMTs
PR-8720 - KAFKA-9971: Error Reporting in Sink Connectors (KIP-610)
PR-8736 - KAFKA-9802; Increase transaction timeout in system tests to reduce flakiness
PR-8731 - KAFKA-10050: kafka_log4j_appender.py fixed for JDK11
PR-8589 - KAFKA-9146: KIP-571 Add option to force delete active members in StreamsResetter
PR-8691 - KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter
PR-8709 - KAFKA-9952; Remove immediate fetch completion logic on high watermark updates
PR-8692 - KAFKA-10018: Change command line tools from /bin/sh to /bin/bash
PR-8697 - KAFKA-9983: KIP-613, add INFO level e2e latency metrics
PR-8376 - KAFKA-9724 Newer clients not always sending fetch request to older brokers
PR-8504 - KAFKA-9298: reuse mapped stream error in joins
PR-8716 - KAFKA-6145: KIP-441: Fix assignor config passthough
PR-6824 - KAFKA-7523: Add ConnectedStoreProvider to Processor API
PR-8221 - KAFKA-9561: update task input partitions after rebalance
PR-7886 - KAFKA-9353: Add groupInstanceId to DescribeGroup for better visibility
PR-8722 - KAFKA-5295: Allow source connectors to specify topic-specific settings for new topics (KIP-158)
PR-8395 - Added doc for KIP-535 and updated it for KIP-562
PR-8118 - KAFKA-9472: Remove deleted Connect tasks from status store
PR-2604 - KAFKA-4794: Add access to OffsetStorageReader from SourceConnector
PR-8698 - KAFKA-10022:console-producer supports the setting of client.id
PR-6284 - KAFKA-6755: Allow literal value for MaskField SMT
PR-8357 - KAFKA-9767: Add logging to basic auth rest extension
PR-8620 - KAFKA-9944: Added supporting customized HTTP response headers for Kafka Connect.
PR-8511 - KAFKA-9888: Copy connector configs before passing to REST extensions
PR-8654 - KAFKA-9931: Implement KIP-605 to expand support for Connect worker internal topic configurations
PR-8713 - KAFKA-6145: Add unit tests for assignments of only stateless tasks
PR-8696 - KAFKA-6145: KIP-441: Enforce Standby Task Stickiness
PR-8679 - KAFKA-10003: Mark KStream.through() as deprecated
PR-8658 - KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas
PR-8379 - KAFKA-9780: Deprecate commit records without record metadata
PR-8651 - kafkatest: Deploy VerifiableClient in constructor to avoid test timeouts
PR-8472 - KAFKA-9855 - return cached Structs for Schemas with no fields
PR-8608 - KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties
PR-8444 - KAFKA-8869: Remove task configs for deleted connectors from config snapshot
PR-7942 - KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect
PR-8661 - KAFKA-9603: Do not turn on bulk loading for segmented stores on stand-by tasks
PR-8671 - KAFKA-9859 / Add topics generated by KTable FK join to internal topic matching logic
PR-8681 - KAFKA-10010: Should make state store registration idempotent
PR-8682 - KAFKA-10011: Remove task id from lockedTaskDirectories during handleLostAll
PR-8673 - KAFKA-9992: EmbeddedKafkaCluster.deleteTopicAndWait not working with kafka_2.13
PR-8689 - KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173
PR-8675 - KAFKA-10004: ConfigCommand fails to find default broker configs without ZK
PR-8569 - KIP-551: Expose disk read and write metrics
PR-8678 - Update Gradle to 6.4.1
PR-8659 - KAFKA-9617 Replica Fetcher can mark partition as failed when max.message.bytes is changed
PR-8618 - KAFKA-9955: Prevent SinkTask::close from shadowing other exceptions
PR-8558 - KAFKA-8611 / KIP-221 documentation
PR-8670 - KAFKA-10001: Should trigger store specific callback if it is also a listener
PR-8090 - KAFKA-9537 - Cleanup error messages for abstract transformations
PR-8674 - KAFKA-9996: Upgrade zookeeper to 3.5.8
PR-8667 - KAFKA-9994: Handle task migrated inside corruption path
PR-8665 - KAFKA-9984 Should fail the subscription when pattern is empty
PR-8222 - KAFKA-9650: include human readable units in ms and bytes configs
PR-8588 - KAFKA-6145: KIP-441: Improve assignment balance
PR-313 - KAFKA-2641: Upgrade path for ZK authentication
PR-8550 - KAFKA-9850 Move KStream#repartition operator validation during Topology build process
PR-8648 - KAFKA-9966: Add internal assignment listener to stabilize EOS version 2 upgrade test
PR-8596 - KAFKA-9821: persist followup rebalance in assignment and consolidate rebalance triggering mechanisms
PR-8647 - KAFKA-9669; Loosen validation of inner offsets for older message formats
PR-8254 - KIP-557: Add Emit On Change Support
PR-8626 - KAFKA-9921: explicit handling of null values with retainDuplicates
PR-8643 - KAFKA-9956: Authorizer APIs may be invoked more than once for a given request
PR-8632 - KAFKA-9972: Only commit tasks with valid states
PR-8628 - KAFKA-9942: Fixes ConfigCommand client quotas w/ default users.
PR-8613 - KAFKA-6145: Set HighAvailabilityTaskAssignor as default in streams_upgrade_test.py
PR-8230 - KAFKA-9667: Connect JSON serde strip trailing zeros
PR-8483 - KAFKA-9865: Expose output topic names from TopologyTestDriver
PR-8360 - KAFKA-9768: Fix handling of rest.advertised.listener config
PR-7950 - KAFKA-9419: Integer Overflow Possible with CircularIterator
PR-8602 - KAFKA-9947; Ensure proper shutdown of components in TransactionsBounceTest
PR-8591 - KAFKA-6342: Move workaround for JSON parsing of non-escaped strings
PR-8609 - KAFKA-9946; StopReplicaRequest deletePartition changes may cause premature topic deletion handling in controller
PR-8496 - KAFKA-9748: Add Streams EOS version 2 integration test
PR-8607 - KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
PR-8527 - Remove dependencies on deprecated –zookeeper command flags in junit tests
PR-8551 - KAFKA-9918 SslEngineFactory is NOT closed when channel is closing
PR-8565 - KAFKA-9798: Send one round synchronously before starting the async producer
PR-8533 - KAFKA-9589: Fixed bug in V2 log validator tests
PR-8577 - use appropriate fn for readability. (maybe)
PR-8586 - KAFKA-9939; Fix overcounting delayed fetches in request rate metrics
PR-8595 - KAFKA-8410: Revert Part 1: processor context bounds (#8414)
PR-8597 - KAFKA-6145: KIP 441 remove balance factor
PR-8590 - KAFKA-6145: Remove check to reuse previous assignment
PR-8204 - KAFKA-9633: Ensure ConfigProviders are closed
PR-8554 - KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd
PR-8567 - KAFKA-9652: Fix throttle metric in RequestChannel and request log due to KIP-219
PR-8559 - KAFKA-9922:Update examples README
PR-8442 - KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses
PR-8578 - KAFKA-9875: Make integration tests more resilient
PR-8582 - KAFKA-9932: Don’t load configs from ZK when the log has already been loaded
PR-8574 - KAFKA-9925: decorate pseudo-topics with app id
PR-8580 - KAFKA-9832: fix attempt to commit non-running tasks
PR-8540 - KAFKA-9127: don’t create StreamThreads for global-only topology
PR-8568 - KAFKA-9176: Retry on getting local stores from KafkaStreams
PR-8548 - KAFKA-9823: Follow-up, check state for handling commit error response
PR-8541 - KAFKA-6145: KIP-441: Add TaskAssignor class config
PR-8564 - KAFKA-9921: disable caching on stores configured to retain duplicates
PR-8566 - Fix minor code issue
PR-8525 - KAFKA-9885; Evict last members of a group when the maximum allowed is reached
PR-8524 - KAFKA-9866: Avoid election for topics where preferred leader is not in ISR
PR-8509 - KAFKA-9839: Broker should accept control requests with newer broker epoch
PR-8224 - KAFKA-9704: Fix the issue z/OS won’t let us resize file when mmap.
PR-8326 - KAFKA-8639: Replace AddPartitionsToTxn with Automated Protocol
PR-8445 - KAFKA-9823: Remember the sent generation for the coordinator request
PR-8536 - KAFKA-9883: Add better error message when REST API forwards a request and leader is not known
PR-8537 - KAFKA-9907: Switch default build to Scala 2.13
PR-8464 - KAFKA-9852: Change the max duration that calls to the buffer pool can block from 2000ms to 10ms
PR-8417 - KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy
PR-8530 - KAFKA-9388: Refactor integration tests to always use different application ids
PR-8482 - KAFKA-9863: update the deprecated –zookeeper option in the documentation into –bootstrap-server
PR-8497 - KAFKA-6145: KIP-441 Build state constrained assignment from balanced one
PR-8520 - Add explicit grace period to tumbling window example
PR-8522 - KAFKA-9868: Reduce transaction log partitions for embed broker
PR-8523 - Adding github whitelist
PR-8475 - KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence
PR-8501 - KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test
PR-8448 - KAFKA-9796; Broker shutdown could be stuck forever under certain conditions
PR-8471 - KAFKA-9854 Re-authenticating causes mismatched parse of response
PR-8476 - KAFKA-9838; Add log concurrency test and fix minor race condition
PR-8378 - KAFKA-9779: Add Stream System Test 2.5
PR-6210 - KAFKA-7885: TopologyDescription violates equals-hashCode contract.
PR-8092 - KAFKA-9540: Move “Could not find the standby task while closing it” log to debug level
PR-8256 - KAFKA-9675: Fix bug that prevents RocksDB metrics to be updated
PR-8440 - KAFKA-9832: extend Kafka Streams EOS system test
PR-8492 - KAFKA-9739: Fix for 2.5 branch
PR-8403 - KAFKA-9797; Fix TestSecurityRollingUpgrade.test_enable_separate_interbroker_listener
PR-8477 - KAFKA-9864: avoid expensive QuotaViolationException usage
PR-8469 - [KAFKA-9826] Handle an unaligned first dirty offset during log cleaning.
PR-8470 - KAFKA-8611 / Refactor KStreamRepartitionIntegrationTest
PR-8257 - KAFKA-9539; Add leader epoch in StopReplicaRequest (KIP-570)
PR-8353 - KAFKA-9764: Remove stream simple benchmark suite
PR-8457 - KAFKA-9842; Add test case for OffsetsForLeaderEpoch grouping in Fetcher
PR-8077 - KAFKA-9583: use topic-partitions grouped by node to send OffsetsForLeaderEpoch requests
PR-8474 - KAFKA-9853: Improve performance of Log.fetchOffsetByTimestamp
PR-7015 - KAFKA-8436: use automated protocol for AddOffsetsToTxn
PR-8399 - KAFKA-3720: Change TimeoutException to BufferExhaustedException when no memory can be allocated for a record within max.block.ms
PR-8443 - KAFKA-9832: Extend Streams system tests for EOS version 2
PR-8436 - KAFKA-6145: KIP-441 avoid unnecessary movement of standbys
PR-7170 - KIP-221 / Add KStream#repartition operation
PR-7844 - KAFKA-9309: Add the ability to translate Message classes to and from JSON
PR-8441 - KAFKA-9748: Extend Streams integration tests for EOS version 2
PR-8212 - KAFKA-9642: Change “BigDecimal(double)” constructor to “BigDecimal.valueOf(double)”
PR-8317 - KAFKA-9691: Fix NPE by waiting for reassignment request
PR-8315 - KAFKA-9433: Use automated protocol for AlterConfigs request and response
PR-8425 - KAFKA-6145: KIP-441 Move tasks with caught-up destination clients right away
PR-8451 - KAFKA-9835; Protect FileRecords.slice from concurrent write
PR-8409 - KAFKA-6145: KIP-441 Pt. 6 Trigger probing rebalances until group is stable
PR-8338 - KAFKA-8890: KIP-519- Make SSL context/engine configuration extensible
PR-8405 - KAFKA-9801: Still trigger rebalance when static member joins in CompletingRebalance phase
PR-8414 - KAFKA-8410: Part 1: processor context bounds
PR-8423 - KAFKA-9818: improve error message to debug test
PR-8420 - KAFKA-9815; Ensure consumer always re-joins if JoinGroup fails
PR-8371 - KAFKA-9753: A few more metrics to add
PR-8402 - KAFKA-9793: Expand the try-catch for task commit in HandleAssignment
PR-8383 - KAFKA-9784: Add OffsetFetch to group concurrency test
PR-8422 - KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator
PR-8418 - KAFKA-9807; Protect LSO reads from concurrent high-watermark updates
PR-8408 - KAFKA-9810: Document Connect Root REST API on /
PR-8412 - KAFKA-9750; Fix race condition with log dir reassign completion
PR-8411 - KAFKA-9812: fix infinite loop in test code
PR-8400 - KAFKA-9739: Fixes null key changing child node
PR-8375 - KAFKA-9776: Downgrade TxnCommit API v3 when broker doesn’t support
PR-8407 - KAFKA-9809: Shrink transaction timeout for streams
PR-8359 - KAFKA-9778: Add validateConnector functionality to the EmbeddedConnectCluster
PR-8337 - KAFKA-6145: Pt. 5 Implement high availability assignment
PR-8331 - KAFKA-9748: extend EosIntegrationTest for EOS version 2
PR-7755 - KAFKA-9233: Fix IllegalStateException in Fetcher retrieval of beginni…
PR-8269 - KAFKA-9659: Add more log4j when updating static member mappings
PR-8356 - KAFKA-9441: Cleanup Streams metrics for removed task commit latency metrics
PR-8370 - KAFKA-9753: Add active tasks process ratio
PR-8389 - KAFKA-9777; Remove txn purgatory to fix race condition on txn completion
PR-8279 - KAFKA-9706: Handle null keys/values in Flatten transformation
PR-8367 - KAFKA-9719: Streams with EOS version 2 should fail fast for older brokers
PR-8388 - KAFKA-9759: Add doc change for KIP-562
PR-8350 - KAFKA-9760: Add KIP-447 protocol change to upgrade notes
PR-8382 - KAFKA-9685: PT2, avoid unnecessary set creation in ACL matching
PR-8164 - KAFKA-9600: EndTxn should enforce strict epoch checking if from client
PR-8368 - KAFKA-9770: Close underlying state store also when flush throws
PR-8369 - KAFKA-9771: Port patch for inter-worker Connect SSL from Jetty 9.4.25
PR-8358 - KAFKA-9756: Process more than one record of one task at a time
PR-8320 - KAFKA-8470: State change logs should not be in TRACE level
PR-8280 - KAFKA-9707: Fix InsertField.Key should apply to keys of tombstone records
PR-8332 - KAFKA-9688: kafka-topic.sh should show KIP-455 adding and removing replicas
PR-8346 - KAFKA-9373: Reduce shutdown time by avoiding unnecessary loading of indexes
PR-6741 - Improve performance of checkpointHighWatermarks, patch 1/2
PR-8138 - KAFKA-9573: Fix JVM options to run early versions of Kafka on the latest JVMs
PR-8343 - KAFKA-9758: Doc changes for KIP-523 and KIP-527
PR-8290 - KAFKA-9677: Fix consumer fetch with small consume bandwidth quotas
PR-8339 - KAFKA-9752; New member timeout can leave group rebalance stuck
PR-8285 - KAFKA-9700:Fix negative estimatedCompressionRatio issue
PR-8287 - KAFKA-9711 The authentication failure caused by SSLEngine#beginHandsh…
PR-8334 - KAFKA-6145: Add balanced assignment algorithm
PR-8336 - KAFKA-9749: TransactionMarkerRequestCompletionHandler should treat KAFKA_STORAGE
PR-8327 - KAFKA-9743: Catch commit offset exception to eventually close dirty tasks
PR-7939 - KAFKA-9407: Return Immutable List from SchemaSourceTask
PR-8330 - KAFKA-9742: Fix broken StandbyTaskEOSIntegrationTest
PR-8226 - KAFKA-9651: Fix ArithmeticException (÷ by 0) in Partitioner impls
PR-8318 - KAFKA-9451: Enable producer per thread for Streams EOS
PR-8205 - KAFKA-9634: Add note about thread safety of ConfigProvider
PR-8252 - KAFKA-6145: Pt 2.5 Compute overall task lag per client
PR-8325 - KAFKA-9741: Update ConsumerGroupMetadata before calling onPartitionsRevoked()
PR-8324 - KAFKA-9701 (fix): Only check protocol name when generation is valid
PR-8307 - KAFKA-9727: cleanup the state store for standby task dirty close and check null for changelogs
PR-8262 - KAFKA-6145: Add constrained balanced assignment algorithm
PR-8319 - KAFKA-9734: Fix IllegalState in Streams transit to standby
PR-8155 - KAFKA-9553: Improve measurement for loading groups and transactions
PR-8244 - KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API
PR-8223 - KAFKA-9654 ReplicaAlterLogDirsThread can’t be created again if the pr…
PR-8284 - KAFKA-9225: Bump rocksdb 5.18.3 to 5.18.4
PR-8218 - KAFKA-9441: Unify committing within TaskManager
PR-7972 - KAFKA-9435: DescribeLogDirs automated protocol
PR-7039 - KAFKA-8618: Replace Txn marker with automated protocol
PR-8299 - KAFKA-9568: enforce rebalance if client endpoint has changed
PR-8313 - KAFKA-5604: Remove the redundant TODO marker on the Streams side
PR-8253 - KAFKA-9656: Return COORDINATOR_NOT_AVAILABLE for older producer clients
PR-8161 - KAFKA-9047: Making AdminClient group operations respect retries and backoff
PR-7936 - KAFKA-9404: Use ArrayList instead of LinkedList in Sensor Class
PR-8260 - KAFKA-9625: Fixing IncrementalAlterConfigs with respect to Broker Configs
PR-8278 - KAFKA-8803: Remove timestamp check in completeTransitionTo
PR-8289 - KAFKA-9712: Catch and handle exception thrown by reflections scanner
PR-8236 - KAFKA-9670: Reduce allocations in Metadata Response preparation
PR-8083 - KIP-546 (1/2): Implements describeClientQuotas() and alterClientQuotas().
PR-8267 - KAFKA-6647: Do note delete the lock file while holding the lock
PR-8246 - KAFKA-6145: Pt 2. Include offset sums in subscription
PR-8293 - KAFKA-9715: Eliminate reference to interBrokerProtocolVersion in TransactionStateManager
PR-8294 - KAFKA-9718; Don’t log passwords for AlterConfigs in request logs
PR-7957 - KAFKA-8768: DeleteRecords request/response automated protocol
PR-8265 - KAFKA-9657: Throw upon offset fetch unsupported stable flag protocol
PR-8266 - KAFKA-9695; Handle null config values for createTopics, alterConfigs
PR-8272 - KAFKA-9701: Add more debug log on client to reproduce the issue
PR-8177 - KAFKA-9605: Do not attempt to abort batches when txn manager is in fatal error
PR-7876 - KAFKA-9344:Logged consumer config does not always match actual config…
PR-8232 - KAFKA-9658: Fix removing user quotas
PR-8255 - KAFKA-9686 MockConsumer#endOffsets should be idempotent
PR-8235 - KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode
PR-283 - KAFKA-2391: add timeout to KafkaConsumer blocking calls
PR-282 - KAFKA-2428
PR-8220 - KAFKA-9645: Fallback to unsubscribe during Task Migrated
PR-8228 - KAFKA-9663: Doc some null returns in KafkaStreams
PR-284 - KAFKA-2620: Introduce Scalariform
PR-8233 - KAFKA-9668: Iterating over KafkaStreams.getAllMetadata() results in ConcurrentModificationException
PR-279 - KAFKA-2479: Transient exception
PR-8227 - KAFKA-9662: Wait for consumer offset reset in throttle test to avoid losing early messages
PR-8121 - KAFKA-6145: Pt 1. Bump protocol version and encode task lag map
PR-8213 - KAFKA-9615: Clean up task/producer create and close
PR-8229 - KAFKA-9661: Propagate includeSynonyms option to AdminClient in ConfigCommand
PR-254 - KAFKA-2588 ReplicaManager partitionCount metric should actually be re…
PR-277 - Kafka-2587: Only notification handler will update the cache and all verifications will use waitUntilTrue.
PR-8209 - KAFKA-9632; Fix MockScheduler synchronization for safe use in Log/Partition tests
PR-8186 - KAFKA-9618: Directory deletion failure leading to error task RocksDB open
PR-275 - KAFKA-2527; System Test for Quotas in Ducktape
PR-8208 - KAFKA-8895: delete all topics before recreating
PR-8197 - KAFKA-9573: Fix VerifiableProducer and VerifiableConsumer to work with older Kafka versions
PR-1286 - KAFKA-3618: Handle ApiVersionsRequest before SASL authentication
PR-8087 - KAFKA-9525: add enforceRebalance method to Consumer API
PR-8190 - KAFKA-9623: Keep polling until the task manager is no longer rebalancing in progress
PR-8187 - KAFKA-9620: Do not throw in the middle of consumer user callbacks
PR-273 - KAFKA-2517; Performance Regression post SSL implementation (zero copy)
PR-8168 - KAFKA-9607: Do not clear partition queues during close
PR-8169 - KAFKA-9610: Do not throw illegal state when remaining partitions are not empty
PR-8173 - KAFKA-9614: Not initialize topology twice in StreamTask
PR-8165 - KAFKA-9601: Stop logging raw connector config values
PR-8166 - KAFKA-9602: Close the stream internal producer only in EOS
PR-8062 - KAFKA-9498; Topic validation during the topic creation triggers unnecessary TopicChange events
PR-8029 - KAFKA-8147: Add changelog topic configuration to KTable suppress
PR-8167 - KAFKA-9533: Revert ValueTransform forwards null
PR-8009 - KAFKA-9308: Reworded the ssl part of the security documentation
PR-8159 - KAFKA-9599 create unique sensor to record group rebalance
PR-8145 - KAFKA-9581: Remove rebalance exception withholding
PR-8142 - KAFKA-9577: SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version
PR-8105 - KAFKA-9441: Add internal StreamsProducer
PR-263 - KAFKA 2578 Client Metadata internal state should be synchronized
PR-8143 - KAFKA-9582: Do not abort transaction in unclean close
PR-8111 - KAFKA-9206: throw a KafkaException when encountering CORRUPT_MESSAGE
PR-8058 - KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted
PR-8116 - KAFKA-9562: part 1: ignore exceptions while flushing stores in close(dirty)
PR-7337 - KAFKA-8910: Incorrect javadoc at KafkaProducer.InterceptorCallback#onCompletion
PR-8108 - KAFKA-9533: ValueTransform forwards null values
PR-7913 - KAFKA-9343: Add ps command for Kafka and zookeeper process on z/OS.
PR-8119 - KAFKA-9558: Fixing retry logic for getListOffsetsCalls
PR-8089 - KAFKA-6266: Repeated occurrence of WARN Resetting first dirty offset
PR-8125 - KAFKA-9515: Upgrade ZooKeeper to 3.5.7
PR-8106 - KAFKA-9319: Fix generation of CA certificate for system tests.
PR-8109 - KAFKA-9545: Fix subscription bugs from Stream refactoring
PR-8088 - KAFKA-9535: Update metadata upon retrying partitions for ListOffset
PR-7389 - KAFKA-8805: Bump producer epoch on recoverable errors
PR-8060 - KAFKA-9274: Gracefully handle timeout exception
PR-7781 - KAFKA-8904: Improve producer’s topic metadata fetching.
PR-8085 - KAFKA-9556: Fix two issues with KIP-558 and expand testing coverage
PR-8112 - KAFKA-9557: correct thread process-rate sensor to measure throughput
PR-8023 - KAFKA-8507 kip 499 Unify connection name flag for command line tool
PR-7674 - KAFKA-9106 make metrics exposed via jmx configurable
PR-7731 - KAFKA-9204: allow ReplaceField SMT to handle tombstone records
PR-7733 - KAFKA-9192: fix NPE when for converting optional json schema in structs
PR-8000 - KAFKA-9417: New Integration Test for KIP-447
PR-8031 - KAFKA-9447: Add new customized EOS model example
PR-8091 - KAFKA-6607: Commit correct offsets for transactional input data
PR-8053 - KAFKA-9499; Improve deletion process by batching more aggressively
PR-7955 - KAFKA-9423: Refine layout of configuration options on website and make individual settings directly linkable
PR-8015 - KAFKA-9500: Fix topology bug in foreign key joins
PR-8065 - KAKFA-9503: Fix TopologyTestDriver output order
PR-7996 - KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS
PR-8054 - KAFKA-9390: Make serde pseudo-topics unique
PR-8024 - KAFKA-9483; Add Scala KStream#toTable to the Streams DSL
PR-8040 - KAFKA-6607: Commit correct offsets for transactional input data
PR-8059 - KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent fields
PR-8061 - KAFKA-9517: Fix default serdes with FK join
PR-8033 - KAFKA-9487: Follow-up PR of Kafka-9445
PR-8081 - KAFKA-9523: Migrate BranchedMultiLevelRepartitionConnectedTopologyTest into a unit test
PR-8018 - KAFKA-9480: Fix bug that prevented to measure task-level process-rate
PR-8039 - KAFKA-9505: Only loop over topics-to-validate in retries
PR-8027 - KAFKA-7658: Follow up to original PR
PR-8057 - KAFKA-9507 AdminClient should check for missing committed offsets
PR-8056 - KAFKA-9519: Deprecating ZK for ConfigCommand
PR-8048 - KAFKA-9509: Fixing flakiness of MirrorConnectorsIntegrationTest.testReplication
PR-252 - KAFKA-2586; Enable SSL for inter-broker communication when SSL is enabled in tests
PR-255 - KAFKA-2593 Key value stores can use specified serializers and deserializers
PR-7770 - KAFKA-9261; Client should handle unavailable leader metadata
PR-7997 - KAFKA-9113: Clean up task management and state management
PR-8037 - KAFKA-9491; Increment high watermark after full log truncation
PR-8007 - KAFKA-9477 Document RoundRobinAssignor as an option for partition.assignment.strategy
PR-7568 - KAFKA-9074: Correct Connect’s Values.parseString to properly parse a time and timestamp literal
PR-8030 - KAFKA-9492; Ignore record errors in ProduceResponse for older versions
PR-8028 - KAFKA-9490: Fix generics for Grouped
PR-8026 - Bump trunk to 2.6.0-SNAPSHOT
PR-7725 - KAFKA-9027, KAFKA-9028: Convert create/delete acls requests/response to use generated protocol

ksqlDB¶

PR-6126 - fix: create /var/lib/kafka-streams directory on RPM installation
PR-6165 - fix: allow expressions in flat map (6.0.x)
PR-5952 - METRICS-1988: add confluent-telemetry directory to ksql-run-class
PR-5937 - fix: avoid losing cause of processing errors
PR-5895 - fix: adds a handler to gracefully shutdown
PR-5885 - fix: close query on invalid use of HTTP/2 with /query endpoint (6.0.x)
PR-5871 - fix: remove schema compat check if schema exists
PR-5844 - fix: Reuse KsqlClient instance for inter node requests (#5742)
PR-5837 - fix: always use the changelog subject in table state stores
PR-5786 - fix: better error on DECIMAL type with PROTOBUF format
PR-5818 - fix: Allow backported ALPN to be used and warn if no OpenSSL
PR-5783 - fix: SEC-1034: log4j migration to confluent repackaged version
PR-5788 - fix: circumvent KAFKA-10179 by forcing changelog topics for tables
PR-5763 - fix: ksqlDB should not truncate decimals
PR-5761 - support UDTFs on streams without keys
PR-5756 - fix: do not overwrite schemas from a CREATE STREAM/TABLE
PR-5730 - fix: make sure old query stream doesn’t block on close
PR-5684 - fix: join queries with SELECT * and WHERE fail
PR-5703 - fix: support CASE statements returning NULL
PR-5656 - fix: allow empty structs in schema inference
PR-5645 - fix: ensure only deserializable cmds are written to command topic
PR-5644 - fix: support GROUP BY with no source columns used
PR-5638 - feat: implements ARRAY_JOIN as requested in (#5028) (#5474)
PR-5617 - fix: don’t fail if broker does not support AuthorizedOperations
PR-5621 - fix: /inserts-stream endpoint now supports nested types
PR-5563 - feat: new split_to_map udf
PR-5559 - feat: add CHR UDF
PR-5597 - fix: fail AVRO/Protobuf/JSON Schema statements if SR is missing
PR-5591 - fix: make stream and column names case-insensitive in /inserts-stream
PR-5592 - fix: disallow requests to /inserts-stream if insert values disabled
PR-5576 - fix: Stream with no key should not fail if key has data
PR-5568 - fail aggregations that don’t select any value columns
PR-5516 - feat: reload TLS certificate without restarting server
PR-5560 - fix: set retention.ms to -1 instead of Long.MAX_VALUE
PR-5548 - feat: new UDFs for set-like operations on Arrays
PR-5552 - fix: improve print topic format detection
PR-5528 - feat: create MetricsContext for ksql metrics reporters
PR-5534 - feat: new CONCAT_WS UDF
PR-5531 - fix: inserts now support zero-valued decimals
PR-5536 - feat: new UDFs for working with Maps
PR-5546 - feat: new string UDFs LPad, RPad
PR-5523 - feat: Add encode udf
PR-5542 - feat: support TIMESTAMP being a key column
PR-5535 - feat: new UUID UDF
PR-5532 - fix: Prevent memory leaks caused by pull query logging
PR-5533 - feat: keys can now be named using the KEY syntax
PR-5529 - fix: deals with issue #5521 by adding more descriptive error message
PR-5513 - fix: update Concat UDF to new framework and make variadic
PR-5504 - feat: Adds UDF regexp_replace
PR-5501 - feat: Adds udf regexp_split_to_array
PR-5495 - feat: turn on snappy compression for produced data
PR-5515 - fix: Retry on connection closed
PR-5505 - feat: new UDFs for array max/min/sort
PR-5507 - feat: Adds UDF regexp_extract_all
PR-4909 - feat: add extra log messages for pull queries
PR-5506 - fix: allow dynamic construction of an ARRAY of STRUCTS with duplicate values #5436
PR-5461 - fix: KSQL does not accept queries when running QueryLimit - 1 queries
PR-5497 - build: reference plugin URL as property (MINOR)
PR-5478 - fix: Remove stacktrace from error message
PR-5435 - fix(client): fail queryID future if executeQuery() request fails (MINOR)
PR-5469 - fix: /inserts-stream endpoint now accepts complex types
PR-5477 - fix: Improve pull query error logging
PR-5470 - ST-1323: Add license file so sources get attached
PR-5459 - fix: avoid unnecessary warnings from PRINT
PR-5448 - feat(client): support (non-streaming) insert into in Java client
PR-5455 - fix: allow setting auto.offset.reset=latest on /query-stream endpoint
PR-5425 - fix: exclude window bounds from persistent query value & schema match
PR-5431 - fix: fail on WINDOW clause without matching GROUP BY
PR-5340 - feat: Expose Vert.x metrics
PR-5444 - Fix markdown for function references
PR-5385 - feat: INSTR function #881
PR-5412 - feat: introduce RegexClassifier for classifying errors via cfg
PR-5392 - feat: Pull Queries: QPS check utilizes internal API flag to determine if forwarded
PR-5355 - How-to guides: first set
PR-5391 - fix(client): expose query ID from executeQuery() return type
PR-5386 - fix: Block writer thread if response output buffer is full
PR-5374 - feat: expose JMX metric that classifies an error state
PR-5363 - feat: drop WITH(KEY) syntax
PR-5371 - feat(client): support push query termination in Java client
PR-5382 - MINOR: Use kafka.scala.version variable in pom file
PR-5380 - MINOR: Update mbknor-jackson-jsonschema_2.12 artifact to mbknor-jackson-jsonschema_2.13
PR-5381 - fix: Override Scala version in our build
PR-5360 - fix(client): Java client error handling
PR-5270 - fix: remove leading zeros when casting decimal to string
PR-5273 - feat: implement earliest_by_offset() UDAF
PR-4975 - fix: fix NPE in latest_by_offset if first element being processed has null…
PR-4782 - feat: Implement latest_by_offset() UDAF
PR-5212 - feat: Adds the ability have internal endpoints listen on ksql.internal.listener
PR-5332 - fix: Allow value delimiter to be specified for datagen
PR-5093 - Any key name
PR-5200 - feat(client): Java client with push + pull query support
PR-5287 - fix: allow structs in schema provider return types
PR-5205 - fix: /query-stream endpoint should serialize Struct (MINOR)
PR-5239 - fix: Move Cors handler in front of /chc handlers
PR-5124 - Backport array len
PR-5213 - fix: use sr’s jackson-jsonschema version
PR-5158 - feat: Remove dependency to ksqldb-common
PR-5198 - fix: use schema in annotation as schema provider if present
PR-5193 - feat: Do not allow access to new streaming endpoints using HTTP1.x
PR-5189 - build: pin streams to 5.5.0 instead of 5.5.0-SNAPSHOT
PR-5162 - bug: stop multi-way joins failing on ambiguous column errors
PR-5150 - feat: klip-14 - rowtime as pseduocolumn
PR-5145 - fix: do not allow implicit casting in UDAF function lookup
PR-5122 - Event-driven microservice tutorial
PR-5104 - feat: fail startup if command contains incompatible version
PR-5105 - feat: transient queries added to show queries output
PR-5137 - BREAKING CHANGE: minor: allow more precise configuration of rocks bounds
PR-5142 - fix: support quotes in explain statements (MINOR)
PR-5126 - feat: Implement cluster health check endpoints
PR-5123 - refactor: Migrate old websocket query endpoint
PR-5139 - fix: include lower-case identifiers among those that need quotes (#3723)
PR-5081 - feat: add multi-join expression support
PR-5075 - fix: replace ‘null’ in explain plan with correct op type (MINOR)
PR-5023 - fix: Improve error message for where/having type errors
PR-5064 - feat: support multi-way joins
PR-5054 - fix: add content type header to ported endpoints response (MINOR)
PR-5069 - fix: make endpoints available while waiting for precondition
PR-5048 - fix: reject requests to new API server if server is not ready
PR-5047 - fix: make endpoints available while waiting for precondition
PR-5059 - fix: Make sure internal client is configured for TLS
PR-4985 - fix: Don’t set chunked transfer encoding for HTTP2 proxy
PR-4974 - fix: Catch server startup exceptions
PR-4945 - feat: Support wildcards for unauthed paths
PR-4944 - feat: CORS support for Vert.x based ksqlDB
PR-4851 - feat: Migrate /ksql and /ksql/terminate endpoints to Vert.x
PR-5019 - fix: improve handling of NULLs
PR-5052 - fix: json creator for QueryStatusCount should match getter (MINOR)
PR-5029 - feat: add KsqlQueryStatus to decouple from KafkaStreams.State
PR-4921 - Two new tutorials: materialized views + streaming ETL pipelines
PR-5013 - feat: support more advanced suite of LIKE expressions
PR-4952 - Support aliases in PARTITION BY and GROUP BY
PR-5002 - feat: speed up restarts by not building topologies for terminated queries
PR-4986 - Distinguish keys for tables by naming them PRIMARY KEY
PR-4875 - feat: scatter gather query status from all servers in cluster for ‘SHOW QUERIES [EXTENDED]’ statement
PR-4993 - fix: Allows unclosed quote characters to exist in comments
PR-5012 - fix: allow trailing slash in listeners config (MINOR)
PR-5005 - fix: query id for TERMINATE should be case insensitive
PR-4977 - fix: generated aliases for struct field access no longer require quoting
PR-5006 - Remove unused / duplicate methods (MINOR)
PR-4951 - feat: Adds rate limiting to pull queries
PR-4972 - fix: When pull query metrics enabled, throws NPE
PR-4956 - fix: output valid multiline queries when running SHOW QUERIES
PR-4966 - fix: Pass correct request configs when making pull query request
PR-4942 - fix: do not allow grouping sets
PR-4926 - fix: throw error when column does not exist during INSERT INTO
PR-4940 - fix: do not allow GROUP BY and PARTITION BY on boolean expressions
PR-4908 - fix: Don’t wait for streams thread to be in running state
PR-4878 - fix: handle leap days correctly during timestamp extraction
PR-4893 - fix: Infer TLS based on scheme of server string
PR-4889 - fix: If startup hangs (esp on preconditions), shutdown server correctly
PR-4872 - fix: ‘drop (stream|table) if exists’ fails if source does not exist
PR-4827 - fix: avoid duplicate column name errors from auto-generated aliases
PR-4822 - fix: Stop PARTITION BY and UDTFs that fail from terminating the query
PR-4850 - feat: expand IFNULL to support all types
PR-4859 - fix: Filter out hosts with no lag info by default
PR-4868 - test: add test coverage to ensure that historical commands from previous versions always work in the current version
PR-4837 - fix: Returns empty lag info for a dead host rather than last received lags
PR-4867 - fix: avoid long possible format lists on PRINT TOPIC for nulls
PR-4829 - feat: add COALESCE function
PR-4653 - fix: CommandRunner metric has correct metric displayed when thread dies
PR-4823 - fix: do not filter out rows where PARTITION BY resolves to null
PR-4814 - fix: use describeTopics() for Kafka healtcheck probe
PR-4771 - feat: add internal request to KsqlRequestConfig and SessionProperties
PR-4816 - fix: do not lose old key values on repartition
PR-4830 - fix: add deserializer for SqlType
PR-4809 - fix: Removes unnecessary error logging for heartbeat since this is ex…
PR-4754 - fix: better error code on shutdown
PR-4805 - fix: only create processing log stream if it doesn’t exist
PR-4743 - fix: make CLI unit-tests platform independent
PR-4785 - fix: docker image directories in light of renames (MINOR)
PR-4608 - feat: Add metrics for pull queries endpoint
PR-4778 - feat: Security plumbing
PR-4777 - fix: add transfer-encoding:chunked header if Jetty omits this
PR-4733 - feat: support for tunable retention, grace period for windowed tables
PR-4728 - feat: add REGEXP_EXTRACT UDF
PR-4717 - fix: create schemas at topic creation
PR-4709 - fix: stop logging about command topic creation on startup if exists (MINOR)
PR-4725 - feat: add ARRAY_LENGTH UDF
PR-4713 - feat: Integrate new API with existing security framework
PR-4714 - test: avoid NPE if user puts input topic name in output
PR-4685 - feat: basic auth for new API server
PR-4681 - fix: fix the error message for failing to wait for CommandRunner to catch up in DistributingExecutor
PR-4632 - fix: change default exception handling for timestamp extractors to log-and-continue
PR-4675 - fix: change configOverrides back to streamsProperties
PR-4597 - fix: added special handling for forwarded pull query request
PR-4650 - fix: Return immutable list of streams metadata
PR-4643 - fix: idempotent terminate that can handle hung streams
PR-4489 - fix: csas/ctas with timestamp column is used for output rowtime
PR-4625 - fix: support partial schemas
PR-4509 - feat: display errors in CLI in red text
PR-4621 - fix: patch KafkaStreamsInternalTopicsAccessor as KS internals changed
PR-4606 - fix: StoreQueryParameters name change
PR-4573 - feat: add the topic name to deserialization errors
PR-4575 - feat: log groupby errors to processing logger
PR-4590 - feat: Simple perf tool for new API
PR-4581 - feat: Provide upper limit on number of push queries
PR-4511 - [SECURITY] Use HTTPS to resolve dependencies in Maven Build
PR-4512 - feat: Plug insert streams into backend
PR-4551 - feat: enhance PRINT TOPIC’s format detection

REST Proxy¶

PR-738 - KC-613: Remove share/java/kafka-rest dependencySet.
PR-737 - KC-613: Split share/java/kafka-rest in kafka-rest-bin and kafka-rest-lib.
PR-734 - CLIENTS-1659: Remove V1 API Kafka core dependency and associated Zookeeper Client configs
PR-731 - CLIENTS-1657: Explicitly depend on Guava.
PR-729 - METRICS-1988: add confluent-telemetry directory to kafka-rest-run-class
PR-723 - Refactor requests to fix go sdk generation bug and add parameterized servers to openapi-spec
PR-726 - Add [Consumer|Producer]Performance tools back
PR-722 - Add reassignment API to openapi spec.
PR-721 - Add ACL APIs to openapi spec.
PR-719 - Add consumers (admin) APIs to openapi spec.
PR-718 - CNKAF-1017: Update v3 error response format
PR-715 - Add config endpoints to openapi spec.
PR-714 - OpenAPI Spec for a small subset of the V3 APIs.
PR-712 - Move MetricsReporter Config extraction to rest-utils
PR-711 - CLIENTS-1613: Add configuration to enable/disable V2 and V3 APIs.
PR-710 - Remove cluster-wide topic configs.
PR-709 - KC-613: Use slf4j version from common pom.xml.
PR-707 - Change resource label resource.kafka.cluster.id to resource.cluster.id
PR-703 - Enable setting cluster_id on KafkaRestMetricsContext
PR-706 - CNKAF-951: Add catchingCompose utility to handle exceptions.
PR-704 - CLIENTS-1599: Add constructor accepting path to KafkaRestApplication.
PR-697 - CLIENTS-1597: Add relationship reassignment and refactor reassignment resource
PR-701 - Rename alter config UPDATE to SET.
PR-700 - Add Batch Alter (UPDATE/DELETE) Broker/Cluster/Topic configs.
PR-696 - Add LIST/GET Consumer Group/Consumer/Consumer Assignment.
PR-693 - Add search/create/delete ACLs APIs.
PR-687 - CNKAF-739: Reassignment endpoint for a given topic name
PR-685 - CNKAF-739 - Reassignments endpoint for given topic name and partition id
PR-692 - Remove AdminClientWrapper.
PR-691 - Migrate off JSON:API cleanup.
PR-682 - CNKAF-739: Add Rest endpoint for list all partition reassignments
PR-690 - Switch from application/vnd.api+json to application/json.
PR-689 - KIP-606 MetricsContext
PR-686 - Migrate off JSON:API.
PR-683 - KIP-226: Add cluster-wide dynamic default configs.
PR-678 - ST-1323: Add license file so sources get attached
PR-679 - Migrate internal entities to Google’s Auto-Value.
PR-680 - Add static factory to Cluster.
PR-677 - KIP-226: Add support for configs source and synonyms.
PR-675 - Give more unique names to the testing containers.
PR-674 - Open visibility of configurePre/PostResourceHandling.
PR-672 - Fix REST Proxy injection-based authentication.
PR-671 - Make ParititonData public
PR-670 - Add JsonCreator to v3 entities
PR-667 - Copy admin client config from REST Proxy. Need for authentication tests.
PR-658 - Remove all methods from AdminClientWrapper.
PR-665 - CPKAFKA-4979: Remove non-determinism from searchReplicasByBroker.
PR-662 - Add find partition replicas by broker id endpoint.
PR-660 - CLIENTS-1505: Add broker config delete endpoint
PR-659 - CLIENTS-1504: Add update endpoint for broker config
PR-654 - Remove AdminClientWrapper.getBrokerIds() and add JSON:API errors.
PR-656 - Fix default Content-Type, when none or any (/) is provided.
PR-655 - Remove spurious topic/partitions existence test when producing.
PR-647 - Remove V1 APIs and all their dependencies.
PR-652 - CLIENTS-1501: Add broker configs endpoints
PR-617 - Fix compile error due to KIP-515 (ZK TLS)
PR-191 - SEC-1034: log4j migration to confluent repackaged version
PR-199 - Rest CORS option skip will now default to true
PR-195 - Add a config flag to skip OPTION when CORS is enabled
PR-193 - Removed OPTION omission for CORS
PR-192 - All original configs must be passed to MetricReporters
PR-190 - Use prefix to propagate metrics reporter configs
PR-186 - Extensible Application Metrics instance configuration
PR-180 - KIP-606 MetricsContext
PR-183 - ST-1323: Add license file so sources get attached
PR-182 - Add getter for Application’s Metrics object
PR-178 - Migrate rest-utils to use AK library instead of common-metrics
PR-177 - SEC-929-Customized HTTP response headers.
PR-172 - SEC-592: Customize Jetty request queue and threadpool size
PR-174 - Catch invalid format exception: #501

Schema Registry¶

PR-1624 - DG-857 Fix Avro converter for top-level bytes object
PR-1616 - DG-845 Have typeName override Object.class type; fixes #1615
PR-1611 - Correct License in Client library
PR-1598 - CP-644: Include CP jars required for Security and monitoring
PR-1601 - DG-820: Fix telemetry reporter handling of leadership changes
PR-1590 - DG-810: Explicitly add kafkastore.topic config to MetricsReporter instances
PR-1594 - DG-813 If use.latest.version is true, use on Protobuf dependencies
PR-1588 - DG-811 Allow JSON schema spec version to be specified
PR-1589 - DG-812 Derive file extension during download
PR-1587 - Upgrade to Kotlin 1.4.0
PR-1586 - DG-800 Cache ProtobufSchema using name in Converter
PR-1528 - SEC-1034: log4j migration to confluent repackaged version
PR-1580 - DG-789 Fix for CVE-2020-15824
PR-1579 - DG-788 Allow anyOf to be backward compat with oneOf/allOf
PR-1578 - DG-767 Calculate proper defaults for client ssl configs
PR-1576 - DG-767 Use passed in defaults for client ssl configs
PR-1577 - DG-774 restore ssl client configs
PR-1575 - DG-780 Add support for number with format for JSON Schema
PR-1569 - METRICS-1988: add confluent-telemetry directory to schema-registry-run-class
PR-1574 - DG-774 Avoid regenerating default names for Protobuf data
PR-1567 - DG-740 Convert connect defaults for JSON Schema
PR-1563 - DG-713: Disable automatic setup of telemetry reporting
PR-1561 - DG-723 Enable defaults for JSON Schema
PR-1556 - DG-715: Change telemetry cluster label names
PR-1550 - DG-695 Normalize JSON ref schemas before comparing
PR-1548 - ST-3988: update swagger-core to 1.6.2
PR-1547 - ST-3988: update swagger-core to 1.6.2
PR-1538 - DG-670 Treat anyOf same as oneOf in JSON Schema converter
PR-1537 - DG-652 Fix missing negation in conditional
PR-1535 - DG-652 Handle imports during protobuf compat checks
PR-1531 - DG-628 Allow changes to id during json schema compat checks
PR-1530 - DG-607 Add lenient processing for numbers in oneof
PR-1523 - DG-607 Ensure fields for union are optional
PR-1520 - Handle null error stream in RestService
PR-1522 - DG-606 Add option to not generate oneof for nullable fields
PR-1517 - Upgrade maven-plugin dependencies to 3.6 in 5.4.x
PR-1510 - Add metric name test
PR-1509 - align naming conventions
PR-1508 - Add telemetry reporter SR config option
PR-1507 - Use config originals for retrieving TR enablement
PR-1493 - Use MetricsContext to set JMX prefix
PR-1502 - Split telemetry cluster label
PR-1500 - Only add TelemetryReporter if explicitly enabled in config
PR-1483 - Telemetry reporter integration
PR-1497 - DG-554 Pass refs when forwarding register requests
PR-1496 - DG-535 Handle recursive Protobuf schemas
PR-1495 - DG-402 Missed some places where dotted names can occur
PR-1492 - Fix NPE during tombstone metric update
PR-1485 - Add node information to SchemaRegistryProtocol
PR-1487 - DG-527 Omit defaults for proto 3 schemas (fixed in wire)
PR-1482 - DG-499 Fix JSON Schema compat checks for partial models
PR-1484 - DG-524 Catch NPE during Avro compat check
PR-1480 - DG-27 Use leader/follower terminology
PR-1477 - Allow custom update handlers; fixes #382
PR-1476 - Add unit test for mvn validate action
PR-1468 - Implement new Schema Registry Metrics
PR-1474 - Add validate action for SR maven plugin
PR-1473 - Add ability to use latest version during serialization
PR-1471 - DG-436 Pass existing Avro schema instead of regenerating
PR-1470 - Allow unknown properties while deserializing SchemaRegistryProtocol.Assignment
PR-1469 - Fix message name on schema when version is included; fixes #1467
PR-1466 - DG-450 Perform proper matching of subschemas
PR-1461 - DG-268 Deprecate ZK-based coordination
PR-1455 - Fix Scala 2.13 deprecations
PR-1450 - DG-400 Support enums using allOf in JSONSchema converter
PR-1451 - Upgrade swagger deps
PR-1449 - DG-402 Add support for packages in ProtobufConverter
PR-1440 - Fix null schema type
PR-1433 - use property for jackson-jsonschema
PR-1436 - feat: adds subjectName in the exception for incompatible schema
PR-1437 - Fix cyclic schema references
PR-1431 - Cherry-pick for permanent delete
PR-1426 - DG-377 Allow generated classes to use refs during serialization
PR-1427 - DG-399 Use envelope in JsonSchemaUtils.toObject
PR-1417 - Allow id to be passed to console prod/cons; fixes #160
PR-1422 - Fix cycles in Avro schemaEquals
PR-1420 - SEC-929–Added one testing for verifying a feature added to rest-utils repo.
PR-1416 - DG-399 Use envelope in JsonSchemaUtils.toObject
PR-1397 - DG-91 Schema permanent delete.
PR-1389 - DG-91 Ability to permanently delete schemas
PR-1369 - DG-267 - Fix Protobuf converter enum handling
PR-1368 - Performance: Use singleton ObjectMapper
PR-1344 - Enhanced serialization exception
PR-1334 - DG-237: Migrate SchemaRegistry CP KafkaMetrics to AK KafkaMetrics
PR-1326 - Add API to retrieve supported schema types

Commercial Features¶

Confluent Server¶

CPKAFKA-6013: Skip tier state presence validation for deleted partitions
KSTORAGE-1082: tier replication causes incorrect HWM after restore (#2507)
KSTORAGE-1086: Better validations to check if tiering is disabled
[CNKAF-1144] Confluent.operator.managed property (#2484)
[Backport]METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
KSTORAGE-1040: Better checks to prevent tiering from being disabled dynamically
METRICS-2159: use a loading cache to avoid expensive pattern matching calls
CPKAFKA-5882. Add copyDependantLibs dependency to ce-sbk and ce-broker-plugins module
KC-845: Ensure LogConfig includes broker defaults to pass schema config validation
CPKAFKA-5873 Don’t load Netty classes when using default SSL engine
CONFLUENT: Remove shading from ce-broker-plugins and ce-sbk
CPKAFKA-5851: Align joda-time version with common
CNKAF-863: Only showed broker removal status for given broker.
KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
CPKAFKA-5849: Don’t log password configs in request log filter
CONFLUENT: Remove unused jacksonDatabind version definition
CPKAFKA-5795: Change guava dependency to 28.1-jre
CPKAFKA-5796: Remove ZK dependency from ce-licensing
CPKAFKA-5835: Fix regression in cluster link compatibility test
METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
[CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub. (#2408)
backport METRICS-2098 exclude zstd/lz4 from telemetry-clients to avoid conflicting versions
METRICS-2090: Remove duplicate labels on metrics
METRICS-2097: Don’t capture ‘total_size` and ‘tier_size` metrics
KC-637 cherry pick to 6.0.x
KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms. (#2353)
SEC-866-6.0.x again
CONFLUENT: Remove the soak clients status reporter cronjob
METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
[CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces (#2326)
CNKAF-1058; Ensure that Control Request Batching does not send LeaderAndIsrRequest with yet unknown broker(s)
KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA (#2309)
backport METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics (#2282)
CPKAFKA-5762: Fix topic deletion in system tests
CNKAF-1039: Update error for insufficient metrics to compute plan
KC-671: SBK system test with CL
METRICS-1988: add confluent-telemetry directory to kafka classpath
CONFLUENT: Bump cc-trogdor chart docker image to cc-trogdor:v0.207.0-5.3.0-ce-SNAPSHOT
[Cherry 2367] [SEC-1345] Read HTTP Reason phrase when connection closed
Revert “SEC-866 - Migrate audit log “other” category into “management” category. (#2330)”
backport METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar (#2332)
[CLIENTS-1633] Schedule MDS url refresher asynchronously
Increase root FS to 20GB
METRICS-1763, METRICS-1767 Configure telemetry metrics whitelist for Connect and MQTT Proxy Proactive Support (#2241)
SEC-866 - Migrate audit log “other” category into “management” category.
KC-391: Source topic deletion system test for CL
KAFKA-10282; Remove Log metrics immediately when deleting log (#2342)
CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only - cherry pick to 6.0
Rebalance Cluster System Test
CNKAF-1008: Add Broker/Remove Broker proposal logging
KC-633: Auth failures system test for Cluster Linking
[CHERRY-2308] Move kafka-http-server project into clients project
KC-638: System tests with tier storage enabled on destination
KC-754: Cluster link compatibility system tests
KC-631 Good path cluster linking test using SASL_SSL
backport METRICS-1992: Fix schema registry cluster id resource label (#2285)
CONFLUENT: Fix broken cluster link system tests in 6.0.x
CNKAF-1029: Ensure Executor state is reset after unexpected exceptions
KC-735: Improve cluster link addition.
CNKAF-1035: Roll cluster while a new broker is getting added
METRICS-1998: add rbac metrics to default metrics list (#2269)
backport METRICS-1993: add connect.cluster.id label (#2264)
Updated default topic replicas factor to 3 for audit log.
KC-741: Don’t include passwords in cluster link request logs
KC-736: Allow deletion of cluster links if mirrored topics are marked for deletion
KSTORAGE-896: fix tier deletion manager system test (#2256)
CNKAF-822: Cherry pick add broker system test to 6.0.x
KC-725: Backport of KC-725 to 6.0.x
backport METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist. (#2266)
backport METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup (#2253)
KSTORAGE-864 Fetch requests to return NotLeaderForPartition during ULE recovery
CNKAF-819: Test SBK broker removal with roll
METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
Backporting fix for METRICS-1764 and METRICS-1766
update kafkatest template for balancer replication factor (for < 3-node clusters)
KC-638: Cluster linking system test with tiered storage enabled in source cluster
CNKAF-825: Introduce a system test for replica placement constraints
CNKAF-1014: Rename usages of SBK in logs to DataBalancer
CNKAF-613: Introduce a broker removal system test
METRICS-1894:Telemetry Reporter is missing labels configured in prope…
METRICS-1817: add include regex default value to Provider interface
METRICS-1938: upgrade jose4j version (#2219)–backport
METRICS-1929: use advertised listeners/host-port when deriving local producer config
KSTORAGE-892 Re add couple of lines of ULE code that got deleted duri…
Backporting config/events Telemetry Reporter changes along with all shading issues.
Added one more toJson method in protoserde for webhook alert
(backport) MMA-8432: fix wrong package name
CNKAF-1004: Don’t describe removed broker configs
METRICS-1895: Update telemetry client version
KC-705: Retry metadata topic consumer.position() until authorizer startup times out
METRICS-1918: bump gson from 2.8.5 -> 2.8.6
(backport) prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
(Backport) include UnderMinIsrPartitionCount in the default broker metrics list
CPKAFKA-2852: CCloud metadata listener thread should not die on unexpected exception
SEC-1276 Allow empty resourceNames in CRNs
SEC-384: Use the correct log level for debug output
KC-634: Cluster link system tests with broker failures
CNKAF-964: Shade opencensus protobufs into SBK jar (#2155)
Add KafkaRest Metrics Provider
KSTORAGE-845 Check if tier topic exists already before calling AdminZkClient#createTopic
CNKAF-952: Update javadoc for removeBroker ConfluentAdmin API with InvalidBrokerRemovalException
CNKAF-851: log goals skipped during detection in one line
KC-630: Cluster link system test support with good path mirror test
CONFLUENT: Prevent log start offset overrun during retention
KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order (#2133)
Back port [METRICS-1815] Connect client metrics missing some resource level tag…
CNKAF-936: Remove dot-handling logic in metrics processor (#2121)
METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
Deprecate KafkaHttpServlet in favour of rest-utils Application. (#2135)
[METRICS-1844, METRICS-1839, METRICS-1756,METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
[CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle, handle ab…
METRICS-1801: align telemetry domain metric names
METRICS-1835 Shade confluent-serializers dependencies in confluent-metrics (#2111)
METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter (#2096)
METRICS-1810 use SBK topic replicas config if present (#2109)
fix: fixes exception in server shutdown due to metrics registration
METRICS-1809: add telemetry settings to default server.properties file
[CNKAF-927] Fix typo in server.properties. (#2090)
[CNKAF-924, CNKAF-926] Fix Databalancer Topic RF config, remove Teleme…
CONFLUENT: Added check for CL metrics for 6.0.x
CPKAFKA-4955: Fix failing testTopicDeletion
Removed two testing files accidentally added.
SEC-1130-Inject ConfluentAdmin from ce-kafka
backport METRICS-1820: use password object for telemetry api secret
CNKAF-893: Update SBK to work with secure Zookeeper
METRICS-1816, METRICS-1705 local telemetry exporter metrics whitelist configuration for SBK (#2031)
CONFLUENT: deleting extra topic config change in cluster linking test
CNKAF-909: Add direct dependency on opencensus for SBK (#2057)
KAFKA-10149: Allow auto preferred leader election when partitions are…
CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
SEC-1034: use confluent-log4j instead of log4j
Add RackAwareGoal to goal list
CNKAF-884: Remove unnecessary metrics (#2059)
Migrate MDS to run on KafkaHttpServer implementation
KC-505: integration tests for CL metrics for 6.0.x
METRICS-1717 use shared domain for jvm level metrics (#1949)
CNKAF-854: Disabling SBK should persist the broker removal cancellation status
[CNKAF-873] Don’t require network capacity configs. (#1996)
SEC-1183 - Treat empty authority in CRN pattern as a wildcard
CNKAF-882: Add proposal logging for add/remove
CNKAF-837: Add broker should be able to add removed brokers.
KC-393: Integration tests for mirroring compacted topics
Add inter broker listener endpoint information to HTTP server binder.
CNKAF-773: Do not throw an exception when the broker is not present in the cluster model
CNKAF-847: Complete removal callback when there are no proposals to execute
KSTORAGE-802: Allow specifying AWS credentials through configuration file
[CNKAF-848] Set preferred user SBK config in server.properties. (#1999)
[CNKAF-719] Add option to disable dynamic throttle setting when stati…
KSTORAGE-819: Force roll segments only for partitions eligible for tiering
KSTORAGE-799: reduce Log.lock contention in ArchiveTask segment selection logic
KSTORAGE-774: Fix active segment roll logic when roll.min.bytes setting is enabled
KSTORAGE-766: Ensure delete segment throttler does not block active segment rolls
KSTORAGE-769: Fix to remove the costly toSet conversions in disk throttler
KSTORAGE-758: Include ERROR status in TierPartitionStatus.isOpen()
KSTORAGE-595 NPE in TierDeletedPartitionsCoordinator after tier topic is reassigned / deleted from broker
KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util (#1906)
KC-611: Fix mirror topic create processing incorrect config.
KSTORAGE-547: state recovery injection tooling
CONFLUENT: Fix Confluent provider start future completion, add unit tests
[CNKAF-881] SBK: Don’t treat all new brokers as empty. (#2005)
SBK/CruiseControl: low utilization fix
CNAKF-833, CNKAF-842: Fix missing request rate metric and enable tele…
RackAwareGoal should allow for uneven replica distribution based on t…
CNKAF-788: Introduce proper API exceptions in the broker removal API
METRICS-1708: add telemetry version header for forward-compatibility (#1883)
CNKAF-791: Change data format of failedBrokers in ZK
CNKAF-807: Remove zkclient from gradle dependencies.
CNKAF-575: Remove IOtec:ZkClient dependency.
[CNKAF-776] Switch SBK Maven publish group to “io.confluent”
[METRICS-1609] dynamically add telemetry reporter at runtime
KC-389: Retry segment deletion in cluster link test to fix timing issue
KC-389: Cluster link integration tests with tier storage enabled
CPKAFKA-5207: Complete authorizer start future only when sufficient nodes are ready
CNKAF-843: Fix remove broker validation check
Persist the broker removal state and handle failover
CNKAF-773: Do not throw exception when there are no replicas to move on broker removal
CNKAF-428: Integration test for replica placement self-healing
Add retries to telemetry sampler integration test (#1922)
[CNKAF-803] Add a metric to track the number of pending add-broker requests
KC-72: Add topic listing for cluster links.
CPKAFKA-5207: Fail fast during broker start up if multiple brokers use same MDS URL
KC-536 bug fix to make offsets/acls configs dynamic (#1893)
CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
fix telemetry reporter reconfiguration
update ksql and streams provider
CNKAF-649: Handle broker restart during broker removal
KC-390: Cluster link authorization tests
backport [METRICS-1564] telemetry - derive interbroker client configs (#1816)
(Backport) make commit-id optional, update whitelist to include C3 metrics
SEC-1083 - AuditAdmin role’s scopeType = Cluster
KC-392: Integration tests for cluster link quotas on source and destination
KC-582: Add multi-tenant support for alterMirrors.
CNKAF-795: Ensure SBK clients have bootstrap.servers configured
Backport PR 1897 to 6.0.x
SEC-1083 - Add AuditAdmin role (#1861)
SEC-1039 - TopicSpec builder sets partitions and RF.
CNKAF-794: Properly wrap broker removal exceptions in listener and improve remove broker test
Removed disk throttling dependency on produce backpressure config
Jvm Collector for Telemetry Reporter
METRICS-1698 Setting default telemetry.enabled and exporter.enabled values to false.
KC-224: make leader epoch cache flush asynchronous when stamping new leader epoch
KC-537: apply topic-create-policy when auto-creating topics while fetching metadata
CNKAF-782: MetricsReporterSamplers are configured in an odd manner.
CONFLUENT: Fix cipher name for ChaCha20
CNKAF-428: Replica placement
METRICS-1152 document how to enable the telemetry-reporter in CPD
KC-534: Adds forceful cluster link deletion.
KCORE-54: Controller should accumulate metadata into fewer requests to mitigate handling delays
CNAKF-645: Add metrics sampler to consume from telemetry reporter
Generic telemetry reporter
KSTORAGE-649: Remove duplicate library dependencies accidentally included in merge.
[cnkaf-730] part 2: Completion callbacks for add and remove brokers
[CNKAF-730] Minimal addBrokers support for SBK
KC-207: Destination CL Metrics
METRICS-1362 Updating telemetry dependency version from 1.87 to 1.98 to set new content-type
KSTORAGE-649 and KSTORAGE-660: Support sts::AssumeRole and SSE-KMS
CNKAF-697 - High bandwidth throttles during cluster roll
Make ClusterLinkIntegrationTest run as integration test
CNKAF-760: Integrate api state persistence with remove broker
Upgrade cloudevents from v0.3 to v1.0
CC-8138: Add Connect feature to inject Confluent license properties into connector properties before startup
KC-571: Move tenant prefix to cluster link metadata instead of configs
CIAM-69 Use authorizer setting to configure CC RBAC roles
[CNKAF-759] Deal with non-Anomaly executions.
SEC-994: Update Cluster Registry doc
KC-387: Multi-tenant integration tests for security updates, offset migration
KSTORAGE-752: removed disk throttler dependency on logDir existence at server startup
KC-518: Use cluster link ID as primary key for cluster link metadata.
Refactor Event Logger to make it generic
KSTORAGE-699 Rationalize ArchiveTask/DeletionTask Exceptions
CNKAF-649: Introduce integration test for SBK’s AddBroker, RemoveBroker and DescribeBroker APIs
Add confluent metrics labels for connect metrics
CNKAF-649 - Compose the broker removal operation into multiple interruptible tasks
KSTORAGE-740: Add log closing time metrics
SEC-951: Upgraded bc-fips to 1.0.2
Add Confluent metrics tag Kafka server and client
KSTORAGE-730: Allow max.partition.fetch.bytes to be configurable for tier fetches
CNKAF-752: Store and fetch remove broker api state
KSTORAGE-746; Make timing in TierTopicListenersTest deterministic
CPKAFKA-4744; Fix log cleaner test failure caused by concurrent cleaning
KC-407: Micro-benchmark for ClusterLinkFetcherThread based on ReplicaFetcherThreadBenchmark
Introducing disk-usage based throttling for producers and replication followers
cherry-pick upstream KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter (#8691)
KCORE-142 new tests and bug fix for offset command config
[CNKAF-741] Handle duplicate metrics in the DataBalancerMetricsRegistry
Add domain to Context class
CNKAF-702: At SBK startup check for api state persistence topic
KC-382: Allow alter cluster configs for subset of configs in CCloud
KC-533: Allow headers to be injected into RBAC records
CIAM-64 Allow running other Providers with the MultiTenantAuthorizer
KSTORAGE-741: Deserialize errorOffsetAndEpoch from Header during scan and init
CNKAF-649 - Broker Removal operation with in-memory state
CNKAF-617: Complete the DescribeBrokerRemovals API handler
KSTORAGE-743: prevent tier.feature use with IBP < 2.4
METRICS-1555 Exposing proxy configs as top level telemetry configs and making them dynamically configurable
KCORE-142 - Offset Migration for Cluster Linking
[CNKAF-637] Config variable for auto-healing
KSTORAGE-425 Add tiered storage metrics to MetricsReporter
CONFLUENT: Bump major CCloud version #major
KC-370: Support Authentication events in ConfluentAuditLogProvider
CPKAFKA-3743; Consumer group tool does not work on internal endpoint
KSTORAGE-720: Always return tiered storage related configurations in DescribeConfigResponse
[CNKAF-742] Detect new, empty brokers in controller; call into SBK
KC-120: Integration Tests for ZK-Based ACLs for Periodic ACL Sync
CONFLUENT: Add PowerMockIgnore for Scram API in RBAC test to fix classloader issues in subsequent tests
SEC-959 Cluster Registry - Full Cluster Info CRUD Gatekeeping
KC-517: Update cluster link integration test to use admin client.
SEC-1015: Added kafka Metrics support to Confluent Authorizer, RBAC and AuditLog.
CNKAF-649: Revise SBK Executor’s locking strategy
KCORE-154: add tiered storage and CP to downgrade system test
KC-128: Validate source topic read access on mirror topic creation.
KCORE-329 (2/2): Adds ability to alter a topic’s mirror in TopicCommand.
Created Broker removal CLI
KSTORAGE-726: handle offset gaps in tier fetches
CPKAFKA-2789; Add support-metrics-fullcollector to ce-kafka
KC-506: Use cluster link tags for cluster link fetcher metrics
KSTORAGE-627: TierMetadataComparator tool for tier-topic recovery pipeline
KCORE 506: migrate secrets to Vault
Spelling fix in LdapGroupManager
KCORE-95: Encrypt sensitive cluster link configs stored in ZooKeeper
KCORE-437: Refactor ClusterLink managers to cleanly disable/enable
CONFLUENT: Improve performance of matchingAcls and AclSeqs.find
KCORE-49: Disables mirror topic partition and restricted config updates.
KCORE-485: Process mirror state during replication only on active destination leader
KCORE-487: Add AuthenticationEvent and update Audit log event schema
CONFLUENT: Introduce separate methods in KafkaApis for consumer and follower fetch handling
Update Netty to 4.1.50 and clean up logging of NettySslEngineFactory
CIAM-45 Allow RoleBindings at path scopes
CNKAF-636: SBK broker failure detection
KSTORAGE-627: TierPartitionForceRestore, TierPartitionState hotswap and tier task recovery
CNKAF-701: Refactor SampleStore topic existence check
Added tests for KafkaCruiseControl.
CNKAF-649: Add BrokerRemoval state machine
KSTORAGE-573: epoch state cache is flushed multiple times in recovery
[METRICS-1234] telemetry - add support for multiple exporters of the same type
KCORE-389: Add initial version of Audit API and update existing code
CNKAF-677: Remove ZK fallback for SBK and move methods from ExecutorUtils to Executor
CNKAF-684: Honor low utilization threshold settings for self-healing …
CIAM-28 Add ConfluentCloudCrnAuthority with special treatment of path
KSTORAGE-717: Filter all Confluent-specific topic configs from DescribeConfigsResponse
KSTORAGE-652: FileTierPartitionState base path is not updated correctly
CNKAF-699: Skip num cores or disk capacity update if node missing
KCORE-340: Propagate new partitions in source cluster to controller
KSTORAGE-653: re-add check_cluster_state check to tier upgrade test
KCORE-141: Tenant prefixed cluster links and topic mirroring for Cloud
KCORE-331: _confluent-license topic exponential backoff
fix some downstream validation issue
Add shutdown hook to KafkaHttpServlet interface
SEC-1008 Do not close null AdminClient
CONFLUENT: Remove ConfluentLeaderAndIsr and TierListOffset requests/responses
CNKAF-704: Remove tenant prefix from CreatePartition responses
CNKAF-314: Add DescribeBrokerRemovals{Request,Response} RPC and Admin Client API
CONFLUENT: Set Scala 2.13 as the default Scala version
KCORE-436: Client interceptor for destination tenant prefixing in cluster link clients
CIAM-25 Allow CRNs with Paths but no Clusters
emit deltas as cumulative metrics
CNKAF-649: (part 1/2) Introduce ShutdownManager for SBK broker removal
CNKAF-697 (part1): Increase min broker quota constants
CONFLUENT: Improve error message when cluster linking is disabled
KSTORAGE-412 Server side handling for the unclean leader flag from LeaderAndIsr re…
METRICS-1514 collect one-minute rates from yammer meters
Upgrade telemetry api to latest version
CONFLUENT: Enable fatal warnings with Scala 2.13 and fix warnings
CONFLUENT: Make createSslEngine(String, String) public again
KCORE-329: Alter mirrors API.
KCORE-426: Server should not resolve DNS on establishment of SASL connection
CPKAFKA-4923: Disable audit logging in RbacTest::test_ldap_to_rbac_authorizer_upgrade
CNKAF-632: RemoveBrokers api plumbing
switch downstream validation
[MINOR] Remove scoverage plugin from SBK build.
Bump Chart versions
Merge changes from Apache Kafka
CONFLUENT: Always close consumer in ConsumeBenchWorker
KCORE-438 (1/2): Remove topic read test on cluster link creation.
CNKAF-638: Introduce SBK internal RPC client
KCORE-41: Disable local produce requests for mirror topics.
CONFLUENT: Fix build warnings in new ACL sync code
CLIENTS-1476: Create Kafka HTTP Server interfaces and loading mechanism.
KCORE-99: Periodic ACL Migration
CNKAF-622: Properly handle dynamic config changes.
KCORE-435: Propagate cluster link create/delete using ZK change notification
KCORE-427: Add broker config to enable cluster linking feature
CONFLUENT: Don’t reach into java.lang.reflect.Field internals
KSTORAGE-683: Alter fencing logic for recovery work
CPKAFKA-4982 -Change Trogdor’s Helm chart to deploy in parallel.
KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
KCORE-167: Process replication failures with retry timeout to move partition to failed state
CONFLUENT: Fix locking during metadata restart for link reconfiguration
KSTORAGE-677: FIX: Reset archiver max lag whenever there is no lag
remove unnecessary labels from kafka server resource
CNKAF-629: Add InitiateShutdown Request
KCORE-55: Adds mirrorTopic and clusterLink to topic creation API.
KCORE-53: Dynamic security config update for inter-cluster connections
CNKAF-483: Added static and dynamic configs for request backpressure tuning
KSTORAGE-26: Fix non-deterministic TierEpochStateReplicationTest
CNKAF-593: Consolidate all thread creation in KafkaCruiseControlThreadFactory
KCORE-104: Fix truncation during unclean leader election for linked partition
CONFLUENT: Simplify Gauge definitions in Partition
CNKAF-610: Rename StartRebalance RPC and DrainBroker Admin API to RemoveBrokers
[CNKAF-606] Stop proposal execution on Cruise Control shutdown/SBK deactivation
[CNKAF-558] Handle SSL interbroker listeners for SBK clients.
CONFLUENT: Ensure license validator is closed in test to avoid impacting subsequent test
KSTORAGE-648: remove S3TierObjectStore multi-part log segment upload support
CNKAF-596: Add a test to verify SBK API access with RBAC
KSTORAGE-628: Implement TierPartitionFence injector tool for recovery
KSTORAGE-63: pt2, add TierTopicListeners, add getAndRemoveAll and tests
KSTORAGE-63: pt1, add TierPartitionForceRestore metadata
SEC-912 - Allow MDS to make changes to dynamic configs
KCORE-341: Fix permission type in authorizer debug log entry
SOAK: Revise soak workloads
[SEC-912] Allow MDS to make changes to dynamic configs
KSTORAGE-631: ducker up should not require GCS credentials to be activated in Dockerfile
CNKAF-577: Handle null config values when describing static throttle configs
CONFLUENT: Add support for Scala 2.13
WFLOW-51: Implement a beginShutdown hook which allows for triggering ControlledShutdown
KCORE-42 (4/4): Adds mirror SyncTopicsConfigs task.
KCORE-48: Broker implementation for cluster linking APIs.
KCORE-48 (partial): Adds ClusterLinkCommand.
KCORE-330: Allow brokers to complete start up with configured license
[CNKAF-594] Support broker shutdown for SBK
[CNKAF-492] Set some more config parameters for SBK
Update cloud Dockerfile to use JDK 14.0.1
CNKAF-313: Migrate SBK to use DataBalancerMetricsRegistry
CNKAF-549: Add metric for self healing enabled
Instructions for exporting metrics to Sandbox env
KSTORAGE-644: Separate fencing error metrics for primary & catchup consumers
CONFLUENT: Update JDK to 11.0.7 by bumping cc-base to 5.1.0
KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
KCORE-137 (partial): Expands per-topic cluster link state.
[SEC-895] ConfluentAuditLogProvider should handle reconfiguration wit…
CONFLUENT: Reduce allocations in requests via buffer caching
KAFKA-9864: Avoid expensive QuotaViolationException usage (#8477)
CNKAF-50: Add metrics for replication throttling
KCORE-51: Adds cluster link metadata object.
CNKAF-585; Multi tenant interceptor rejects create topic requests with NO_REPLICATION_FACTOR or NO_NUM_PARTITIONS when quotas are enabled
KCORE-316: Don’t load configs from ZK when the log has already been loaded
Fix MultiTenantTest
CPKAFKA-1836: Simplify forwarding scripts by just adding .sh suffix
CONFLUENT: add utility method to write public keys to pem files (#19)
CAAS-1960: Record authenticated tenant connection/disconnection metrics
KAFKA-7304: Limit connection accept rate to prioritize other channels (#29)
CONFLUENT: Add Aegis Proxy
CONFLUENT: Update cc-base to v2.3.0
Merge changes from apache kafka
CPKAFKA-2164: Ensure that we specify a charset when writing out and reading string bytes
CONFLUENT: Fix unused compiler warnings
[SEC-901] Speed up SameClusterTest
KCORE-42 (partial): Adds functions for merging source + mirror topics…
SEC-728 - MDS Cluster Registry - Enable Dynamic Configuration
KCORE-48 (partial): Adds ClusterLinkClientManager for managing admin …
Towards KCORE-165 (partial): Adds ClusterLinkScheduler for handling cluster l…
Towards KCORE-136: Cluster linking APIs client+RPC wiring.
[CNKAF-582] Handle topic creation during checkStartupComponents
CNKAF-372: Disk capacity sizing
CONFLUENT: Remove unnecessary allocations in hot matchingAclExists
METRICS-1415 fix collection of integer gauges
CNKAF-313: Adds a MetricsRegistry wrapper class for SBK
[CNKAF-370] Support dynamic enable/disable of SBK
[CNKAF-492] Adjust the default parameters of the KafkaCruiseControlConfig.
KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
KSTORAGE-628: Introduce RecoveryUtils class with recovery tooling related functions
CPKAFKA-4647: Disable audit logging in RbacTest::test_broker_failure test
KSTORAGE-628: Introduce TierPartitionFence event type + fencing logic
add AppInfo version+commitId to metrics messages
KSTORAGE-577: System test for fencing logic, along with CLI tool to trigger fencing
KSTORAGE-632: Better logging for replica fetcher state transitions for tiered storage
KSTORAGE-381 Tooling around TierMetadataValidator to use tier-validator as a cc-service
CNKAF-536: Reduce number of warnings when SBK creating clients
[METRICS-1361] telemetry-api -> opencensus-proto
CNKAF-537: Add drainbrokers adminClientAPI
CNKAF-562: Change log level for self-healing being triggered from WARN to INFO.
KSTORAGE-613: fix log start offset initialization on full truncation
KSTORAGE-626: Remove metrics reporter configuration when configuring internal tier topic clients
[CNKAF-528] Use the inter-broker listener for SBK clients.
Prevent warning message on every client metadata update
KSTORAGE-580: Return a generic error message when compaction is enabled on a tiered topic
KCORE-40: Inter-cluster fetcher for cluster linking
Update role, Owner tags for cost reporting
KCORE-297: Fix wildcard principals for centralized ACLs and no.acl.found for ConfluentServerAuthorizer
CNKAF-541: Reduce the number of tests run in RandomClusterTest
METRICS-1232 - Whitelist check for individual and derived metrics
[METRICS-1233] make telemetry metrics whitelist a dynamic config
CNKAF-548: Skip checking quotas when recording replication traffic
Set TopicExclusion string in SBK-generated CruiseControl config.
KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422) (#1477)
KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422)
KSTORAGE-331, KSTORAGE-392: add and improve tier component listener metrics and heartbeats
CPKAFKA-4753: Add metrics tracking total txn time, txn timeouts, and illegal state errors
CPKAFKA-4649: Ensure LDAP group manager is recreated on writer failover
SBK MINOR: Drain permits in startUp
CNKAF-319: Add Start rebalance protocol
modify kafka_path to only set project for confluentplatform
CNKAF-520, CNKAF-525: Wait for KafkaSampleStore check to complete at startup
KSTORAGE-411: Pass a hint down from controller to the leader in case of an unclean leader election
KSTORAGE-453: Persist leader epoch in tier checkpoint file
re-schedule pr scans to weekly
[CNKAF-492] SBK: CruiseControl Property defaults
Publish preview artifacts with preview release settings
CNKAF-50: Add static broker-level replication throttles
SBK: Implement KafkaDataBalancer startup/shutdown methods
CPKAFKA-4772: Fix PhysicalClusterMetadataTest’s tests
METRICS-1292 Dynamic Update of API key and secret in Kafka.
[CNKAF-216] Derive SBK capacity values from config parameters directly.
CONFLUENT: Make Rest Client Http Authenticator configurable
CNKAF 362: Topic exclusion logic for SBK in KafkaDataBalancer
CNKAF-305: Dynamic configs for sbk - enable/disable, throttle
CONFLUENT: Update cc-trogdor-service and cc-soak-clients to 0.803.0-6.0.0-ce-SNAPSHOT
KSTORAGE-545: Handle failed deletions from GcsTierObjectStore
CONFLUENT :HttpBearerCredentialProvider must implement configurable
ST-3292: migrate to semaphore2
CNKAF-19: ADB: Soak test self-healing
KSTORAGE-579: FIX fencing advancement issues
KSTORAGE-574: producer and epoch state restore is not resilient to failures
KSTORAGE-399 : Crash broker if FileTierPartitionState encounters corrupted flushed state file during initialization.
SEC-162: RestClient refactor regression
[CNKAF-479] Support executing ce-sbk objects with the rest of Kafka
[CNKAF-491] Remove clusterConfigs functionality from the TopicConfigProvider.
Revert “KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319) (#1401)”
KSTORAGE-482 Enhancing TierMetadataValidator tool to verify objects in S3 TierObjectStore
KSTORAGE-559: add maximum eligible segments deleted on every cleanup run
CNKAF-486: Add filter to exclude internal topics
Put an initial implementation of the DataBalancer interface in ce-sbk.
KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319)
KSTORAGE-568: bound hotset retention by LSO
KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions)
CPKAFKA-4670 Fix for system tests running 0.8.2.2 version producers
CNKAF-324: Allow configuring a delay before broker registration with ZK.
CNKAF 447: Update SBK mode config to a flag
[CNKAF-477] Restore ConfluentMetricsReporterSampler
KSTORAGE-572: FIX: Access fenced partitions metric under lock in TierTopicConsumer
CPKAFKA-4676: TierRoundtripTest.test_tier_roundtrip timeout increase
KSTORAGE-548: Check for missing init leader messages
KCORE-234: Upgrade netty to 4.1.48.Final and netty-tcnative to 2.0.29.Final
MCM-965: Reduced physical cluster metadata reload period to 2 mins
CC-7979: Add new topic tracking operations of KIP-558 to default RBAC roles
KAFKA-9685: Solve Set concatenation perf issue in AclAuthorizer (cherry-pick)
Bump up rollingupgrade deb package from 0.5.0 to 0.9.0
[CPKAFKA-4658] Restore removed SchemaRegistryUrl Config.
KCORE-225: Schedule periodic license validation task at fixed rate
DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client (#1374)
DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client
KSTORAGE-465: GCS tier deletion manager system test support
v5.5.x KSTORAGE-524: Fix potential NPE in DelayedFetch
KCORE-199: Set default values for schema configs on topics to avoid NPE
CNKAF 289: Increase partition count in soak cluster
[CNKAF-283] Put heavily trimmed CruiseControl into ce-kafka/ce-sbk
KSTORAGE-550 Narrow scope of KafkaException to avoid handling of any unwanted bug.
CONFLUENT MINOR: Revert #799, remove batch verifier from eos topic consumers
KCORE-111: Default replica placement constraints for external topics
CONFLUENT: fix exhaustive match warnings in TierArchiver
KSTORAGE-354: avoid roundtrip and connection in GCS fetch path
KCORE-175: Extend TopicPlacement to include assignment validation logic
SEC-159: RestRequest pluggable HTTP auth
KSTORAGE-543: Fix AutoAbortingS3InputStream: account for # read bytes being -1
KSTORAGE-214, KSTORAGE-407: Bump FileTierPartitionState version
Tier Storage system test for Consumer.offsetForTimes API
Refactoring FIPS implementation
KSTORAGE-389: Stronger detection of duplicate message processing for tiered storage
CPKAFKA-2336: Log layer metrics for tiered storage
CNKAF-305: Remove topic config from DynamicBalancer config
METRICS-1138 Removing shading of transitive dependencies due to telemetry-client jar
KCORE-190: Add max LSO lag metric
KSTORAGE-468 : Metadata states validation insync with log start offset.
CNKAF-304: Default value for rack aware flag
KCORE-104: Re-instate PartitionLockTest.testAppendReplicaFetchWithUpdateIsr
KSTORAGE-524: Fix potential NPE in DelayedFetch
SEC-804: Wait for config update before verification in audit config change test
CPKAFKA-1953: resume materialization from TierTopicManager commit positions
Log TierPartitionStatus to differentiate online vs catchup consumer
KSTORAGE-450: default confluent.tier.fetcher.num.threads = 4
KSTORAGE-457 ListOffsetRequest v0 does not account for tiered section…
KSTORAGE-330, KSTORAGE-328, KSTORAGE-325 TierStorageMaterialization recovery and validation tool.
KSTORAGE-452: Improve README for system tests w/ Tiered Storage
KSTORAGE-400: Handle duplicate PartitionDeleteInitiate messages
KSTORAGE-420: missing toString on MergedLog
KSTORAGE-460: Disable preferred tier reads for replica fetch
CPKAFKA-4369. Cherry pick of #1228 into 5.5.x
KSTORAGE-214, KSTORAGE-407: Fence user partition failures when materializing Tier Topic events
KSTORAGE-510: Ensure that PendingFetch::finish() stops the PendingFetch
Reduced test strength to ignore consumer data coverage caused by catc…
CPKAFKA-4126: Add prefix to system test config to match MDS changes
SEC-147: Refactor RestClient
KSTORAGE-519: Fix: Ensure unknown exceptions from DeletionTask are isolated to user partition
KSTORAGE-511: Add size asserts for TierPartitionStateEntry
KCORE-183: Avoid new allocations in MultiTenantRequestContext.filteredMetadataResponse call
KSTORAGE-344: TierMaterializationValidation as service for System Tests
CONFLUENT: Fix bug in result handling from readFromLocalLog
CNKAF-323: Add metric with per-tenant partition count
KCORE-151: Allow some reassignment of offline brokers
KSTORAGE-508: System test s3 prefix random generation incorrectly app…
KSTORAGE-505: Tier deletion manager system test failing in vagrant runs
KSTORAGE-424: Allow use of SSL for internal tier topic clients
KSTORAGE-384 Adding system test for tiered state file corruption
Merge branch ‘ccs-master’
KSTORAGE-502: Improve documentation for tiered storage configurations
CNKAF-305: Dynamic configs for SBK - kafka-side changes
KCORE-93: Removing ZK utils from ce-kafka
KCORE-110: Introducing new flag to display partitions violating topic replica constraints
CNKAF-304: Update network config name
CNKAF-304: Add static configs for self-balancing kafka
KCORE-104: Re-introduce PartitionLockTest
CONFLUENT: Update protobuf to 3.11.4 (#1237)
Update protobuf to 3.11.4
CPKAFKA-2700; Disable license validator in MultiTenantAuthorizer
Define a DataBalancer object in the Kafka Controller.
SEC-744 Make Audit Log Configs visible to AdminClient
METRICS-582: Adding tracking for kafka.broker.rack in telemetry
SEC-752 AuditLog FetchFollower
KSTORAGE-163: Initial implementation of TierFetcher memory bounding.
CPKAFKA-2705: Added retention to soak topics
KSTORAGE-479: Revert “KSTORAGE-464: Fix: ensure a queued TierTopic ev…
Add manual testing sections to ce-metrics README
SEC-746 Ignore unknown JSON properties for forwards compatibility
KSTORAGE-464: Fix: ensure a queued TierTopic event future throws when replaced
KSTORAGE-337/KSTORAGE-348: tier deletion manager system test
CONFLUENT: Add check in consume test util to verify records were consumed
METRICS-541 increase metric reporter default publishing time from 15 se…
CPKAFKA-3878: Re-enable scala inlining
CPKAFKA-4369. Copy Kafka configs when creating LogConfig- Part 1 of 2
CONFLUENT: Log better message on soak clients task creation 409 error response
TierArchiver: sleep outside of synchronized block in main loop

Security¶

Validate KSQL/RBAC cannot initialize on non-SASL protocols
Check authentication.roles in KSQL bearer token requests
CLIENTS-1658: Remove kafka core dependency from kafka-rest-security-plugin
SEC-1034: log4j migration to confluent repackaged version
remove test scope from connect-runtime dependency
DG-706: fix bug in schema registry client tests, as well as some refactor
DG-708: Skip Schema Validation query range check for first lookup
Include Kafka dependencies to KSQL packaging
Bump Vert.x version in confluent-ksql-security-plugin to 3.9.1
fix: Make rest-utils compile scope as ksqlDB no longer pulls this in
SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
Remove AdminClientWrapper.
Switch from application/vnd.api+json to application/json.
Migrate off JSON:API.
SEC-951: Upgraded bc-fips jars to 1.0.2
Add authorizations for mode endpoints
Add test that reproduces rest-proxy authorization bug.
Fix authentication for non-producer POST requests in kafka-rest.
Fix issue SSL class could not be found
Remove obsolete kafka-rest dependencies.
Fix bugs in Vertx auth plugin loginservice initialisation and unimplemented response method
Fix Content-Type for produce request.
Fix test due to ksql refactoring.
Remove kafka-rest V1 APIs and all their dependencies.
DG-403 Add missing method to retrieve schema refs
Vertx auth adaptors
SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
Add confluent-security-plugins-common module back
Remove support-metrics-common dependency
KCORE-82: Removing ZK licensing code and introducing LicenseManager
Use kafka.metrics.KafkaYammerMetrics instead of com.yammer.metrics.Metrics
Fixing compiling error due to KIP-515

MQTT Proxy¶

CC-11294: Bumped up Netty version to be consistent with Kafka and rest of CP.
METRICS-1989: confluent-kafka-mqtt depends on confluent-telemetry
CC-10900: Add confluent-telemetry to production jars.
CC-9739: Pass producer-striped configs and bootstrap.servers to AdminClient when getting Kafka cluster id.
CC-9786: Depend on confluent-metrics in tests only.
fix for METRICS-1733 change telemetry config constant names
CC-8363: Configure telemetry reporter.

Metadata Service¶

SEC-1421 - Set default replicator factor when creating audit log topic
SEC-1369 - Update MDS to non-broken version of Maven Assembly plugin
SEC-1034: log4j migration to confluent repackaged version
SEC-866 - move audit log “other” to “management”
[SEC-1345] Read HTTP Reason phrase when connection closed
Move kafka-http-server project into clients project
SEC-1323 - Make MDS Api doc not have everything be Bold
Fixed code which hard code security protocl to SASL_PLAINTEXT, should…
SEC-1268 - Backout confluent.security.event.logger.destination.log.* properties
SEC-1259 - Clarify audit log config API spec
SEC-777 - Cluster Registry & MDS should be lenient with it’s JSON Parsing
SEC-777 Cluster Registry & MDS should be lenient with it’s JSON Parsing
SEC-1222 Role Resources For Principal API throw a null pointer exception if ClusterName is passed in ResourceRequest
SEC-1089 CR C3 - Return Nice Names for all other Private UI endpoints
Fixed defect that RbacApiAppConfig missed all audit config with prefix confluent.security.
CLIENTS-1599: Convert KafkaHttpServlet to rest-utils Application.
SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
SEC-1212 - Do not create a new admin client if admin config not changed
SEC-1097: Disallow reassignment of name <=> scope.
SEC-1088 CR C3 - Return Nice Names for C3 visibility endpoint
MMA-7805: Fix for correctly packaging cluster-information-migration-script
MMA-7805 Script to import and export cluster information data to and from Registry
SEC-1191 - Assign default topics to produce, consume and other category
SEC-1021: updated validation + make error messages useful
SEC-1130 - Inject ConfluentAdmin from ce-kafka
Migrate MDS to KafkaHttpServlet
Added more testing cases for verifying audit log backend API.
SEC-1127 - Fixed a defect that AuditLogConfigSpec didn’t get updated in DynamicC…
SEC-988 Cluster Registry Fix to make CLI backwards compatible
SEC-1122 - Fix 6.0.x build break
SEC-986: Lockdown cluster name
SEC-940 - Make clusters optional in MdsScope in swagger spec
SEC-1037: Return sorted ‘full’ clusters followed by sorted ‘redacted’ clusters for cluster registry
SEC-1003 Cluster Registry : Migrate from “name” to “clusterName”
SEC-1020 - Cluster Info Protocol Validation
SEC-982: Nice name gatekeeping
CIAM-12 - Try loading token.key.path as a public key for cloud
SEC-731: Single shot overwrite of multiple clusters
SEC-959: Full Cluster Info CRUD Gatekeeping for Cluster Registry
SEC-1019 Cluster Registry : Introduce security protocol field to ClusterInfo
SEC-985: MDS CR NiceNames : Update basic RBAC endpoints
SEC-1015: Fixed compile error due to interface changed by SEC-1015 in ce-kafka.
For backward compatibility we name the json scope object as ‘scope’
Add support for Cluster Scope object with clusters or paths to MdsSco…
CIAM-49 Spec file for MDS API v2alpha1 - fix whitespace
CIAM-49 Spec file for MDS API v2alpha1
SEC-1001: Require at least one host in a Cluster
SEC-926: Using dynamic config for cluster registry backend
SEC-941 Add Http api that returns an array of Mds Scope object.
SEC-942: MDS CR NiceNames : Introduce MdsScope Object
SEC-965: Sorted-ness of ClusterRegistry
SEC-943 - MDS CR : Investigate creating ResourceType for CR CRUD
Fix OpenAPI specification format
SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
SEC-949 - MDS CR : Update CR given Audit logs fix
SEC-728 - Cluster Registry - Make the backend be a broker Dynamic Config
SEC-891 - MDS : Short term fix flakey Token tests by isolating them
[SEC-844] restore /security/1.0 prefix
SEC-871 - MDS Test : Test / document OOTB Ldap and RBAC case sensitivity
SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
SEC-800 - MDS : Remove MdsLoginService.roles
SEC-751 MDS - Clean up the openapi yaml spec for CLI
SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
SEC-727 - Cluster Registry - Lock down the basic list of clusters
SEC-726 - Service Reg - Basic list of clusters

Replicator¶

REP-1176 Log error when connector submission fails for Replicator executable
REP-1131: Add support for Replicator with regex router
REP-1055 Enable converter configs for Replicator connector
REP-787: Synchronize fetching end offsets from source cluster to calculate lag
REP-824: Changed JMX initialization to use method from KIP-606
REP-721: Delete old license validation code from codebase
REP-795 moved translated records log to DEBUG and only print if records were translated
REP-761: ensure offset.topic.commit is false if topic.preserve.partitions is false
REP-755: retrieve port from –connect-url if specified
REP-743: Fix issue where we don’t properly log the topic that got an UnknownTopicPartitionException
REP-754 fix for offset commit failure resulting in loss of offsets
REP-689: Reduce log level of offset commit exception from ERROR to WARN
REP-665: Deleting all ZK dependencies from code base
REP-673: Port ZK dependencies for replicator-verifier into the verifier module
REP-664: Remove ZK dependencies from ReplicatorSourceConnector and ReplicatorSourceTask