Confluent Platform Component Changelogs¶
This topic provides changelogs for the individual Confluent Platform components.
Version 6.0.0¶
Community Features¶
Common¶
- PR-294 - SEC-1307: Backport “log4j replacement with confluent repackaged version”
- PR-278 - SEC-1334: update confluent-log4j version
- PR-277 - KC-613: Update slf4j version to match AK’s.
- PR-274 - ST-3988: Update jackson version to 2.10.5
- PR-275 - SEC-1305: update jackson-bom version
- PR-272 - METRICS-1918: bump gson from 2.8.5 -> 2.8.6
- PR-270 - SEC-1034: log4j migration to confluent repackaged version
- PR-269 - Upgrade Jackson to 2.10.2 and Avro to 1.9.2
- PR-265 - downgrade maven-deploy-plugin to 2.8.2
- PR-258 - update various maven plugins
- PR-257 - ST-1323: Enable source jars if Apache or Confluent license files are found
- PR-262 - fix: Pin the Scala version
- PR-261 - Update ZooKeeper to 3.5.8
- PR-260 - Update Scala version to 2.13
- PR-259 - add property for jackson-jsonschema version
- PR-248 - Upgrade Avro to 1.9.2
Kafka¶
- PR-9197 - Revert KAFKA-9309: Add the ability to translate Message to JSON
- PR-9172 - KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect
- PR-392 - KAFKA-2715: Removed previous system_test folder
- PR-384 - KAFKA-2714: Added integration tests for exceptional cases in fetching
- PR-9089 - KAFKA-10224: Update jersey license from CDDL to EPLv2
- PR-9066 - KAFKA-10287: Skip unknown offsets when computing sum of changelog offsets
- PR-9086 - FIX: Remove staticmethod tag to be able to use logger of instance
- PR-9075 - KAFKA-10306: GlobalThread should fail on InvalidOffsetException
- PR-9065 - KAFKA-10301: Do not clear Partition#remoteReplicasMap during partition assignment updates
- PR-9051 - KAFKA-10268: dynamic config like “–delete-config log.retention.ms” doesn’t work
- PR-9026 - KAFKA-10274; Consistent timeouts in transactions_test
- PR-8979 - KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers
- PR-8239 - KAFKA-9666: Don’t increase transactional epoch when trying to fence if the log append fails
- PR-9009 - KAFKA-6453: Document how timestamps are computed for aggregations and joins
- PR-8994 - KAFKA-10247: Correctly reset state when task is corrupted
- PR-9010 - KAFKA-10262: Ensure that creating task directory is thread safe
- PR-9005 - KAFKA-10263: Do not assign standby for revoking stateless tasks
- PR-8996 - KAFKA-10249: don’t try to read un-checkpointed offsets of in-memory stores
- PR-8934 - KAFKA-10134: Use long poll if we do not have fetchable partitions
- PR-8989 - KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups
- PR-8962 - KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory
- PR-8983 - KAFKA-8398: Prevent NPE in forceUnmap
- PR-8858 - KAFKA-10153: Error Reporting in Connect Documentation
- PR-8917 - KAFKA-10180: Fix security_config caching.
- PR-8905 - KAFKA-10173: Fix suppress changelog binary schema compatibility
- PR-8926 - KAFKA-10166: always write checkpoint before closing an (initialized) task
- PR-8924 - KAFKA-10198: guard against recycling dirty state
- PR-8900 - KAFKA-10169: swallow non-fatal KafkaException and don’t abort transaction during clean close
- PR-8887 - KAFKA-10135: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager…
- PR-8896 - KAFKA-10185: Restoration info logging
- PR-8890 - KAFKA-9891: add integration tests for EOS and StandbyTask
- PR-8822 - KAFKA-10113; Specify fetch offsets correctly in LogTruncationException
- PR-8876 - KAFKA-10167: use the admin client to read end-offset
- PR-8841 - KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition
- PR-8853 - KAFKA-10147 MockAdminClient#describeConfigs(Collection<ConfigResource…
- PR-8882 - KAFKA-10165: Remove Percentiles from e2e metrics
- PR-8856 - KAFKA-10150: task state transitions/management and committing cleanup
- PR-8880 - KAFKA-10169: Error message when transit to Aborting / AbortableError / FatalError
- PR-8872 - Fix log message for transition from standby to active
- PR-8873 - Avoid WARN log message when re-init from checkpoint skipped
- PR-8849 - KAFKA-10144: clean up corrupted standby tasks before attempting a commit
- PR-8764 - KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes
- PR-8787 - KAFKA-10085: correctly compute lag for optimized source changelogs
- PR-8818 - KAFKA-10086: Integration test for ensuring warmups are effective
- PR-8069 - KAFKA-9374: Make connector interactions asynchronous
- PR-8828 - KAFKA-9216: Enforce that Connect’s internal topics use compact cleanup policy
- PR-8829 - KAFKA-10115: Incorporate errors.tolerance with the Errant Record Reporter
- PR-8455 - KAFKA-9845: Warn users about using config providers with plugin.path property
- PR-8502 - KAFKA-9066: Retain metrics for failed tasks
- PR-8847 - KAFKA-7833: Add missing test
- PR-8453 - KAFKA-9841: Revoke duplicate connectors and tasks when zombie workers return with an outdated assignment
- PR-8663 - KAFKA-9985: Sink connector may exhaust broker when writing in DLQ
- PR-8833 - KAFKA-9441: remove prepareClose() to simplify task management
- PR-8825 - KAFKA-7833: Add StateStore name conflict check in InternalTopologyBuilder
- PR-8630 - KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation
- PR-8775 - KAFKA-10079: improve thread-level stickiness
- PR-7384 - KAFKA-8938: Improve allocations during Struct validation in ConnectSchema
- PR-8827 - KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used
- PR-8805 - KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group
- PR-8783 - KAFKA-10063 UnsupportedOperation when querying cleaner metrics after shutdown
- PR-8803 - KAFKA-10102: update ProcessorTopology instead of rebuilding it
- PR-8676 - KAFKA-10005: Decouple RestoreListener from RestoreCallback
- PR-8684 - KAFKA-10012 Reducing memory overhead associated with strings in SelectorMetrics
- PR-8270 - KAFKA-9216: Enforce internal config topic settings for Connect workers during startup
- PR-8820 - KAFKA-10097: Internalize checkpoint data
- PR-8804 - KAFKA-9851: Revoking Connect tasks due to connectivity issues should also clear the running assignment
- PR-8486 - KAFKA-9840: Skip End Offset validation when the leader epoch is not reliable
- PR-8135 - KAFKA-9570: Define SSL configs in all worker config classes, not just distributed
- PR-8814 - KAFKA-10111: Make SinkTaskContext.errantRecordReporter() a default method
- PR-8811 - KAFKA-10110: Corrected potential NPE when null label value added to KafkaMetricsContext
- PR-8776 - KAFKA-9441: Improve Kafka Streams task management
- PR-8759 - KAFKA-10066: TestOutputTopic should pass record headers into deserializers
- PR-8755 - KAFKA-10069: Correctly remove user-defined “predicate” and “negate” configs from transformation properties
- PR-8644 - KAFKA-9313: Set use_all_dns_ips as the new default for client.dns.lookup (KIP-602)
- PR-8784 - KAFKA-9788: Use distinct names for transaction and group load time sensors
- PR-8792 - KAFKA-10089 The stale ssl engine factory is not closed after reconfigure
- PR-8782 - KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends
- PR-8785 - KAFKA-10084: Fix EosTestDriver end offset
- PR-8786 - KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests
- PR-8737 - KAFKA-9945: TopicCommand should support –if-exists and –if-not-exists when –bootstrap-server is used
- PR-8695 - KAFKA-9320: Enable TLSv1.3 by default (KIP-573)
- PR-8777 - KAFKA-10082: Fix the failed testMultiConsumerStickyAssignment
- PR-8706 - KAFKA-10030 allow fetching a key from a single partition
- PR-8760 - Kafka-10064 Add documentation for KIP-571
- PR-8668 - KAFKA-9987: optimize sticky assignment algorithm for same-subscription case
- PR-7956 - KAFKA-9392; Clarify deleteAcls javadoc and add test for create/delete timing
- PR-8769 - KAFKA-10074: Improve performance of matchingAcls
- PR-8723 - KIP-569-KAFKA-9494: DescribeConfigsResponse - include additional metadata information
- PR-8739 - KAFKA-10056; Ensure consumer metadata contains new topics on subscription change
- PR-8705 - KAFKA-10029; Don’t update completedReceives when channels are closed to avoid ConcurrentModificationException
- PR-8238 - KAFKA-9130: KIP-518 Allow listing consumer groups per state
- PR-8248 - KAFKA-9501: convert between active and standby without closing stores
- PR-8699 - KAFKA-9673: Filter and Conditional SMTs
- PR-8720 - KAFKA-9971: Error Reporting in Sink Connectors (KIP-610)
- PR-8736 - KAFKA-9802; Increase transaction timeout in system tests to reduce flakiness
- PR-8731 - KAFKA-10050: kafka_log4j_appender.py fixed for JDK11
- PR-8589 - KAFKA-9146: KIP-571 Add option to force delete active members in StreamsResetter
- PR-8691 - KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter
- PR-8709 - KAFKA-9952; Remove immediate fetch completion logic on high watermark updates
- PR-8692 - KAFKA-10018: Change command line tools from /bin/sh to /bin/bash
- PR-8697 - KAFKA-9983: KIP-613, add INFO level e2e latency metrics
- PR-8376 - KAFKA-9724 Newer clients not always sending fetch request to older brokers
- PR-8504 - KAFKA-9298: reuse mapped stream error in joins
- PR-8716 - KAFKA-6145: KIP-441: Fix assignor config passthough
- PR-6824 - KAFKA-7523: Add ConnectedStoreProvider to Processor API
- PR-8221 - KAFKA-9561: update task input partitions after rebalance
- PR-7886 - KAFKA-9353: Add groupInstanceId to DescribeGroup for better visibility
- PR-8722 - KAFKA-5295: Allow source connectors to specify topic-specific settings for new topics (KIP-158)
- PR-8395 - Added doc for KIP-535 and updated it for KIP-562
- PR-8118 - KAFKA-9472: Remove deleted Connect tasks from status store
- PR-2604 - KAFKA-4794: Add access to OffsetStorageReader from SourceConnector
- PR-8698 - KAFKA-10022:console-producer supports the setting of client.id
- PR-6284 - KAFKA-6755: Allow literal value for MaskField SMT
- PR-8357 - KAFKA-9767: Add logging to basic auth rest extension
- PR-8620 - KAFKA-9944: Added supporting customized HTTP response headers for Kafka Connect.
- PR-8511 - KAFKA-9888: Copy connector configs before passing to REST extensions
- PR-8654 - KAFKA-9931: Implement KIP-605 to expand support for Connect worker internal topic configurations
- PR-8713 - KAFKA-6145: Add unit tests for assignments of only stateless tasks
- PR-8696 - KAFKA-6145: KIP-441: Enforce Standby Task Stickiness
- PR-8679 - KAFKA-10003: Mark KStream.through() as deprecated
- PR-8658 - KAFKA-9980: Fix bug where alterClientQuotas could not set default client quotas
- PR-8379 - KAFKA-9780: Deprecate commit records without record metadata
- PR-8651 - kafkatest: Deploy VerifiableClient in constructor to avoid test timeouts
- PR-8472 - KAFKA-9855 - return cached Structs for Schemas with no fields
- PR-8608 - KAFKA-9950: Construct new ConfigDef for MirrorTaskConfig before defining new properties
- PR-8444 - KAFKA-8869: Remove task configs for deleted connectors from config snapshot
- PR-7942 - KAFKA-9409: Supplement immutability of ClusterConfigState class in Connect
- PR-8661 - KAFKA-9603: Do not turn on bulk loading for segmented stores on stand-by tasks
- PR-8671 - KAFKA-9859 / Add topics generated by KTable FK join to internal topic matching logic
- PR-8681 - KAFKA-10010: Should make state store registration idempotent
- PR-8682 - KAFKA-10011: Remove task id from lockedTaskDirectories during handleLostAll
- PR-8673 - KAFKA-9992: EmbeddedKafkaCluster.deleteTopicAndWait not working with kafka_2.13
- PR-8689 - KAFKA-6145: Add unit tests to verify fix of bug KAFKA-9173
- PR-8675 - KAFKA-10004: ConfigCommand fails to find default broker configs without ZK
- PR-8569 - KIP-551: Expose disk read and write metrics
- PR-8678 - Update Gradle to 6.4.1
- PR-8659 - KAFKA-9617 Replica Fetcher can mark partition as failed when max.message.bytes is changed
- PR-8618 - KAFKA-9955: Prevent SinkTask::close from shadowing other exceptions
- PR-8558 - KAFKA-8611 / KIP-221 documentation
- PR-8670 - KAFKA-10001: Should trigger store specific callback if it is also a listener
- PR-8090 - KAFKA-9537 - Cleanup error messages for abstract transformations
- PR-8674 - KAFKA-9996: Upgrade zookeeper to 3.5.8
- PR-8667 - KAFKA-9994: Handle task migrated inside corruption path
- PR-8665 - KAFKA-9984 Should fail the subscription when pattern is empty
- PR-8222 - KAFKA-9650: include human readable units in ms and bytes configs
- PR-8588 - KAFKA-6145: KIP-441: Improve assignment balance
- PR-313 - KAFKA-2641: Upgrade path for ZK authentication
- PR-8550 - KAFKA-9850 Move KStream#repartition operator validation during Topology build process
- PR-8648 - KAFKA-9966: Add internal assignment listener to stabilize EOS version 2 upgrade test
- PR-8596 - KAFKA-9821: persist followup rebalance in assignment and consolidate rebalance triggering mechanisms
- PR-8647 - KAFKA-9669; Loosen validation of inner offsets for older message formats
- PR-8254 - KIP-557: Add Emit On Change Support
- PR-8626 - KAFKA-9921: explicit handling of null values with retainDuplicates
- PR-8643 - KAFKA-9956: Authorizer APIs may be invoked more than once for a given request
- PR-8632 - KAFKA-9972: Only commit tasks with valid states
- PR-8628 - KAFKA-9942: Fixes ConfigCommand client quotas w/ default users.
- PR-8613 - KAFKA-6145: Set HighAvailabilityTaskAssignor as default in streams_upgrade_test.py
- PR-8230 - KAFKA-9667: Connect JSON serde strip trailing zeros
- PR-8483 - KAFKA-9865: Expose output topic names from TopologyTestDriver
- PR-8360 - KAFKA-9768: Fix handling of rest.advertised.listener config
- PR-7950 - KAFKA-9419: Integer Overflow Possible with CircularIterator
- PR-8602 - KAFKA-9947; Ensure proper shutdown of components in TransactionsBounceTest
- PR-8591 - KAFKA-6342: Move workaround for JSON parsing of non-escaped strings
- PR-8609 - KAFKA-9946; StopReplicaRequest deletePartition changes may cause premature topic deletion handling in controller
- PR-8496 - KAFKA-9748: Add Streams EOS version 2 integration test
- PR-8607 - KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
- PR-8527 - Remove dependencies on deprecated –zookeeper command flags in junit tests
- PR-8551 - KAFKA-9918 SslEngineFactory is NOT closed when channel is closing
- PR-8565 - KAFKA-9798: Send one round synchronously before starting the async producer
- PR-8533 - KAFKA-9589: Fixed bug in V2 log validator tests
- PR-8577 - use appropriate fn for readability. (maybe)
- PR-8586 - KAFKA-9939; Fix overcounting delayed fetches in request rate metrics
- PR-8595 - KAFKA-8410: Revert Part 1: processor context bounds (#8414)
- PR-8597 - KAFKA-6145: KIP 441 remove balance factor
- PR-8590 - KAFKA-6145: Remove check to reuse previous assignment
- PR-8204 - KAFKA-9633: Ensure ConfigProviders are closed
- PR-8554 - KAFKA-9919: Add logging to KafkaBasedLog::readToLogEnd
- PR-8567 - KAFKA-9652: Fix throttle metric in RequestChannel and request log due to KIP-219
- PR-8559 - KAFKA-9922:Update examples README
- PR-8442 - KAFKA-9830: Implement AutoCloseable in ErrorReporter and subclasses
- PR-8578 - KAFKA-9875: Make integration tests more resilient
- PR-8582 - KAFKA-9932: Don’t load configs from ZK when the log has already been loaded
- PR-8574 - KAFKA-9925: decorate pseudo-topics with app id
- PR-8580 - KAFKA-9832: fix attempt to commit non-running tasks
- PR-8540 - KAFKA-9127: don’t create StreamThreads for global-only topology
- PR-8568 - KAFKA-9176: Retry on getting local stores from KafkaStreams
- PR-8548 - KAFKA-9823: Follow-up, check state for handling commit error response
- PR-8541 - KAFKA-6145: KIP-441: Add TaskAssignor class config
- PR-8564 - KAFKA-9921: disable caching on stores configured to retain duplicates
- PR-8566 - Fix minor code issue
- PR-8525 - KAFKA-9885; Evict last members of a group when the maximum allowed is reached
- PR-8524 - KAFKA-9866: Avoid election for topics where preferred leader is not in ISR
- PR-8509 - KAFKA-9839: Broker should accept control requests with newer broker epoch
- PR-8224 - KAFKA-9704: Fix the issue z/OS won’t let us resize file when mmap.
- PR-8326 - KAFKA-8639: Replace AddPartitionsToTxn with Automated Protocol
- PR-8445 - KAFKA-9823: Remember the sent generation for the coordinator request
- PR-8536 - KAFKA-9883: Add better error message when REST API forwards a request and leader is not known
- PR-8537 - KAFKA-9907: Switch default build to Scala 2.13
- PR-8464 - KAFKA-9852: Change the max duration that calls to the buffer pool can block from 2000ms to 10ms
- PR-8417 - KAFKA-8955: Add an AbstractResponse#errorCounts(Stream) and tidy
- PR-8530 - KAFKA-9388: Refactor integration tests to always use different application ids
- PR-8482 - KAFKA-9863: update the deprecated –zookeeper option in the documentation into –bootstrap-server
- PR-8497 - KAFKA-6145: KIP-441 Build state constrained assignment from balanced one
- PR-8520 - Add explicit grace period to tumbling window example
- PR-8522 - KAFKA-9868: Reduce transaction log partitions for embed broker
- PR-8523 - Adding github whitelist
- PR-8475 - KAFKA-6145: KIP-441: Add test scenarios to ensure rebalance convergence
- PR-8501 - KAFKA-9881: Convert integration test to verify measurements from RocksDB to unit test
- PR-8448 - KAFKA-9796; Broker shutdown could be stuck forever under certain conditions
- PR-8471 - KAFKA-9854 Re-authenticating causes mismatched parse of response
- PR-8476 - KAFKA-9838; Add log concurrency test and fix minor race condition
- PR-8378 - KAFKA-9779: Add Stream System Test 2.5
- PR-6210 - KAFKA-7885: TopologyDescription violates equals-hashCode contract.
- PR-8092 - KAFKA-9540: Move “Could not find the standby task while closing it” log to debug level
- PR-8256 - KAFKA-9675: Fix bug that prevents RocksDB metrics to be updated
- PR-8440 - KAFKA-9832: extend Kafka Streams EOS system test
- PR-8492 - KAFKA-9739: Fix for 2.5 branch
- PR-8403 - KAFKA-9797; Fix TestSecurityRollingUpgrade.test_enable_separate_interbroker_listener
- PR-8477 - KAFKA-9864: avoid expensive QuotaViolationException usage
- PR-8469 - [KAFKA-9826] Handle an unaligned first dirty offset during log cleaning.
- PR-8470 - KAFKA-8611 / Refactor KStreamRepartitionIntegrationTest
- PR-8257 - KAFKA-9539; Add leader epoch in StopReplicaRequest (KIP-570)
- PR-8353 - KAFKA-9764: Remove stream simple benchmark suite
- PR-8457 - KAFKA-9842; Add test case for OffsetsForLeaderEpoch grouping in Fetcher
- PR-8077 - KAFKA-9583: use topic-partitions grouped by node to send OffsetsForLeaderEpoch requests
- PR-8474 - KAFKA-9853: Improve performance of Log.fetchOffsetByTimestamp
- PR-7015 - KAFKA-8436: use automated protocol for AddOffsetsToTxn
- PR-8399 - KAFKA-3720: Change TimeoutException to BufferExhaustedException when no memory can be allocated for a record within max.block.ms
- PR-8443 - KAFKA-9832: Extend Streams system tests for EOS version 2
- PR-8436 - KAFKA-6145: KIP-441 avoid unnecessary movement of standbys
- PR-7170 - KIP-221 / Add KStream#repartition operation
- PR-7844 - KAFKA-9309: Add the ability to translate Message classes to and from JSON
- PR-8441 - KAFKA-9748: Extend Streams integration tests for EOS version 2
- PR-8212 - KAFKA-9642: Change “BigDecimal(double)” constructor to “BigDecimal.valueOf(double)”
- PR-8317 - KAFKA-9691: Fix NPE by waiting for reassignment request
- PR-8315 - KAFKA-9433: Use automated protocol for AlterConfigs request and response
- PR-8425 - KAFKA-6145: KIP-441 Move tasks with caught-up destination clients right away
- PR-8451 - KAFKA-9835; Protect FileRecords.slice from concurrent write
- PR-8409 - KAFKA-6145: KIP-441 Pt. 6 Trigger probing rebalances until group is stable
- PR-8338 - KAFKA-8890: KIP-519- Make SSL context/engine configuration extensible
- PR-8405 - KAFKA-9801: Still trigger rebalance when static member joins in CompletingRebalance phase
- PR-8414 - KAFKA-8410: Part 1: processor context bounds
- PR-8423 - KAFKA-9818: improve error message to debug test
- PR-8420 - KAFKA-9815; Ensure consumer always re-joins if JoinGroup fails
- PR-8371 - KAFKA-9753: A few more metrics to add
- PR-8402 - KAFKA-9793: Expand the try-catch for task commit in HandleAssignment
- PR-8383 - KAFKA-9784: Add OffsetFetch to group concurrency test
- PR-8422 - KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator
- PR-8418 - KAFKA-9807; Protect LSO reads from concurrent high-watermark updates
- PR-8408 - KAFKA-9810: Document Connect Root REST API on /
- PR-8412 - KAFKA-9750; Fix race condition with log dir reassign completion
- PR-8411 - KAFKA-9812: fix infinite loop in test code
- PR-8400 - KAFKA-9739: Fixes null key changing child node
- PR-8375 - KAFKA-9776: Downgrade TxnCommit API v3 when broker doesn’t support
- PR-8407 - KAFKA-9809: Shrink transaction timeout for streams
- PR-8359 - KAFKA-9778: Add validateConnector functionality to the EmbeddedConnectCluster
- PR-8337 - KAFKA-6145: Pt. 5 Implement high availability assignment
- PR-8331 - KAFKA-9748: extend EosIntegrationTest for EOS version 2
- PR-7755 - KAFKA-9233: Fix IllegalStateException in Fetcher retrieval of beginni…
- PR-8269 - KAFKA-9659: Add more log4j when updating static member mappings
- PR-8356 - KAFKA-9441: Cleanup Streams metrics for removed task commit latency metrics
- PR-8370 - KAFKA-9753: Add active tasks process ratio
- PR-8389 - KAFKA-9777; Remove txn purgatory to fix race condition on txn completion
- PR-8279 - KAFKA-9706: Handle null keys/values in Flatten transformation
- PR-8367 - KAFKA-9719: Streams with EOS version 2 should fail fast for older brokers
- PR-8388 - KAFKA-9759: Add doc change for KIP-562
- PR-8350 - KAFKA-9760: Add KIP-447 protocol change to upgrade notes
- PR-8382 - KAFKA-9685: PT2, avoid unnecessary set creation in ACL matching
- PR-8164 - KAFKA-9600: EndTxn should enforce strict epoch checking if from client
- PR-8368 - KAFKA-9770: Close underlying state store also when flush throws
- PR-8369 - KAFKA-9771: Port patch for inter-worker Connect SSL from Jetty 9.4.25
- PR-8358 - KAFKA-9756: Process more than one record of one task at a time
- PR-8320 - KAFKA-8470: State change logs should not be in TRACE level
- PR-8280 - KAFKA-9707: Fix InsertField.Key should apply to keys of tombstone records
- PR-8332 - KAFKA-9688: kafka-topic.sh should show KIP-455 adding and removing replicas
- PR-8346 - KAFKA-9373: Reduce shutdown time by avoiding unnecessary loading of indexes
- PR-6741 - Improve performance of checkpointHighWatermarks, patch 1/2
- PR-8138 - KAFKA-9573: Fix JVM options to run early versions of Kafka on the latest JVMs
- PR-8343 - KAFKA-9758: Doc changes for KIP-523 and KIP-527
- PR-8290 - KAFKA-9677: Fix consumer fetch with small consume bandwidth quotas
- PR-8339 - KAFKA-9752; New member timeout can leave group rebalance stuck
- PR-8285 - KAFKA-9700:Fix negative estimatedCompressionRatio issue
- PR-8287 - KAFKA-9711 The authentication failure caused by SSLEngine#beginHandsh…
- PR-8334 - KAFKA-6145: Add balanced assignment algorithm
- PR-8336 - KAFKA-9749: TransactionMarkerRequestCompletionHandler should treat KAFKA_STORAGE
- PR-8327 - KAFKA-9743: Catch commit offset exception to eventually close dirty tasks
- PR-7939 - KAFKA-9407: Return Immutable List from SchemaSourceTask
- PR-8330 - KAFKA-9742: Fix broken StandbyTaskEOSIntegrationTest
- PR-8226 - KAFKA-9651: Fix ArithmeticException (÷ by 0) in Partitioner impls
- PR-8318 - KAFKA-9451: Enable producer per thread for Streams EOS
- PR-8205 - KAFKA-9634: Add note about thread safety of ConfigProvider
- PR-8252 - KAFKA-6145: Pt 2.5 Compute overall task lag per client
- PR-8325 - KAFKA-9741: Update ConsumerGroupMetadata before calling onPartitionsRevoked()
- PR-8324 - KAFKA-9701 (fix): Only check protocol name when generation is valid
- PR-8307 - KAFKA-9727: cleanup the state store for standby task dirty close and check null for changelogs
- PR-8262 - KAFKA-6145: Add constrained balanced assignment algorithm
- PR-8319 - KAFKA-9734: Fix IllegalState in Streams transit to standby
- PR-8155 - KAFKA-9553: Improve measurement for loading groups and transactions
- PR-8244 - KAFKA-8820: kafka-reassign-partitions.sh should support the KIP-455 API
- PR-8223 - KAFKA-9654 ReplicaAlterLogDirsThread can’t be created again if the pr…
- PR-8284 - KAFKA-9225: Bump rocksdb 5.18.3 to 5.18.4
- PR-8218 - KAFKA-9441: Unify committing within TaskManager
- PR-7972 - KAFKA-9435: DescribeLogDirs automated protocol
- PR-7039 - KAFKA-8618: Replace Txn marker with automated protocol
- PR-8299 - KAFKA-9568: enforce rebalance if client endpoint has changed
- PR-8313 - KAFKA-5604: Remove the redundant TODO marker on the Streams side
- PR-8253 - KAFKA-9656: Return COORDINATOR_NOT_AVAILABLE for older producer clients
- PR-8161 - KAFKA-9047: Making AdminClient group operations respect retries and backoff
- PR-7936 - KAFKA-9404: Use ArrayList instead of LinkedList in Sensor Class
- PR-8260 - KAFKA-9625: Fixing IncrementalAlterConfigs with respect to Broker Configs
- PR-8278 - KAFKA-8803: Remove timestamp check in completeTransitionTo
- PR-8289 - KAFKA-9712: Catch and handle exception thrown by reflections scanner
- PR-8236 - KAFKA-9670: Reduce allocations in Metadata Response preparation
- PR-8083 - KIP-546 (1/2): Implements describeClientQuotas() and alterClientQuotas().
- PR-8267 - KAFKA-6647: Do note delete the lock file while holding the lock
- PR-8246 - KAFKA-6145: Pt 2. Include offset sums in subscription
- PR-8293 - KAFKA-9715: Eliminate reference to interBrokerProtocolVersion in TransactionStateManager
- PR-8294 - KAFKA-9718; Don’t log passwords for AlterConfigs in request logs
- PR-7957 - KAFKA-8768: DeleteRecords request/response automated protocol
- PR-8265 - KAFKA-9657: Throw upon offset fetch unsupported stable flag protocol
- PR-8266 - KAFKA-9695; Handle null config values for createTopics, alterConfigs
- PR-8272 - KAFKA-9701: Add more debug log on client to reproduce the issue
- PR-8177 - KAFKA-9605: Do not attempt to abort batches when txn manager is in fatal error
- PR-7876 - KAFKA-9344:Logged consumer config does not always match actual config…
- PR-8232 - KAFKA-9658: Fix removing user quotas
- PR-8255 - KAFKA-9686 MockConsumer#endOffsets should be idempotent
- PR-8235 - KAFKA-9176: Do not update limit offset if we are in RESTORE_ACTIVE mode
- PR-283 - KAFKA-2391: add timeout to KafkaConsumer blocking calls
- PR-282 - KAFKA-2428
- PR-8220 - KAFKA-9645: Fallback to unsubscribe during Task Migrated
- PR-8228 - KAFKA-9663: Doc some null returns in KafkaStreams
- PR-284 - KAFKA-2620: Introduce Scalariform
- PR-8233 - KAFKA-9668: Iterating over KafkaStreams.getAllMetadata() results in ConcurrentModificationException
- PR-279 - KAFKA-2479: Transient exception
- PR-8227 - KAFKA-9662: Wait for consumer offset reset in throttle test to avoid losing early messages
- PR-8121 - KAFKA-6145: Pt 1. Bump protocol version and encode task lag map
- PR-8213 - KAFKA-9615: Clean up task/producer create and close
- PR-8229 - KAFKA-9661: Propagate includeSynonyms option to AdminClient in ConfigCommand
- PR-254 - KAFKA-2588 ReplicaManager partitionCount metric should actually be re…
- PR-277 - Kafka-2587: Only notification handler will update the cache and all verifications will use waitUntilTrue.
- PR-8209 - KAFKA-9632; Fix MockScheduler synchronization for safe use in Log/Partition tests
- PR-8186 - KAFKA-9618: Directory deletion failure leading to error task RocksDB open
- PR-275 - KAFKA-2527; System Test for Quotas in Ducktape
- PR-8208 - KAFKA-8895: delete all topics before recreating
- PR-8197 - KAFKA-9573: Fix VerifiableProducer and VerifiableConsumer to work with older Kafka versions
- PR-1286 - KAFKA-3618: Handle ApiVersionsRequest before SASL authentication
- PR-8087 - KAFKA-9525: add enforceRebalance method to Consumer API
- PR-8190 - KAFKA-9623: Keep polling until the task manager is no longer rebalancing in progress
- PR-8187 - KAFKA-9620: Do not throw in the middle of consumer user callbacks
- PR-273 - KAFKA-2517; Performance Regression post SSL implementation (zero copy)
- PR-8168 - KAFKA-9607: Do not clear partition queues during close
- PR-8169 - KAFKA-9610: Do not throw illegal state when remaining partitions are not empty
- PR-8173 - KAFKA-9614: Not initialize topology twice in StreamTask
- PR-8165 - KAFKA-9601: Stop logging raw connector config values
- PR-8166 - KAFKA-9602: Close the stream internal producer only in EOS
- PR-8062 - KAFKA-9498; Topic validation during the topic creation triggers unnecessary TopicChange events
- PR-8029 - KAFKA-8147: Add changelog topic configuration to KTable suppress
- PR-8167 - KAFKA-9533: Revert ValueTransform forwards null
- PR-8009 - KAFKA-9308: Reworded the ssl part of the security documentation
- PR-8159 - KAFKA-9599 create unique sensor to record group rebalance
- PR-8145 - KAFKA-9581: Remove rebalance exception withholding
- PR-8142 - KAFKA-9577: SaslClientAuthenticator incorrectly negotiates SASL_HANDSHAKE version
- PR-8105 - KAFKA-9441: Add internal StreamsProducer
- PR-263 - KAFKA 2578 Client Metadata internal state should be synchronized
- PR-8143 - KAFKA-9582: Do not abort transaction in unclean close
- PR-8111 - KAFKA-9206: throw a KafkaException when encountering CORRUPT_MESSAGE
- PR-8058 - KAFKA-9481: Graceful handling TaskMigrated and TaskCorrupted
- PR-8116 - KAFKA-9562: part 1: ignore exceptions while flushing stores in close(dirty)
- PR-7337 - KAFKA-8910: Incorrect javadoc at KafkaProducer.InterceptorCallback#onCompletion
- PR-8108 - KAFKA-9533: ValueTransform forwards null values
- PR-7913 - KAFKA-9343: Add ps command for Kafka and zookeeper process on z/OS.
- PR-8119 - KAFKA-9558: Fixing retry logic for getListOffsetsCalls
- PR-8089 - KAFKA-6266: Repeated occurrence of WARN Resetting first dirty offset
- PR-8125 - KAFKA-9515: Upgrade ZooKeeper to 3.5.7
- PR-8106 - KAFKA-9319: Fix generation of CA certificate for system tests.
- PR-8109 - KAFKA-9545: Fix subscription bugs from Stream refactoring
- PR-8088 - KAFKA-9535: Update metadata upon retrying partitions for ListOffset
- PR-7389 - KAFKA-8805: Bump producer epoch on recoverable errors
- PR-8060 - KAFKA-9274: Gracefully handle timeout exception
- PR-7781 - KAFKA-8904: Improve producer’s topic metadata fetching.
- PR-8085 - KAFKA-9556: Fix two issues with KIP-558 and expand testing coverage
- PR-8112 - KAFKA-9557: correct thread process-rate sensor to measure throughput
- PR-8023 - KAFKA-8507 kip 499 Unify connection name flag for command line tool
- PR-7674 - KAFKA-9106 make metrics exposed via jmx configurable
- PR-7731 - KAFKA-9204: allow ReplaceField SMT to handle tombstone records
- PR-7733 - KAFKA-9192: fix NPE when for converting optional json schema in structs
- PR-8000 - KAFKA-9417: New Integration Test for KIP-447
- PR-8031 - KAFKA-9447: Add new customized EOS model example
- PR-8091 - KAFKA-6607: Commit correct offsets for transactional input data
- PR-8053 - KAFKA-9499; Improve deletion process by batching more aggressively
- PR-7955 - KAFKA-9423: Refine layout of configuration options on website and make individual settings directly linkable
- PR-8015 - KAFKA-9500: Fix topology bug in foreign key joins
- PR-8065 - KAKFA-9503: Fix TopologyTestDriver output order
- PR-7996 - KAFKA-9355: Fix bug that removed RocksDB metrics after failure in EOS
- PR-8054 - KAFKA-9390: Make serde pseudo-topics unique
- PR-8024 - KAFKA-9483; Add Scala KStream#toTable to the Streams DSL
- PR-8040 - KAFKA-6607: Commit correct offsets for transactional input data
- PR-8059 - KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent fields
- PR-8061 - KAFKA-9517: Fix default serdes with FK join
- PR-8033 - KAFKA-9487: Follow-up PR of Kafka-9445
- PR-8081 - KAFKA-9523: Migrate BranchedMultiLevelRepartitionConnectedTopologyTest into a unit test
- PR-8018 - KAFKA-9480: Fix bug that prevented to measure task-level process-rate
- PR-8039 - KAFKA-9505: Only loop over topics-to-validate in retries
- PR-8027 - KAFKA-7658: Follow up to original PR
- PR-8057 - KAFKA-9507 AdminClient should check for missing committed offsets
- PR-8056 - KAFKA-9519: Deprecating ZK for ConfigCommand
- PR-8048 - KAFKA-9509: Fixing flakiness of MirrorConnectorsIntegrationTest.testReplication
- PR-252 - KAFKA-2586; Enable SSL for inter-broker communication when SSL is enabled in tests
- PR-255 - KAFKA-2593 Key value stores can use specified serializers and deserializers
- PR-7770 - KAFKA-9261; Client should handle unavailable leader metadata
- PR-7997 - KAFKA-9113: Clean up task management and state management
- PR-8037 - KAFKA-9491; Increment high watermark after full log truncation
- PR-8007 - KAFKA-9477 Document RoundRobinAssignor as an option for partition.assignment.strategy
- PR-7568 - KAFKA-9074: Correct Connect’s Values.parseString to properly parse a time and timestamp literal
- PR-8030 - KAFKA-9492; Ignore record errors in ProduceResponse for older versions
- PR-8028 - KAFKA-9490: Fix generics for Grouped
- PR-8026 - Bump trunk to 2.6.0-SNAPSHOT
- PR-7725 - KAFKA-9027, KAFKA-9028: Convert create/delete acls requests/response to use generated protocol
ksqlDB¶
- PR-6126 - fix: create /var/lib/kafka-streams directory on RPM installation
- PR-6165 - fix: allow expressions in flat map (6.0.x)
- PR-5952 - METRICS-1988: add confluent-telemetry directory to ksql-run-class
- PR-5937 - fix: avoid losing cause of processing errors
- PR-5895 - fix: adds a handler to gracefully shutdown
- PR-5885 - fix: close query on invalid use of HTTP/2 with /query endpoint (6.0.x)
- PR-5871 - fix: remove schema compat check if schema exists
- PR-5844 - fix: Reuse KsqlClient instance for inter node requests (#5742)
- PR-5837 - fix: always use the changelog subject in table state stores
- PR-5786 - fix: better error on DECIMAL type with PROTOBUF format
- PR-5818 - fix: Allow backported ALPN to be used and warn if no OpenSSL
- PR-5783 - fix: SEC-1034: log4j migration to confluent repackaged version
- PR-5788 - fix: circumvent KAFKA-10179 by forcing changelog topics for tables
- PR-5763 - fix: ksqlDB should not truncate decimals
- PR-5761 - support UDTFs on streams without keys
- PR-5756 - fix: do not overwrite schemas from a CREATE STREAM/TABLE
- PR-5730 - fix: make sure old query stream doesn’t block on close
- PR-5684 - fix: join queries with SELECT * and WHERE fail
- PR-5703 - fix: support CASE statements returning NULL
- PR-5656 - fix: allow empty structs in schema inference
- PR-5645 - fix: ensure only deserializable cmds are written to command topic
- PR-5644 - fix: support GROUP BY with no source columns used
- PR-5638 - feat: implements ARRAY_JOIN as requested in (#5028) (#5474)
- PR-5617 - fix: don’t fail if broker does not support AuthorizedOperations
- PR-5621 - fix: /inserts-stream endpoint now supports nested types
- PR-5563 - feat: new split_to_map udf
- PR-5559 - feat: add CHR UDF
- PR-5597 - fix: fail AVRO/Protobuf/JSON Schema statements if SR is missing
- PR-5591 - fix: make stream and column names case-insensitive in /inserts-stream
- PR-5592 - fix: disallow requests to /inserts-stream if insert values disabled
- PR-5576 - fix: Stream with no key should not fail if key has data
- PR-5568 - fail aggregations that don’t select any value columns
- PR-5516 - feat: reload TLS certificate without restarting server
- PR-5560 - fix: set retention.ms to -1 instead of Long.MAX_VALUE
- PR-5548 - feat: new UDFs for set-like operations on Arrays
- PR-5552 - fix: improve print topic format detection
- PR-5528 - feat: create MetricsContext for ksql metrics reporters
- PR-5534 - feat: new CONCAT_WS UDF
- PR-5531 - fix: inserts now support zero-valued decimals
- PR-5536 - feat: new UDFs for working with Maps
- PR-5546 - feat: new string UDFs LPad, RPad
- PR-5523 - feat: Add encode udf
- PR-5542 - feat: support TIMESTAMP being a key column
- PR-5535 - feat: new UUID UDF
- PR-5532 - fix: Prevent memory leaks caused by pull query logging
- PR-5533 - feat: keys can now be named using the KEY syntax
- PR-5529 - fix: deals with issue #5521 by adding more descriptive error message
- PR-5513 - fix: update Concat UDF to new framework and make variadic
- PR-5504 - feat: Adds UDF regexp_replace
- PR-5501 - feat: Adds udf regexp_split_to_array
- PR-5495 - feat: turn on snappy compression for produced data
- PR-5515 - fix: Retry on connection closed
- PR-5505 - feat: new UDFs for array max/min/sort
- PR-5507 - feat: Adds UDF regexp_extract_all
- PR-4909 - feat: add extra log messages for pull queries
- PR-5506 - fix: allow dynamic construction of an ARRAY of STRUCTS with duplicate values #5436
- PR-5461 - fix: KSQL does not accept queries when running QueryLimit - 1 queries
- PR-5497 - build: reference plugin URL as property (MINOR)
- PR-5478 - fix: Remove stacktrace from error message
- PR-5435 - fix(client): fail queryID future if executeQuery() request fails (MINOR)
- PR-5469 - fix: /inserts-stream endpoint now accepts complex types
- PR-5477 - fix: Improve pull query error logging
- PR-5470 - ST-1323: Add license file so sources get attached
- PR-5459 - fix: avoid unnecessary warnings from PRINT
- PR-5448 - feat(client): support (non-streaming) insert into in Java client
- PR-5455 - fix: allow setting auto.offset.reset=latest on /query-stream endpoint
- PR-5425 - fix: exclude window bounds from persistent query value & schema match
- PR-5431 - fix: fail on WINDOW clause without matching GROUP BY
- PR-5340 - feat: Expose Vert.x metrics
- PR-5444 - Fix markdown for function references
- PR-5385 - feat: INSTR function #881
- PR-5412 - feat: introduce RegexClassifier for classifying errors via cfg
- PR-5392 - feat: Pull Queries: QPS check utilizes internal API flag to determine if forwarded
- PR-5355 - How-to guides: first set
- PR-5391 - fix(client): expose query ID from executeQuery() return type
- PR-5386 - fix: Block writer thread if response output buffer is full
- PR-5374 - feat: expose JMX metric that classifies an error state
- PR-5363 - feat: drop WITH(KEY) syntax
- PR-5371 - feat(client): support push query termination in Java client
- PR-5382 - MINOR: Use kafka.scala.version variable in pom file
- PR-5380 - MINOR: Update mbknor-jackson-jsonschema_2.12 artifact to mbknor-jackson-jsonschema_2.13
- PR-5381 - fix: Override Scala version in our build
- PR-5360 - fix(client): Java client error handling
- PR-5270 - fix: remove leading zeros when casting decimal to string
- PR-5273 - feat: implement earliest_by_offset() UDAF
- PR-4975 - fix: fix NPE in latest_by_offset if first element being processed has null…
- PR-4782 - feat: Implement latest_by_offset() UDAF
- PR-5212 - feat: Adds the ability have internal endpoints listen on ksql.internal.listener
- PR-5332 - fix: Allow value delimiter to be specified for datagen
- PR-5093 - Any key name
- PR-5200 - feat(client): Java client with push + pull query support
- PR-5287 - fix: allow structs in schema provider return types
- PR-5205 - fix: /query-stream endpoint should serialize Struct (MINOR)
- PR-5239 - fix: Move Cors handler in front of /chc handlers
- PR-5124 - Backport array len
- PR-5213 - fix: use sr’s jackson-jsonschema version
- PR-5158 - feat: Remove dependency to ksqldb-common
- PR-5198 - fix: use schema in annotation as schema provider if present
- PR-5193 - feat: Do not allow access to new streaming endpoints using HTTP1.x
- PR-5189 - build: pin streams to 5.5.0 instead of 5.5.0-SNAPSHOT
- PR-5162 - bug: stop multi-way joins failing on ambiguous column errors
- PR-5150 - feat: klip-14 - rowtime as pseduocolumn
- PR-5145 - fix: do not allow implicit casting in UDAF function lookup
- PR-5122 - Event-driven microservice tutorial
- PR-5104 - feat: fail startup if command contains incompatible version
- PR-5105 - feat: transient queries added to show queries output
- PR-5137 - BREAKING CHANGE: minor: allow more precise configuration of rocks bounds
- PR-5142 - fix: support quotes in explain statements (MINOR)
- PR-5126 - feat: Implement cluster health check endpoints
- PR-5123 - refactor: Migrate old websocket query endpoint
- PR-5139 - fix: include lower-case identifiers among those that need quotes (#3723)
- PR-5081 - feat: add multi-join expression support
- PR-5075 - fix: replace ‘null’ in explain plan with correct op type (MINOR)
- PR-5023 - fix: Improve error message for where/having type errors
- PR-5064 - feat: support multi-way joins
- PR-5054 - fix: add content type header to ported endpoints response (MINOR)
- PR-5069 - fix: make endpoints available while waiting for precondition
- PR-5048 - fix: reject requests to new API server if server is not ready
- PR-5047 - fix: make endpoints available while waiting for precondition
- PR-5059 - fix: Make sure internal client is configured for TLS
- PR-4985 - fix: Don’t set chunked transfer encoding for HTTP2 proxy
- PR-4974 - fix: Catch server startup exceptions
- PR-4945 - feat: Support wildcards for unauthed paths
- PR-4944 - feat: CORS support for Vert.x based ksqlDB
- PR-4851 - feat: Migrate /ksql and /ksql/terminate endpoints to Vert.x
- PR-5019 - fix: improve handling of NULLs
- PR-5052 - fix: json creator for QueryStatusCount should match getter (MINOR)
- PR-5029 - feat: add KsqlQueryStatus to decouple from KafkaStreams.State
- PR-4921 - Two new tutorials: materialized views + streaming ETL pipelines
- PR-5013 - feat: support more advanced suite of LIKE expressions
- PR-4952 - Support aliases in PARTITION BY and GROUP BY
- PR-5002 - feat: speed up restarts by not building topologies for terminated queries
- PR-4986 - Distinguish keys for tables by naming them PRIMARY KEY
- PR-4875 - feat: scatter gather query status from all servers in cluster for ‘SHOW QUERIES [EXTENDED]’ statement
- PR-4993 - fix: Allows unclosed quote characters to exist in comments
- PR-5012 - fix: allow trailing slash in listeners config (MINOR)
- PR-5005 - fix: query id for TERMINATE should be case insensitive
- PR-4977 - fix: generated aliases for struct field access no longer require quoting
- PR-5006 - Remove unused / duplicate methods (MINOR)
- PR-4951 - feat: Adds rate limiting to pull queries
- PR-4972 - fix: When pull query metrics enabled, throws NPE
- PR-4956 - fix: output valid multiline queries when running SHOW QUERIES
- PR-4966 - fix: Pass correct request configs when making pull query request
- PR-4942 - fix: do not allow grouping sets
- PR-4926 - fix: throw error when column does not exist during INSERT INTO
- PR-4940 - fix: do not allow GROUP BY and PARTITION BY on boolean expressions
- PR-4908 - fix: Don’t wait for streams thread to be in running state
- PR-4878 - fix: handle leap days correctly during timestamp extraction
- PR-4893 - fix: Infer TLS based on scheme of server string
- PR-4889 - fix: If startup hangs (esp on preconditions), shutdown server correctly
- PR-4872 - fix: ‘drop (stream|table) if exists’ fails if source does not exist
- PR-4827 - fix: avoid duplicate column name errors from auto-generated aliases
- PR-4822 - fix: Stop PARTITION BY and UDTFs that fail from terminating the query
- PR-4850 - feat: expand IFNULL to support all types
- PR-4859 - fix: Filter out hosts with no lag info by default
- PR-4868 - test: add test coverage to ensure that historical commands from previous versions always work in the current version
- PR-4837 - fix: Returns empty lag info for a dead host rather than last received lags
- PR-4867 - fix: avoid long possible format lists on PRINT TOPIC for nulls
- PR-4829 - feat: add COALESCE function
- PR-4653 - fix: CommandRunner metric has correct metric displayed when thread dies
- PR-4823 - fix: do not filter out rows where PARTITION BY resolves to null
- PR-4814 - fix: use describeTopics() for Kafka healtcheck probe
- PR-4771 - feat: add internal request to KsqlRequestConfig and SessionProperties
- PR-4816 - fix: do not lose old key values on repartition
- PR-4830 - fix: add deserializer for SqlType
- PR-4809 - fix: Removes unnecessary error logging for heartbeat since this is ex…
- PR-4754 - fix: better error code on shutdown
- PR-4805 - fix: only create processing log stream if it doesn’t exist
- PR-4743 - fix: make CLI unit-tests platform independent
- PR-4785 - fix: docker image directories in light of renames (MINOR)
- PR-4608 - feat: Add metrics for pull queries endpoint
- PR-4778 - feat: Security plumbing
- PR-4777 - fix: add transfer-encoding:chunked header if Jetty omits this
- PR-4733 - feat: support for tunable retention, grace period for windowed tables
- PR-4728 - feat: add REGEXP_EXTRACT UDF
- PR-4717 - fix: create schemas at topic creation
- PR-4709 - fix: stop logging about command topic creation on startup if exists (MINOR)
- PR-4725 - feat: add ARRAY_LENGTH UDF
- PR-4713 - feat: Integrate new API with existing security framework
- PR-4714 - test: avoid NPE if user puts input topic name in output
- PR-4685 - feat: basic auth for new API server
- PR-4681 - fix: fix the error message for failing to wait for CommandRunner to catch up in DistributingExecutor
- PR-4632 - fix: change default exception handling for timestamp extractors to log-and-continue
- PR-4675 - fix: change configOverrides back to streamsProperties
- PR-4597 - fix: added special handling for forwarded pull query request
- PR-4650 - fix: Return immutable list of streams metadata
- PR-4643 - fix: idempotent terminate that can handle hung streams
- PR-4489 - fix: csas/ctas with timestamp column is used for output rowtime
- PR-4625 - fix: support partial schemas
- PR-4509 - feat: display errors in CLI in red text
- PR-4621 - fix: patch KafkaStreamsInternalTopicsAccessor as KS internals changed
- PR-4606 - fix: StoreQueryParameters name change
- PR-4573 - feat: add the topic name to deserialization errors
- PR-4575 - feat: log groupby errors to processing logger
- PR-4590 - feat: Simple perf tool for new API
- PR-4581 - feat: Provide upper limit on number of push queries
- PR-4511 - [SECURITY] Use HTTPS to resolve dependencies in Maven Build
- PR-4512 - feat: Plug insert streams into backend
- PR-4551 - feat: enhance PRINT TOPIC’s format detection
REST Proxy¶
- PR-738 - KC-613: Remove share/java/kafka-rest dependencySet.
- PR-737 - KC-613: Split share/java/kafka-rest in kafka-rest-bin and kafka-rest-lib.
- PR-734 - CLIENTS-1659: Remove V1 API Kafka core dependency and associated Zookeeper Client configs
- PR-731 - CLIENTS-1657: Explicitly depend on Guava.
- PR-729 - METRICS-1988: add confluent-telemetry directory to kafka-rest-run-class
- PR-723 - Refactor requests to fix go sdk generation bug and add parameterized servers to openapi-spec
- PR-726 - Add [Consumer|Producer]Performance tools back
- PR-722 - Add reassignment API to openapi spec.
- PR-721 - Add ACL APIs to openapi spec.
- PR-719 - Add consumers (admin) APIs to openapi spec.
- PR-718 - CNKAF-1017: Update v3 error response format
- PR-715 - Add config endpoints to openapi spec.
- PR-714 - OpenAPI Spec for a small subset of the V3 APIs.
- PR-712 - Move MetricsReporter Config extraction to rest-utils
- PR-711 - CLIENTS-1613: Add configuration to enable/disable V2 and V3 APIs.
- PR-710 - Remove cluster-wide topic configs.
- PR-709 - KC-613: Use slf4j version from common pom.xml.
- PR-707 - Change resource label resource.kafka.cluster.id to resource.cluster.id
- PR-703 - Enable setting cluster_id on KafkaRestMetricsContext
- PR-706 - CNKAF-951: Add catchingCompose utility to handle exceptions.
- PR-704 - CLIENTS-1599: Add constructor accepting path to KafkaRestApplication.
- PR-697 - CLIENTS-1597: Add relationship reassignment and refactor reassignment resource
- PR-701 - Rename alter config UPDATE to SET.
- PR-700 - Add Batch Alter (UPDATE/DELETE) Broker/Cluster/Topic configs.
- PR-696 - Add LIST/GET Consumer Group/Consumer/Consumer Assignment.
- PR-693 - Add search/create/delete ACLs APIs.
- PR-687 - CNKAF-739: Reassignment endpoint for a given topic name
- PR-685 - CNKAF-739 - Reassignments endpoint for given topic name and partition id
- PR-692 - Remove AdminClientWrapper.
- PR-691 - Migrate off JSON:API cleanup.
- PR-682 - CNKAF-739: Add Rest endpoint for list all partition reassignments
- PR-690 - Switch from application/vnd.api+json to application/json.
- PR-689 - KIP-606 MetricsContext
- PR-686 - Migrate off JSON:API.
- PR-683 - KIP-226: Add cluster-wide dynamic default configs.
- PR-678 - ST-1323: Add license file so sources get attached
- PR-679 - Migrate internal entities to Google’s Auto-Value.
- PR-680 - Add static factory to Cluster.
- PR-677 - KIP-226: Add support for configs source and synonyms.
- PR-675 - Give more unique names to the testing containers.
- PR-674 - Open visibility of configurePre/PostResourceHandling.
- PR-672 - Fix REST Proxy injection-based authentication.
- PR-671 - Make ParititonData public
- PR-670 - Add JsonCreator to v3 entities
- PR-667 - Copy admin client config from REST Proxy. Need for authentication tests.
- PR-658 - Remove all methods from AdminClientWrapper.
- PR-665 - CPKAFKA-4979: Remove non-determinism from searchReplicasByBroker.
- PR-662 - Add find partition replicas by broker id endpoint.
- PR-660 - CLIENTS-1505: Add broker config delete endpoint
- PR-659 - CLIENTS-1504: Add update endpoint for broker config
- PR-654 - Remove AdminClientWrapper.getBrokerIds() and add JSON:API errors.
- PR-656 - Fix default Content-Type, when none or any (/) is provided.
- PR-655 - Remove spurious topic/partitions existence test when producing.
- PR-647 - Remove V1 APIs and all their dependencies.
- PR-652 - CLIENTS-1501: Add broker configs endpoints
- PR-617 - Fix compile error due to KIP-515 (ZK TLS)
- PR-191 - SEC-1034: log4j migration to confluent repackaged version
- PR-199 - Rest CORS option skip will now default to true
- PR-195 - Add a config flag to skip OPTION when CORS is enabled
- PR-193 - Removed OPTION omission for CORS
- PR-192 - All original configs must be passed to MetricReporters
- PR-190 - Use prefix to propagate metrics reporter configs
- PR-186 - Extensible Application Metrics instance configuration
- PR-180 - KIP-606 MetricsContext
- PR-183 - ST-1323: Add license file so sources get attached
- PR-182 - Add getter for Application’s Metrics object
- PR-178 - Migrate rest-utils to use AK library instead of common-metrics
- PR-177 - SEC-929-Customized HTTP response headers.
- PR-172 - SEC-592: Customize Jetty request queue and threadpool size
- PR-174 - Catch invalid format exception: #501
Schema Registry¶
- PR-1624 - DG-857 Fix Avro converter for top-level bytes object
- PR-1616 - DG-845 Have typeName override Object.class type; fixes #1615
- PR-1611 - Correct License in Client library
- PR-1598 - CP-644: Include CP jars required for Security and monitoring
- PR-1601 - DG-820: Fix telemetry reporter handling of leadership changes
- PR-1590 - DG-810: Explicitly add kafkastore.topic config to MetricsReporter instances
- PR-1594 - DG-813 If use.latest.version is true, use on Protobuf dependencies
- PR-1588 - DG-811 Allow JSON schema spec version to be specified
- PR-1589 - DG-812 Derive file extension during download
- PR-1587 - Upgrade to Kotlin 1.4.0
- PR-1586 - DG-800 Cache ProtobufSchema using name in Converter
- PR-1528 - SEC-1034: log4j migration to confluent repackaged version
- PR-1580 - DG-789 Fix for CVE-2020-15824
- PR-1579 - DG-788 Allow anyOf to be backward compat with oneOf/allOf
- PR-1578 - DG-767 Calculate proper defaults for client ssl configs
- PR-1576 - DG-767 Use passed in defaults for client ssl configs
- PR-1577 - DG-774 restore ssl client configs
- PR-1575 - DG-780 Add support for number with format for JSON Schema
- PR-1569 - METRICS-1988: add confluent-telemetry directory to schema-registry-run-class
- PR-1574 - DG-774 Avoid regenerating default names for Protobuf data
- PR-1567 - DG-740 Convert connect defaults for JSON Schema
- PR-1563 - DG-713: Disable automatic setup of telemetry reporting
- PR-1561 - DG-723 Enable defaults for JSON Schema
- PR-1556 - DG-715: Change telemetry cluster label names
- PR-1550 - DG-695 Normalize JSON ref schemas before comparing
- PR-1548 - ST-3988: update swagger-core to 1.6.2
- PR-1547 - ST-3988: update swagger-core to 1.6.2
- PR-1538 - DG-670 Treat anyOf same as oneOf in JSON Schema converter
- PR-1537 - DG-652 Fix missing negation in conditional
- PR-1535 - DG-652 Handle imports during protobuf compat checks
- PR-1531 - DG-628 Allow changes to id during json schema compat checks
- PR-1530 - DG-607 Add lenient processing for numbers in oneof
- PR-1523 - DG-607 Ensure fields for union are optional
- PR-1520 - Handle null error stream in RestService
- PR-1522 - DG-606 Add option to not generate oneof for nullable fields
- PR-1517 - Upgrade maven-plugin dependencies to 3.6 in 5.4.x
- PR-1510 - Add metric name test
- PR-1509 - align naming conventions
- PR-1508 - Add telemetry reporter SR config option
- PR-1507 - Use config originals for retrieving TR enablement
- PR-1493 - Use MetricsContext to set JMX prefix
- PR-1502 - Split telemetry cluster label
- PR-1500 - Only add TelemetryReporter if explicitly enabled in config
- PR-1483 - Telemetry reporter integration
- PR-1497 - DG-554 Pass refs when forwarding register requests
- PR-1496 - DG-535 Handle recursive Protobuf schemas
- PR-1495 - DG-402 Missed some places where dotted names can occur
- PR-1492 - Fix NPE during tombstone metric update
- PR-1485 - Add node information to SchemaRegistryProtocol
- PR-1487 - DG-527 Omit defaults for proto 3 schemas (fixed in wire)
- PR-1482 - DG-499 Fix JSON Schema compat checks for partial models
- PR-1484 - DG-524 Catch NPE during Avro compat check
- PR-1480 - DG-27 Use leader/follower terminology
- PR-1477 - Allow custom update handlers; fixes #382
- PR-1476 - Add unit test for mvn validate action
- PR-1468 - Implement new Schema Registry Metrics
- PR-1474 - Add validate action for SR maven plugin
- PR-1473 - Add ability to use latest version during serialization
- PR-1471 - DG-436 Pass existing Avro schema instead of regenerating
- PR-1470 - Allow unknown properties while deserializing SchemaRegistryProtocol.Assignment
- PR-1469 - Fix message name on schema when version is included; fixes #1467
- PR-1466 - DG-450 Perform proper matching of subschemas
- PR-1461 - DG-268 Deprecate ZK-based coordination
- PR-1455 - Fix Scala 2.13 deprecations
- PR-1450 - DG-400 Support enums using allOf in JSONSchema converter
- PR-1451 - Upgrade swagger deps
- PR-1449 - DG-402 Add support for packages in ProtobufConverter
- PR-1440 - Fix null schema type
- PR-1433 - use property for jackson-jsonschema
- PR-1436 - feat: adds subjectName in the exception for incompatible schema
- PR-1437 - Fix cyclic schema references
- PR-1431 - Cherry-pick for permanent delete
- PR-1426 - DG-377 Allow generated classes to use refs during serialization
- PR-1427 - DG-399 Use envelope in JsonSchemaUtils.toObject
- PR-1417 - Allow id to be passed to console prod/cons; fixes #160
- PR-1422 - Fix cycles in Avro schemaEquals
- PR-1420 - SEC-929–Added one testing for verifying a feature added to rest-utils repo.
- PR-1416 - DG-399 Use envelope in JsonSchemaUtils.toObject
- PR-1397 - DG-91 Schema permanent delete.
- PR-1389 - DG-91 Ability to permanently delete schemas
- PR-1369 - DG-267 - Fix Protobuf converter enum handling
- PR-1368 - Performance: Use singleton ObjectMapper
- PR-1344 - Enhanced serialization exception
- PR-1334 - DG-237: Migrate SchemaRegistry CP KafkaMetrics to AK KafkaMetrics
- PR-1326 - Add API to retrieve supported schema types
Commercial Features¶
Control Center¶
- Introduced self-balancing cluster options on the UI, enabling users to easily balance workloads and dynamically expand and shrink clusters.
- Introduced Tiered Storage options on the UI, giving users the option to add and scale storage.
- Enabled auto-updates to the user interface by default.
Confluent Server¶
- CPKAFKA-6013: Skip tier state presence validation for deleted partitions
- KSTORAGE-1082: tier replication causes incorrect HWM after restore (#2507)
- KSTORAGE-1086: Better validations to check if tiering is disabled
- [CNKAF-1144] Confluent.operator.managed property (#2484)
- [Backport]METRICS-2089 Adding the correct NOTICE / COPYRIGHT for the confluent-telemetry package
- KSTORAGE-1040: Better checks to prevent tiering from being disabled dynamically
- METRICS-2159: use a loading cache to avoid expensive pattern matching calls
- CPKAFKA-5882. Add copyDependantLibs dependency to ce-sbk and ce-broker-plugins module
- KC-845: Ensure LogConfig includes broker defaults to pass schema config validation
- CPKAFKA-5873 Don’t load Netty classes when using default SSL engine
- CONFLUENT: Remove shading from ce-broker-plugins and ce-sbk
- CPKAFKA-5851: Align joda-time version with common
- CNKAF-863: Only showed broker removal status for given broker.
- KC-613: Use kafka-rest-lib instead of kafka-rest on classpath.
- CPKAFKA-5849: Don’t log password configs in request log filter
- CONFLUENT: Remove unused jacksonDatabind version definition
- CPKAFKA-5795: Change guava dependency to 28.1-jre
- CPKAFKA-5796: Remove ZK dependency from ce-licensing
- CPKAFKA-5835: Fix regression in cluster link compatibility test
- METRICS-2108 Telemetry-client dependency version update from 1.174.0 to 1.192.0 to update default base URL for TelemetryHttpClient
- [CNKAF-1089][RM-32] Remove SBK dependencies on grpc-stub. (#2408)
- backport METRICS-2098 exclude zstd/lz4 from telemetry-clients to avoid conflicting versions
- METRICS-2090: Remove duplicate labels on metrics
- METRICS-2097: Don’t capture ‘total_size` and ‘tier_size` metrics
- KC-637 cherry pick to 6.0.x
- KSTORAGE-957: Only set SSEAwsKeyManagementParams if a customer encryption key is supplied and sseAlgorithm is aws:kms. (#2353)
- SEC-866-6.0.x again
- CONFLUENT: Remove the soak clients status reporter cronjob
- METRICS-2031 Prune the hardcoded Kafka broker default include list to avoid collecting high cardinality metrics.
- METRICS-2048 Sanitize ksql service ids from their metric name to make them more practical to use. Also ignore legacy metrics.
- [CNKAF-1038] SBK Rebalance Cluster Test: add rolling bounces (#2326)
- CNKAF-1058; Ensure that Control Request Batching does not send LeaderAndIsrRequest with yet unknown broker(s)
- KSTORAGE-947: batch tier segment deletion delays to improve deletion SLA (#2309)
- backport METRICS-2010: create a new ‘releaseTarGz’ task for ce-metrics (#2282)
- CPKAFKA-5762: Fix topic deletion in system tests
- CNKAF-1039: Update error for insufficient metrics to compute plan
- KC-671: SBK system test with CL
- METRICS-1988: add confluent-telemetry directory to kafka classpath
- CONFLUENT: Bump cc-trogdor chart docker image to cc-trogdor:v0.207.0-5.3.0-ce-SNAPSHOT
- [Cherry 2367] [SEC-1345] Read HTTP Reason phrase when connection closed
- Revert “SEC-866 - Migrate audit log “other” category into “management” category. (#2330)”
- backport METRICS-2037 remove non-relocated dependencies from confluent-metrics fat jar (#2332)
- [CLIENTS-1633] Schedule MDS url refresher asynchronously
- Increase root FS to 20GB
- METRICS-1763, METRICS-1767 Configure telemetry metrics whitelist for Connect and MQTT Proxy Proactive Support (#2241)
- SEC-866 - Migrate audit log “other” category into “management” category.
- KC-391: Source topic deletion system test for CL
- KAFKA-10282; Remove Log metrics immediately when deleting log (#2342)
- CNKAF-1036: Fix issue where dynamic throttle applies for the current rebalance only - cherry pick to 6.0
- Rebalance Cluster System Test
- CNKAF-1008: Add Broker/Remove Broker proposal logging
- KC-633: Auth failures system test for Cluster Linking
- [CHERRY-2308] Move kafka-http-server project into clients project
- KC-638: System tests with tier storage enabled on destination
- KC-754: Cluster link compatibility system tests
- KC-631 Good path cluster linking test using SASL_SSL
- backport METRICS-1992: Fix schema registry cluster id resource label (#2285)
- CONFLUENT: Fix broken cluster link system tests in 6.0.x
- CNKAF-1029: Ensure Executor state is reset after unexpected exceptions
- KC-735: Improve cluster link addition.
- CNKAF-1035: Roll cluster while a new broker is getting added
- METRICS-1998: add rbac metrics to default metrics list (#2269)
- backport METRICS-1993: add connect.cluster.id label (#2264)
- Updated default topic replicas factor to 3 for audit log.
- KC-741: Don’t include passwords in cluster link request logs
- KC-736: Allow deletion of cluster links if mirrored topics are marked for deletion
- KSTORAGE-896: fix tier deletion manager system test (#2256)
- CNKAF-822: Cherry pick add broker system test to 6.0.x
- KC-725: Backport of KC-725 to 6.0.x
- backport METRICS-1986 Fix to give higher precedence to “confluent.telemetry.metrics.collector.whitelist” over default provider whitelist. (#2266)
- backport METRICS-1985 Add missing metrics to the _local exporter for SBK feature to work as expected.
- KSTORAGE-920: TierTopicConsumer primary consumer long poll throttles catchup (#2253)
- KSTORAGE-864 Fetch requests to return NotLeaderForPartition during ULE recovery
- CNKAF-819: Test SBK broker removal with roll
- METRICS-1762 Configure telemetry metrics whitelist for KSQL Proactive Support
- Backporting fix for METRICS-1764 and METRICS-1766
- update kafkatest template for balancer replication factor (for < 3-node clusters)
- KC-638: Cluster linking system test with tiered storage enabled in source cluster
- CNKAF-825: Introduce a system test for replica placement constraints
- CNKAF-1014: Rename usages of SBK in logs to DataBalancer
- CNKAF-613: Introduce a broker removal system test
- METRICS-1894:Telemetry Reporter is missing labels configured in prope…
- METRICS-1817: add include regex default value to Provider interface
- METRICS-1938: upgrade jose4j version (#2219)–backport
- METRICS-1929: use advertised listeners/host-port when deriving local producer config
- KSTORAGE-892 Re add couple of lines of ULE code that got deleted duri…
- Backporting config/events Telemetry Reporter changes along with all shading issues.
- Added one more toJson method in protoserde for webhook alert
- (backport) MMA-8432: fix wrong package name
- CNKAF-1004: Don’t describe removed broker configs
- METRICS-1895: Update telemetry client version
- KC-705: Retry metadata topic consumer.position() until authorizer startup times out
- METRICS-1918: bump gson from 2.8.5 -> 2.8.6
- (backport) prevent ‘io.confluent.controlcenter’ from being renamed by shadow plugin
- KSTORAGE-822: Expose Tiered Storage configs for GA in 6.0
- (Backport) include UnderMinIsrPartitionCount in the default broker metrics list
- CPKAFKA-2852: CCloud metadata listener thread should not die on unexpected exception
- SEC-1276 Allow empty resourceNames in CRNs
- SEC-384: Use the correct log level for debug output
- KC-634: Cluster link system tests with broker failures
- CNKAF-964: Shade opencensus protobufs into SBK jar (#2155)
- Add KafkaRest Metrics Provider
- KSTORAGE-845 Check if tier topic exists already before calling AdminZkClient#createTopic
- CNKAF-952: Update javadoc for removeBroker ConfluentAdmin API with InvalidBrokerRemovalException
- CNKAF-851: log goals skipped during detection in one line
- KC-630: Cluster link system test support with good path mirror test
- CONFLUENT: Prevent log start offset overrun during retention
- KSTORAGE-861: fatal NPE in TierTopicConsumer thread caused by initialization order (#2133)
- Back port [METRICS-1815] Connect client metrics missing some resource level tag…
- CNKAF-936: Remove dot-handling logic in metrics processor (#2121)
- METRICS-1833 Special handling of KSQL metrics in telemetry reporter that are getting mangled.
- Deprecate KafkaHttpServlet in favour of rest-utils Application. (#2135)
- [METRICS-1844, METRICS-1839, METRICS-1756,METRICS-1853] - Reconcile configs in Telemetry Reporter before creating a ConfluentTelemetryConfig object. Use password object for api.secret in ConfluentTelemetryConfig.java.
- [CNKAF-885, CNKAF-946, CNKAF-947] Set SBK Default Throttle, handle ab…
- METRICS-1801: align telemetry domain metric names
- METRICS-1835 Shade confluent-serializers dependencies in confluent-metrics (#2111)
- METRICS-1733 replace ‘whitelist’ with ‘include’ for telemetry and metrics reporter (#2096)
- METRICS-1810 use SBK topic replicas config if present (#2109)
- fix: fixes exception in server shutdown due to metrics registration
- METRICS-1809: add telemetry settings to default server.properties file
- [CNKAF-927] Fix typo in server.properties. (#2090)
- [CNKAF-924, CNKAF-926] Fix Databalancer Topic RF config, remove Teleme…
- CONFLUENT: Added check for CL metrics for 6.0.x
- CPKAFKA-4955: Fix failing testTopicDeletion
- Removed two testing files accidentally added.
- SEC-1130-Inject ConfluentAdmin from ce-kafka
- backport METRICS-1820: use password object for telemetry api secret
- CNKAF-893: Update SBK to work with secure Zookeeper
- METRICS-1816, METRICS-1705 local telemetry exporter metrics whitelist configuration for SBK (#2031)
- CONFLUENT: deleting extra topic config change in cluster linking test
- CNKAF-909: Add direct dependency on opencensus for SBK (#2057)
- KAFKA-10149: Allow auto preferred leader election when partitions are…
- CNKAF-918: Relax GoalUtils.eligibleBrokers criteria to allow leadership movements off observers.
- SEC-1034: use confluent-log4j instead of log4j
- Add RackAwareGoal to goal list
- CNKAF-884: Remove unnecessary metrics (#2059)
- Migrate MDS to run on KafkaHttpServer implementation
- KC-505: integration tests for CL metrics for 6.0.x
- METRICS-1717 use shared domain for jvm level metrics (#1949)
- CNKAF-854: Disabling SBK should persist the broker removal cancellation status
- [CNKAF-873] Don’t require network capacity configs. (#1996)
- SEC-1183 - Treat empty authority in CRN pattern as a wildcard
- CNKAF-882: Add proposal logging for add/remove
- CNKAF-837: Add broker should be able to add removed brokers.
- KC-393: Integration tests for mirroring compacted topics
- Add inter broker listener endpoint information to HTTP server binder.
- CNKAF-773: Do not throw an exception when the broker is not present in the cluster model
- CNKAF-847: Complete removal callback when there are no proposals to execute
- KSTORAGE-802: Allow specifying AWS credentials through configuration file
- [CNKAF-848] Set preferred user SBK config in server.properties. (#1999)
- [CNKAF-719] Add option to disable dynamic throttle setting when stati…
- KSTORAGE-819: Force roll segments only for partitions eligible for tiering
- KSTORAGE-799: reduce Log.lock contention in ArchiveTask segment selection logic
- KSTORAGE-774: Fix active segment roll logic when roll.min.bytes setting is enabled
- KSTORAGE-766: Ensure delete segment throttler does not block active segment rolls
- KSTORAGE-769: Fix to remove the costly toSet conversions in disk throttler
- KSTORAGE-758: Include ERROR status in TierPartitionStatus.isOpen()
- KSTORAGE-595 NPE in TierDeletedPartitionsCoordinator after tier topic is reassigned / deleted from broker
- KSTORAGE-754: KafkaMetrics Whitelist Creation and new KafkaMetricsWrapper util (#1906)
- KC-611: Fix mirror topic create processing incorrect config.
- KSTORAGE-547: state recovery injection tooling
- CONFLUENT: Fix Confluent provider start future completion, add unit tests
- [CNKAF-881] SBK: Don’t treat all new brokers as empty. (#2005)
- SBK/CruiseControl: low utilization fix
- CNAKF-833, CNKAF-842: Fix missing request rate metric and enable tele…
- RackAwareGoal should allow for uneven replica distribution based on t…
- CNKAF-788: Introduce proper API exceptions in the broker removal API
- METRICS-1708: add telemetry version header for forward-compatibility (#1883)
- CNKAF-791: Change data format of failedBrokers in ZK
- CNKAF-807: Remove zkclient from gradle dependencies.
- CNKAF-575: Remove IOtec:ZkClient dependency.
- [CNKAF-776] Switch SBK Maven publish group to “io.confluent”
- [METRICS-1609] dynamically add telemetry reporter at runtime
- KC-389: Retry segment deletion in cluster link test to fix timing issue
- KC-389: Cluster link integration tests with tier storage enabled
- CPKAFKA-5207: Complete authorizer start future only when sufficient nodes are ready
- CNKAF-843: Fix remove broker validation check
- Persist the broker removal state and handle failover
- CNKAF-773: Do not throw exception when there are no replicas to move on broker removal
- CNKAF-428: Integration test for replica placement self-healing
- Add retries to telemetry sampler integration test (#1922)
- [CNKAF-803] Add a metric to track the number of pending add-broker requests
- KC-72: Add topic listing for cluster links.
- CPKAFKA-5207: Fail fast during broker start up if multiple brokers use same MDS URL
- KC-536 bug fix to make offsets/acls configs dynamic (#1893)
- CNKAF-428: Update ReplicaPlacementGoal to handle replication factor change
- fix telemetry reporter reconfiguration
- update ksql and streams provider
- CNKAF-649: Handle broker restart during broker removal
- KC-390: Cluster link authorization tests
- backport [METRICS-1564] telemetry - derive interbroker client configs (#1816)
- (Backport) make commit-id optional, update whitelist to include C3 metrics
- SEC-1083 - AuditAdmin role’s scopeType = Cluster
- KC-392: Integration tests for cluster link quotas on source and destination
- KC-582: Add multi-tenant support for alterMirrors.
- CNKAF-795: Ensure SBK clients have bootstrap.servers configured
- Backport PR 1897 to 6.0.x
- SEC-1083 - Add AuditAdmin role (#1861)
- SEC-1039 - TopicSpec builder sets partitions and RF.
- CNKAF-794: Properly wrap broker removal exceptions in listener and improve remove broker test
- Removed disk throttling dependency on produce backpressure config
- Jvm Collector for Telemetry Reporter
- METRICS-1698 Setting default telemetry.enabled and exporter.enabled values to false.
- KC-224: make leader epoch cache flush asynchronous when stamping new leader epoch
- KC-537: apply topic-create-policy when auto-creating topics while fetching metadata
- CNKAF-782: MetricsReporterSamplers are configured in an odd manner.
- CONFLUENT: Fix cipher name for ChaCha20
- CNKAF-428: Replica placement
- METRICS-1152 document how to enable the telemetry-reporter in CPD
- KC-534: Adds forceful cluster link deletion.
- KCORE-54: Controller should accumulate metadata into fewer requests to mitigate handling delays
- CNAKF-645: Add metrics sampler to consume from telemetry reporter
- Generic telemetry reporter
- KSTORAGE-649: Remove duplicate library dependencies accidentally included in merge.
- [cnkaf-730] part 2: Completion callbacks for add and remove brokers
- [CNKAF-730] Minimal addBrokers support for SBK
- KC-207: Destination CL Metrics
- METRICS-1362 Updating telemetry dependency version from 1.87 to 1.98 to set new content-type
- KSTORAGE-649 and KSTORAGE-660: Support sts::AssumeRole and SSE-KMS
- CNKAF-697 - High bandwidth throttles during cluster roll
- Make ClusterLinkIntegrationTest run as integration test
- CNKAF-760: Integrate api state persistence with remove broker
- Upgrade cloudevents from v0.3 to v1.0
- CC-8138: Add Connect feature to inject Confluent license properties into connector properties before startup
- KC-571: Move tenant prefix to cluster link metadata instead of configs
- CIAM-69 Use authorizer setting to configure CC RBAC roles
- [CNKAF-759] Deal with non-Anomaly executions.
- SEC-994: Update Cluster Registry doc
- KC-387: Multi-tenant integration tests for security updates, offset migration
- KSTORAGE-752: removed disk throttler dependency on logDir existence at server startup
- KC-518: Use cluster link ID as primary key for cluster link metadata.
- Refactor Event Logger to make it generic
- KSTORAGE-699 Rationalize ArchiveTask/DeletionTask Exceptions
- CNKAF-649: Introduce integration test for SBK’s AddBroker, RemoveBroker and DescribeBroker APIs
- Add confluent metrics labels for connect metrics
- CNKAF-649 - Compose the broker removal operation into multiple interruptible tasks
- KSTORAGE-740: Add log closing time metrics
- SEC-951: Upgraded bc-fips to 1.0.2
- Add Confluent metrics tag Kafka server and client
- KSTORAGE-730: Allow max.partition.fetch.bytes to be configurable for tier fetches
- CNKAF-752: Store and fetch remove broker api state
- KSTORAGE-746; Make timing in TierTopicListenersTest deterministic
- CPKAFKA-4744; Fix log cleaner test failure caused by concurrent cleaning
- KC-407: Micro-benchmark for ClusterLinkFetcherThread based on ReplicaFetcherThreadBenchmark
- Introducing disk-usage based throttling for producers and replication followers
- cherry-pick upstream KAFKA-9960: implement KIP-606 to add metadata context to MetricsReporter (#8691)
- KCORE-142 new tests and bug fix for offset command config
- [CNKAF-741] Handle duplicate metrics in the DataBalancerMetricsRegistry
- Add domain to Context class
- CNKAF-702: At SBK startup check for api state persistence topic
- KC-382: Allow alter cluster configs for subset of configs in CCloud
- KC-533: Allow headers to be injected into RBAC records
- CIAM-64 Allow running other Providers with the MultiTenantAuthorizer
- KSTORAGE-741: Deserialize errorOffsetAndEpoch from Header during scan and init
- CNKAF-649 - Broker Removal operation with in-memory state
- CNKAF-617: Complete the DescribeBrokerRemovals API handler
- KSTORAGE-743: prevent tier.feature use with IBP < 2.4
- METRICS-1555 Exposing proxy configs as top level telemetry configs and making them dynamically configurable
- KCORE-142 - Offset Migration for Cluster Linking
- [CNKAF-637] Config variable for auto-healing
- KSTORAGE-425 Add tiered storage metrics to MetricsReporter
- CONFLUENT: Bump major CCloud version #major
- KC-370: Support Authentication events in ConfluentAuditLogProvider
- CPKAFKA-3743; Consumer group tool does not work on internal endpoint
- KSTORAGE-720: Always return tiered storage related configurations in DescribeConfigResponse
- [CNKAF-742] Detect new, empty brokers in controller; call into SBK
- KC-120: Integration Tests for ZK-Based ACLs for Periodic ACL Sync
- CONFLUENT: Add PowerMockIgnore for Scram API in RBAC test to fix classloader issues in subsequent tests
- SEC-959 Cluster Registry - Full Cluster Info CRUD Gatekeeping
- KC-517: Update cluster link integration test to use admin client.
- SEC-1015: Added kafka Metrics support to Confluent Authorizer, RBAC and AuditLog.
- CNKAF-649: Revise SBK Executor’s locking strategy
- KCORE-154: add tiered storage and CP to downgrade system test
- KC-128: Validate source topic read access on mirror topic creation.
- KCORE-329 (2/2): Adds ability to alter a topic’s mirror in TopicCommand.
- Created Broker removal CLI
- KSTORAGE-726: handle offset gaps in tier fetches
- CPKAFKA-2789; Add support-metrics-fullcollector to ce-kafka
- KC-506: Use cluster link tags for cluster link fetcher metrics
- KSTORAGE-627: TierMetadataComparator tool for tier-topic recovery pipeline
- KCORE 506: migrate secrets to Vault
- Spelling fix in LdapGroupManager
- KCORE-95: Encrypt sensitive cluster link configs stored in ZooKeeper
- KCORE-437: Refactor ClusterLink managers to cleanly disable/enable
- CONFLUENT: Improve performance of matchingAcls and AclSeqs.find
- KCORE-49: Disables mirror topic partition and restricted config updates.
- KCORE-485: Process mirror state during replication only on active destination leader
- KCORE-487: Add AuthenticationEvent and update Audit log event schema
- CONFLUENT: Introduce separate methods in KafkaApis for consumer and follower fetch handling
- Update Netty to 4.1.50 and clean up logging of NettySslEngineFactory
- CIAM-45 Allow RoleBindings at path scopes
- CNKAF-636: SBK broker failure detection
- KSTORAGE-627: TierPartitionForceRestore, TierPartitionState hotswap and tier task recovery
- CNKAF-701: Refactor SampleStore topic existence check
- Added tests for KafkaCruiseControl.
- CNKAF-649: Add BrokerRemoval state machine
- KSTORAGE-573: epoch state cache is flushed multiple times in recovery
- [METRICS-1234] telemetry - add support for multiple exporters of the same type
- KCORE-389: Add initial version of Audit API and update existing code
- CNKAF-677: Remove ZK fallback for SBK and move methods from ExecutorUtils to Executor
- CNKAF-684: Honor low utilization threshold settings for self-healing …
- CIAM-28 Add ConfluentCloudCrnAuthority with special treatment of path
- KSTORAGE-717: Filter all Confluent-specific topic configs from DescribeConfigsResponse
- KSTORAGE-652: FileTierPartitionState base path is not updated correctly
- CNKAF-699: Skip num cores or disk capacity update if node missing
- KCORE-340: Propagate new partitions in source cluster to controller
- KSTORAGE-653: re-add check_cluster_state check to tier upgrade test
- KCORE-141: Tenant prefixed cluster links and topic mirroring for Cloud
- KCORE-331: _confluent-license topic exponential backoff
- fix some downstream validation issue
- Add shutdown hook to KafkaHttpServlet interface
- SEC-1008 Do not close null AdminClient
- CONFLUENT: Remove ConfluentLeaderAndIsr and TierListOffset requests/responses
- CNKAF-704: Remove tenant prefix from CreatePartition responses
- CNKAF-314: Add DescribeBrokerRemovals{Request,Response} RPC and Admin Client API
- CONFLUENT: Set Scala 2.13 as the default Scala version
- KCORE-436: Client interceptor for destination tenant prefixing in cluster link clients
- CIAM-25 Allow CRNs with Paths but no Clusters
- emit deltas as cumulative metrics
- CNKAF-649: (part 1/2) Introduce ShutdownManager for SBK broker removal
- CNKAF-697 (part1): Increase min broker quota constants
- CONFLUENT: Improve error message when cluster linking is disabled
- KSTORAGE-412 Server side handling for the unclean leader flag from LeaderAndIsr re…
- METRICS-1514 collect one-minute rates from yammer meters
- Upgrade telemetry api to latest version
- CONFLUENT: Enable fatal warnings with Scala 2.13 and fix warnings
- CONFLUENT: Make createSslEngine(String, String) public again
- KCORE-329: Alter mirrors API.
- KCORE-426: Server should not resolve DNS on establishment of SASL connection
- CPKAFKA-4923: Disable audit logging in RbacTest::test_ldap_to_rbac_authorizer_upgrade
- CNKAF-632: RemoveBrokers api plumbing
- switch downstream validation
- [MINOR] Remove scoverage plugin from SBK build.
- Bump Chart versions
- Merge changes from Apache Kafka
- CONFLUENT: Always close consumer in ConsumeBenchWorker
- KCORE-438 (1/2): Remove topic read test on cluster link creation.
- CNKAF-638: Introduce SBK internal RPC client
- KCORE-41: Disable local produce requests for mirror topics.
- CONFLUENT: Fix build warnings in new ACL sync code
- CLIENTS-1476: Create Kafka HTTP Server interfaces and loading mechanism.
- KCORE-99: Periodic ACL Migration
- CNKAF-622: Properly handle dynamic config changes.
- KCORE-435: Propagate cluster link create/delete using ZK change notification
- KCORE-427: Add broker config to enable cluster linking feature
- CONFLUENT: Don’t reach into java.lang.reflect.Field internals
- KSTORAGE-683: Alter fencing logic for recovery work
- CPKAFKA-4982 -Change Trogdor’s Helm chart to deploy in parallel.
- KAFKA-9731: Disable immediate fetch response for hw propagation if replica selector is not defined
- KCORE-167: Process replication failures with retry timeout to move partition to failed state
- CONFLUENT: Fix locking during metadata restart for link reconfiguration
- KSTORAGE-677: FIX: Reset archiver max lag whenever there is no lag
- remove unnecessary labels from kafka server resource
- CNKAF-629: Add InitiateShutdown Request
- KCORE-55: Adds mirrorTopic and clusterLink to topic creation API.
- KCORE-53: Dynamic security config update for inter-cluster connections
- CNKAF-483: Added static and dynamic configs for request backpressure tuning
- KSTORAGE-26: Fix non-deterministic TierEpochStateReplicationTest
- CNKAF-593: Consolidate all thread creation in KafkaCruiseControlThreadFactory
- KCORE-104: Fix truncation during unclean leader election for linked partition
- CONFLUENT: Simplify Gauge definitions in Partition
- CNKAF-610: Rename StartRebalance RPC and DrainBroker Admin API to RemoveBrokers
- [CNKAF-606] Stop proposal execution on Cruise Control shutdown/SBK deactivation
- [CNKAF-558] Handle SSL interbroker listeners for SBK clients.
- CONFLUENT: Ensure license validator is closed in test to avoid impacting subsequent test
- KSTORAGE-648: remove S3TierObjectStore multi-part log segment upload support
- CNKAF-596: Add a test to verify SBK API access with RBAC
- KSTORAGE-628: Implement TierPartitionFence injector tool for recovery
- KSTORAGE-63: pt2, add TierTopicListeners, add getAndRemoveAll and tests
- KSTORAGE-63: pt1, add TierPartitionForceRestore metadata
- SEC-912 - Allow MDS to make changes to dynamic configs
- KCORE-341: Fix permission type in authorizer debug log entry
- SOAK: Revise soak workloads
- [SEC-912] Allow MDS to make changes to dynamic configs
- KSTORAGE-631: ducker up should not require GCS credentials to be activated in Dockerfile
- CNKAF-577: Handle null config values when describing static throttle configs
- CONFLUENT: Add support for Scala 2.13
- WFLOW-51: Implement a beginShutdown hook which allows for triggering ControlledShutdown
- KCORE-42 (4/4): Adds mirror SyncTopicsConfigs task.
- KCORE-48: Broker implementation for cluster linking APIs.
- KCORE-48 (partial): Adds ClusterLinkCommand.
- KCORE-330: Allow brokers to complete start up with configured license
- [CNKAF-594] Support broker shutdown for SBK
- [CNKAF-492] Set some more config parameters for SBK
- Update cloud Dockerfile to use JDK 14.0.1
- CNKAF-313: Migrate SBK to use DataBalancerMetricsRegistry
- CNKAF-549: Add metric for self healing enabled
- Instructions for exporting metrics to Sandbox env
- KSTORAGE-644: Separate fencing error metrics for primary & catchup consumers
- CONFLUENT: Update JDK to 11.0.7 by bumping cc-base to 5.1.0
- KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
- KCORE-137 (partial): Expands per-topic cluster link state.
- [SEC-895] ConfluentAuditLogProvider should handle reconfiguration wit…
- CONFLUENT: Reduce allocations in requests via buffer caching
- KAFKA-9864: Avoid expensive QuotaViolationException usage (#8477)
- CNKAF-50: Add metrics for replication throttling
- KCORE-51: Adds cluster link metadata object.
- CNKAF-585; Multi tenant interceptor rejects create topic requests with NO_REPLICATION_FACTOR or NO_NUM_PARTITIONS when quotas are enabled
- KCORE-316: Don’t load configs from ZK when the log has already been loaded
- Fix MultiTenantTest
- CPKAFKA-1836: Simplify forwarding scripts by just adding .sh suffix
- CONFLUENT: add utility method to write public keys to pem files (#19)
- CAAS-1960: Record authenticated tenant connection/disconnection metrics
- KAFKA-7304: Limit connection accept rate to prioritize other channels (#29)
- CONFLUENT: Add Aegis Proxy
- CONFLUENT: Update cc-base to v2.3.0
- Merge changes from apache kafka
- CPKAFKA-2164: Ensure that we specify a charset when writing out and reading string bytes
- CONFLUENT: Fix unused compiler warnings
- [SEC-901] Speed up SameClusterTest
- KCORE-42 (partial): Adds functions for merging source + mirror topics…
- SEC-728 - MDS Cluster Registry - Enable Dynamic Configuration
- KCORE-48 (partial): Adds ClusterLinkClientManager for managing admin …
- Towards KCORE-165 (partial): Adds ClusterLinkScheduler for handling cluster l…
- Towards KCORE-136: Cluster linking APIs client+RPC wiring.
- [CNKAF-582] Handle topic creation during checkStartupComponents
- CNKAF-372: Disk capacity sizing
- CONFLUENT: Remove unnecessary allocations in hot matchingAclExists
- METRICS-1415 fix collection of integer gauges
- CNKAF-313: Adds a MetricsRegistry wrapper class for SBK
- [CNKAF-370] Support dynamic enable/disable of SBK
- [CNKAF-492] Adjust the default parameters of the KafkaCruiseControlConfig.
- KSTORAGE-634 Added stronger bounds to ensure tiered log deletion
- KSTORAGE-628: Introduce RecoveryUtils class with recovery tooling related functions
- CPKAFKA-4647: Disable audit logging in RbacTest::test_broker_failure test
- KSTORAGE-628: Introduce TierPartitionFence event type + fencing logic
- add AppInfo version+commitId to metrics messages
- KSTORAGE-577: System test for fencing logic, along with CLI tool to trigger fencing
- KSTORAGE-632: Better logging for replica fetcher state transitions for tiered storage
- KSTORAGE-381 Tooling around TierMetadataValidator to use tier-validator as a cc-service
- CNKAF-536: Reduce number of warnings when SBK creating clients
- [METRICS-1361] telemetry-api -> opencensus-proto
- CNKAF-537: Add drainbrokers adminClientAPI
- CNKAF-562: Change log level for self-healing being triggered from WARN to INFO.
- KSTORAGE-613: fix log start offset initialization on full truncation
- KSTORAGE-626: Remove metrics reporter configuration when configuring internal tier topic clients
- [CNKAF-528] Use the inter-broker listener for SBK clients.
- Prevent warning message on every client metadata update
- KSTORAGE-580: Return a generic error message when compaction is enabled on a tiered topic
- KCORE-40: Inter-cluster fetcher for cluster linking
- Update role, Owner tags for cost reporting
- KCORE-297: Fix wildcard principals for centralized ACLs and no.acl.found for ConfluentServerAuthorizer
- CNKAF-541: Reduce the number of tests run in RandomClusterTest
- METRICS-1232 - Whitelist check for individual and derived metrics
- [METRICS-1233] make telemetry metrics whitelist a dynamic config
- CNKAF-548: Skip checking quotas when recording replication traffic
- Set TopicExclusion string in SBK-generated CruiseControl config.
- KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422) (#1477)
- KAFKA-9820: validateMessagesAndAssignOffsetsCompressed allocates unused iterator (#8422)
- KSTORAGE-331, KSTORAGE-392: add and improve tier component listener metrics and heartbeats
- CPKAFKA-4753: Add metrics tracking total txn time, txn timeouts, and illegal state errors
- CPKAFKA-4649: Ensure LDAP group manager is recreated on writer failover
- SBK MINOR: Drain permits in startUp
- CNKAF-319: Add Start rebalance protocol
- modify kafka_path to only set project for confluentplatform
- CNKAF-520, CNKAF-525: Wait for KafkaSampleStore check to complete at startup
- KSTORAGE-411: Pass a hint down from controller to the leader in case of an unclean leader election
- KSTORAGE-453: Persist leader epoch in tier checkpoint file
- re-schedule pr scans to weekly
- [CNKAF-492] SBK: CruiseControl Property defaults
- Publish preview artifacts with preview release settings
- CNKAF-50: Add static broker-level replication throttles
- SBK: Implement KafkaDataBalancer startup/shutdown methods
- CPKAFKA-4772: Fix PhysicalClusterMetadataTest’s tests
- METRICS-1292 Dynamic Update of API key and secret in Kafka.
- [CNKAF-216] Derive SBK capacity values from config parameters directly.
- CONFLUENT: Make Rest Client Http Authenticator configurable
- CNKAF 362: Topic exclusion logic for SBK in KafkaDataBalancer
- CNKAF-305: Dynamic configs for sbk - enable/disable, throttle
- CONFLUENT: Update cc-trogdor-service and cc-soak-clients to 0.803.0-6.0.0-ce-SNAPSHOT
- KSTORAGE-545: Handle failed deletions from GcsTierObjectStore
- CONFLUENT :HttpBearerCredentialProvider must implement configurable
- ST-3292: migrate to semaphore2
- CNKAF-19: ADB: Soak test self-healing
- KSTORAGE-579: FIX fencing advancement issues
- KSTORAGE-574: producer and epoch state restore is not resilient to failures
- KSTORAGE-399 : Crash broker if FileTierPartitionState encounters corrupted flushed state file during initialization.
- SEC-162: RestClient refactor regression
- [CNKAF-479] Support executing ce-sbk objects with the rest of Kafka
- [CNKAF-491] Remove clusterConfigs functionality from the TopicConfigProvider.
- Revert “KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319) (#1401)”
- KSTORAGE-482 Enhancing TierMetadataValidator tool to verify objects in S3 TierObjectStore
- KSTORAGE-559: add maximum eligible segments deleted on every cleanup run
- CNKAF-486: Add filter to exclude internal topics
- Put an initial implementation of the DataBalancer interface in ce-sbk.
- KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions) (#1319)
- KSTORAGE-568: bound hotset retention by LSO
- KCORE-114: Allow placement-constraints to be configured for internal topics (offsets + transactions)
- CPKAFKA-4670 Fix for system tests running 0.8.2.2 version producers
- CNKAF-324: Allow configuring a delay before broker registration with ZK.
- CNKAF 447: Update SBK mode config to a flag
- [CNKAF-477] Restore ConfluentMetricsReporterSampler
- KSTORAGE-572: FIX: Access fenced partitions metric under lock in TierTopicConsumer
- CPKAFKA-4676: TierRoundtripTest.test_tier_roundtrip timeout increase
- KSTORAGE-548: Check for missing init leader messages
- KCORE-234: Upgrade netty to 4.1.48.Final and netty-tcnative to 2.0.29.Final
- MCM-965: Reduced physical cluster metadata reload period to 2 mins
- CC-7979: Add new topic tracking operations of KIP-558 to default RBAC roles
- KAFKA-9685: Solve Set concatenation perf issue in AclAuthorizer (cherry-pick)
- Bump up rollingupgrade deb package from 0.5.0 to 0.9.0
- [CPKAFKA-4658] Restore removed SchemaRegistryUrl Config.
- KCORE-225: Schedule periodic license validation task at fixed rate
- DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client (#1374)
- DG-274, DG-133: Add configs for Basic Auth and SSL for embedded Schema Registry client
- KSTORAGE-465: GCS tier deletion manager system test support
- v5.5.x KSTORAGE-524: Fix potential NPE in DelayedFetch
- KCORE-199: Set default values for schema configs on topics to avoid NPE
- CNKAF 289: Increase partition count in soak cluster
- [CNKAF-283] Put heavily trimmed CruiseControl into ce-kafka/ce-sbk
- KSTORAGE-550 Narrow scope of KafkaException to avoid handling of any unwanted bug.
- CONFLUENT MINOR: Revert #799, remove batch verifier from eos topic consumers
- KCORE-111: Default replica placement constraints for external topics
- CONFLUENT: fix exhaustive match warnings in TierArchiver
- KSTORAGE-354: avoid roundtrip and connection in GCS fetch path
- KCORE-175: Extend TopicPlacement to include assignment validation logic
- SEC-159: RestRequest pluggable HTTP auth
- KSTORAGE-543: Fix AutoAbortingS3InputStream: account for # read bytes being -1
- KSTORAGE-214, KSTORAGE-407: Bump FileTierPartitionState version
- Tier Storage system test for Consumer.offsetForTimes API
- Refactoring FIPS implementation
- KSTORAGE-389: Stronger detection of duplicate message processing for tiered storage
- CPKAFKA-2336: Log layer metrics for tiered storage
- CNKAF-305: Remove topic config from DynamicBalancer config
- METRICS-1138 Removing shading of transitive dependencies due to telemetry-client jar
- KCORE-190: Add max LSO lag metric
- KSTORAGE-468 : Metadata states validation insync with log start offset.
- CNKAF-304: Default value for rack aware flag
- KCORE-104: Re-instate PartitionLockTest.testAppendReplicaFetchWithUpdateIsr
- KSTORAGE-524: Fix potential NPE in DelayedFetch
- SEC-804: Wait for config update before verification in audit config change test
- CPKAFKA-1953: resume materialization from TierTopicManager commit positions
- Log TierPartitionStatus to differentiate online vs catchup consumer
- KSTORAGE-450: default confluent.tier.fetcher.num.threads = 4
- KSTORAGE-457 ListOffsetRequest v0 does not account for tiered section…
- KSTORAGE-330, KSTORAGE-328, KSTORAGE-325 TierStorageMaterialization recovery and validation tool.
- KSTORAGE-452: Improve README for system tests w/ Tiered Storage
- KSTORAGE-400: Handle duplicate PartitionDeleteInitiate messages
- KSTORAGE-420: missing toString on MergedLog
- KSTORAGE-460: Disable preferred tier reads for replica fetch
- CPKAFKA-4369. Cherry pick of #1228 into 5.5.x
- KSTORAGE-214, KSTORAGE-407: Fence user partition failures when materializing Tier Topic events
- KSTORAGE-510: Ensure that PendingFetch::finish() stops the PendingFetch
- Reduced test strength to ignore consumer data coverage caused by catc…
- CPKAFKA-4126: Add prefix to system test config to match MDS changes
- SEC-147: Refactor RestClient
- KSTORAGE-519: Fix: Ensure unknown exceptions from DeletionTask are isolated to user partition
- KSTORAGE-511: Add size asserts for TierPartitionStateEntry
- KCORE-183: Avoid new allocations in MultiTenantRequestContext.filteredMetadataResponse call
- KSTORAGE-344: TierMaterializationValidation as service for System Tests
- CONFLUENT: Fix bug in result handling from readFromLocalLog
- CNKAF-323: Add metric with per-tenant partition count
- KCORE-151: Allow some reassignment of offline brokers
- KSTORAGE-508: System test s3 prefix random generation incorrectly app…
- KSTORAGE-505: Tier deletion manager system test failing in vagrant runs
- KSTORAGE-424: Allow use of SSL for internal tier topic clients
- KSTORAGE-384 Adding system test for tiered state file corruption
- Merge branch ‘ccs-master’
- KSTORAGE-502: Improve documentation for tiered storage configurations
- CNKAF-305: Dynamic configs for SBK - kafka-side changes
- KCORE-93: Removing ZK utils from ce-kafka
- KCORE-110: Introducing new flag to display partitions violating topic replica constraints
- CNKAF-304: Update network config name
- CNKAF-304: Add static configs for self-balancing kafka
- KCORE-104: Re-introduce PartitionLockTest
- CONFLUENT: Update protobuf to 3.11.4 (#1237)
- Update protobuf to 3.11.4
- CPKAFKA-2700; Disable license validator in MultiTenantAuthorizer
- Define a DataBalancer object in the Kafka Controller.
- SEC-744 Make Audit Log Configs visible to AdminClient
- METRICS-582: Adding tracking for kafka.broker.rack in telemetry
- SEC-752 AuditLog FetchFollower
- KSTORAGE-163: Initial implementation of TierFetcher memory bounding.
- CPKAFKA-2705: Added retention to soak topics
- KSTORAGE-479: Revert “KSTORAGE-464: Fix: ensure a queued TierTopic ev…
- Add manual testing sections to ce-metrics README
- SEC-746 Ignore unknown JSON properties for forwards compatibility
- KSTORAGE-464: Fix: ensure a queued TierTopic event future throws when replaced
- KSTORAGE-337/KSTORAGE-348: tier deletion manager system test
- CONFLUENT: Add check in consume test util to verify records were consumed
- METRICS-541 increase metric reporter default publishing time from 15 se…
- CPKAFKA-3878: Re-enable scala inlining
- CPKAFKA-4369. Copy Kafka configs when creating LogConfig- Part 1 of 2
- CONFLUENT: Log better message on soak clients task creation 409 error response
- TierArchiver: sleep outside of synchronized block in main loop
Security¶
- Validate KSQL/RBAC cannot initialize on non-SASL protocols
- Check authentication.roles in KSQL bearer token requests
- CLIENTS-1658: Remove kafka core dependency from kafka-rest-security-plugin
- SEC-1034: log4j migration to confluent repackaged version
- remove test scope from connect-runtime dependency
- DG-706: fix bug in schema registry client tests, as well as some refactor
- DG-708: Skip Schema Validation query range check for first lookup
- Include Kafka dependencies to KSQL packaging
- Bump Vert.x version in confluent-ksql-security-plugin to 3.9.1
- fix: Make rest-utils compile scope as ksqlDB no longer pulls this in
- SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
- Remove AdminClientWrapper.
- Switch from application/vnd.api+json to application/json.
- Migrate off JSON:API.
- SEC-951: Upgraded bc-fips jars to 1.0.2
- Add authorizations for mode endpoints
- Add test that reproduces rest-proxy authorization bug.
- Fix authentication for non-producer POST requests in kafka-rest.
- Fix issue SSL class could not be found
- Remove obsolete kafka-rest dependencies.
- Fix bugs in Vertx auth plugin loginservice initialisation and unimplemented response method
- Fix Content-Type for produce request.
- Fix test due to ksql refactoring.
- Remove kafka-rest V1 APIs and all their dependencies.
- DG-403 Add missing method to retrieve schema refs
- Vertx auth adaptors
- SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
- Add confluent-security-plugins-common module back
- Remove support-metrics-common dependency
- KCORE-82: Removing ZK licensing code and introducing LicenseManager
- Use kafka.metrics.KafkaYammerMetrics instead of com.yammer.metrics.Metrics
- Fixing compiling error due to KIP-515
MQTT Proxy¶
- CC-11294: Bumped up Netty version to be consistent with Kafka and rest of CP.
- METRICS-1989: confluent-kafka-mqtt depends on confluent-telemetry
- CC-10900: Add confluent-telemetry to production jars.
- CC-9739: Pass producer-striped configs and bootstrap.servers to AdminClient when getting Kafka cluster id.
- CC-9786: Depend on confluent-metrics in tests only.
- fix for METRICS-1733 change telemetry config constant names
- CC-8363: Configure telemetry reporter.
Metadata Service¶
- SEC-1421 - Set default replicator factor when creating audit log topic
- SEC-1369 - Update MDS to non-broken version of Maven Assembly plugin
- SEC-1034: log4j migration to confluent repackaged version
- SEC-866 - move audit log “other” to “management”
- [SEC-1345] Read HTTP Reason phrase when connection closed
- Move kafka-http-server project into clients project
- SEC-1323 - Make MDS Api doc not have everything be Bold
- Fixed code which hard code security protocl to SASL_PLAINTEXT, should…
- SEC-1268 - Backout confluent.security.event.logger.destination.log.* properties
- SEC-1259 - Clarify audit log config API spec
- SEC-777 - Cluster Registry & MDS should be lenient with it’s JSON Parsing
- SEC-777 Cluster Registry & MDS should be lenient with it’s JSON Parsing
- SEC-1222 Role Resources For Principal API throw a null pointer exception if ClusterName is passed in ResourceRequest
- SEC-1089 CR C3 - Return Nice Names for all other Private UI endpoints
- Fixed defect that RbacApiAppConfig missed all audit config with prefix confluent.security.
- CLIENTS-1599: Convert KafkaHttpServlet to rest-utils Application.
- SEC-1091 - CR C3 - Implement MDS 1.0 Features Endpoint
- SEC-1212 - Do not create a new admin client if admin config not changed
- SEC-1097: Disallow reassignment of name <=> scope.
- SEC-1088 CR C3 - Return Nice Names for C3 visibility endpoint
- MMA-7805: Fix for correctly packaging cluster-information-migration-script
- MMA-7805 Script to import and export cluster information data to and from Registry
- SEC-1191 - Assign default topics to produce, consume and other category
- SEC-1021: updated validation + make error messages useful
- SEC-1130 - Inject ConfluentAdmin from ce-kafka
- Migrate MDS to KafkaHttpServlet
- Added more testing cases for verifying audit log backend API.
- SEC-1127 - Fixed a defect that AuditLogConfigSpec didn’t get updated in DynamicC…
- SEC-988 Cluster Registry Fix to make CLI backwards compatible
- SEC-1122 - Fix 6.0.x build break
- SEC-986: Lockdown cluster name
- SEC-940 - Make clusters optional in MdsScope in swagger spec
- SEC-1037: Return sorted ‘full’ clusters followed by sorted ‘redacted’ clusters for cluster registry
- SEC-1003 Cluster Registry : Migrate from “name” to “clusterName”
- SEC-1020 - Cluster Info Protocol Validation
- SEC-982: Nice name gatekeeping
- CIAM-12 - Try loading token.key.path as a public key for cloud
- SEC-731: Single shot overwrite of multiple clusters
- SEC-959: Full Cluster Info CRUD Gatekeeping for Cluster Registry
- SEC-1019 Cluster Registry : Introduce security protocol field to ClusterInfo
- SEC-985: MDS CR NiceNames : Update basic RBAC endpoints
- SEC-1015: Fixed compile error due to interface changed by SEC-1015 in ce-kafka.
- For backward compatibility we name the json scope object as ‘scope’
- Add support for Cluster Scope object with clusters or paths to MdsSco…
- CIAM-49 Spec file for MDS API v2alpha1 - fix whitespace
- CIAM-49 Spec file for MDS API v2alpha1
- SEC-1001: Require at least one host in a Cluster
- SEC-926: Using dynamic config for cluster registry backend
- SEC-941 Add Http api that returns an array of Mds Scope object.
- SEC-942: MDS CR NiceNames : Introduce MdsScope Object
- SEC-965: Sorted-ness of ClusterRegistry
- SEC-943 - MDS CR : Investigate creating ResourceType for CR CRUD
- Fix OpenAPI specification format
- SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
- SEC-730: Cluster Registry - Cluster Level CRUD “API stub”
- SEC-949 - MDS CR : Update CR given Audit logs fix
- SEC-728 - Cluster Registry - Make the backend be a broker Dynamic Config
- SEC-891 - MDS : Short term fix flakey Token tests by isolating them
- [SEC-844] restore /security/1.0 prefix
- SEC-871 - MDS Test : Test / document OOTB Ldap and RBAC case sensitivity
- SEC-850 - CSP : Make code path from BEARER Auth to BEARER Login explicit
- SEC-800 - MDS : Remove MdsLoginService.roles
- SEC-751 MDS - Clean up the openapi yaml spec for CLI
- SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
- SEC-751 - MDS - Clean up the openapi yaml spec for the CLI
- SEC-727 - Cluster Registry - Lock down the basic list of clusters
- SEC-726 - Service Reg - Basic list of clusters
Replicator¶
- REP-1176 Log error when connector submission fails for Replicator executable
- REP-1131: Add support for Replicator with regex router
- REP-1055 Enable converter configs for Replicator connector
- REP-787: Synchronize fetching end offsets from source cluster to calculate lag
- REP-824: Changed JMX initialization to use method from KIP-606
- REP-721: Delete old license validation code from codebase
- REP-795 moved translated records log to
DEBUG
and only print if records were translated - REP-761: ensure
offset.topic.commit
is false iftopic.preserve.partitions
is false - REP-755: retrieve port from –connect-url if specified
- REP-743: Fix issue where we don’t properly log the topic that got an
UnknownTopicPartitionException
- REP-754 fix for offset commit failure resulting in loss of offsets
- REP-689: Reduce log level of offset commit exception from
ERROR
toWARN
- REP-665: Deleting all ZK dependencies from code base
- REP-673: Port ZK dependencies for replicator-verifier into the verifier module
- REP-664: Remove ZK dependencies from
ReplicatorSourceConnector
andReplicatorSourceTask