Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Release Notes

Confluent Platform 5.2.4 Release Notes

This is a bugfix release of Confluent Platform that provides Confluent users with Apache Kafka 2.2.2, the latest stable version of Kafka, and additional bug fixes.

You are encouraged to upgrade to Confluent Platform 5.2.4, because it includes important bug fixes. The technical details of this release are summarized below.

Community Features

Kafka

  • PR-8339 - KAFKA-9752; New member timeout can leave group rebalance stuck (#8339)
  • PR-8205 - KAFKA-9634: Add note about thread safety in the ConfigProvider interface (#8205)
  • PR-8165 - KAFKA-9601: Stop logging raw connector config values (#8165)
  • PR-8108 - KAFKA-9533: ValueTransform forwards null values (#8108)
  • PR-7731 - KAFKA-9204: allow ReplaceField SMT to handle tombstone records (#7731)
  • PR-7733 - KAFKA-9192: fix NPE when for converting optional json schema in structs (#7733)
  • PR-8059 - KAFKA-7052 Avoiding NPE in ExtractField SMT in case of non-existent fields (#8059)
  • PR-7932 - KAFKA-8764: LogCleanerManager endless loop while compacting/cleaning (#7932)
  • PR-7870 - KAFKA-9254; Overridden topic configs are reset after dynamic default change (#7870)
  • PR-7648 - KAFKA-9143: Log task reconfiguration error only when it happened (#7648)
  • PR-7819 - KAFKA-9024: Better error message when field specified does not exist (#7819)
  • PR-7593 - KAFKA-9083: Various fixes/improvements for Connect’s Values class (#7593)
  • PR-7028 - MINOR: Embedded connect cluster should mask exit procedures by default (#7028)
  • PR-7753 - KAFKA-9232; Coordinator new member timeout does not work for JoinGroup v3 and below (#7753)
  • PR-7840 - KAFKA-9307; Make transaction metadata loading resilient to previous errors (#7840)
  • PR-7117 - KAFKA-8705: Remove parent node after leaving loop to prevent NPE (#7117)
  • PR-7793 - MINOR: clarify node grouping of input topics using pattern subscription (#7793)
  • PR-7723 - KAFKA-9190; Close connections with expired authentication sessions (#7723)
  • PR-6031 - KAFKA-1714: Fix gradle wrapper bootstrapping (#6031)
  • PR-7532 - KAFKA-9051: Prematurely complete source offset read requests for stopped tasks (#7532)
  • PR-7298 - KAFKA-8875; CreateTopic API should check topic existence before replication factor (#7298)
  • PR-7534 - MINOR: log reason for fatal error in locking state dir (#7534)
  • PR-7315 - KAFKA-8340, KAFKA-8819: Use PluginClassLoader while statically initializing plugins (#7315)
  • PR-7524 - KAFKA-8945/KAFKA-8947 backport (#7524)
  • PR-7491 - KAFKA-9014: Fix AssertionError when SourceTask.poll returns an empty list (#7491)
  • PR-7442 - KAFKA-8974: Trim whitespaces in topic names in sink connector configs (#7442)
  • PR-7371 - KAFKA-6290: Support casting from logical types in cast transform (#7371)
  • PR-6914 - KAFKA-8523 Enabling InsertField transform to be used with tombstone events (#6914)
  • PR-7426 - KAFKA-8649: send latest commonly supported version in assignment (#7426)
  • PR-6753 - KAFKA-8379; Fix KafkaAdminClientTest.testUnreachableBootstrapServer (#6753)
  • PR-7377 - KAFKA-8896: Check group state before completing delayed heartbeat (#7377)
  • PR-7411 - MINOR: Update Jackson to 2.10.0 (#7411)
  • PR-7346 - KAFKA-8913: Document topic based configs & ISR settings for Streams apps (#7346)
  • PR-7223 - KAFKA-8816: Make offsets immutable to users of RecordCollector.offsets (#7223)
  • PR-7207 - KAFKA-8412: Fix nullpointer exception thrown on flushing before closing producers (#7207)
  • PR-6993 - KAFKA-8586: Fail source tasks when producers fail to send records (#6993)

Common

  • PR-250 - Upgrade to junit 4.13
  • PR-246 - Update to jackson databind 2.9.10.3
  • PR-235 - MINOR: Bump Guava version to pass dependency check
  • PR-240 - Change default dependency report to xml
  • PR-236 - update owasp dependency check
  • PR-225 - Update to jackson 2.9.10

KSQL

  • PR-4857 - fix: don’t cleanup topics on engine close (#4658)
  • PR-4819 - fix: backport fixes from query close
  • PR-4769 - fix: fix test compile goal
  • PR-4759 - build: disable -Werror compiler argument in tests
  • PR-4527 - test: data providers should be ordered (MINOR)
  • PR-3972 - test: fix flaky test - caused by async ACL creation
  • PR-3807 - build: Inherit jackson from common pom
  • PR-3557 - test: improve stability of tests on older branches
  • PR-3342 - chore: move build failure alerts to the new #ksql-alerts channel

Connectors

Elasticsearch Connector
  • PR-398 - CC-8284: Log errors during index exist check at info level
  • PR-361 - CC-7147: set the max connection per route config for client
  • PR-364 - CC-7367: Add a lot more logging
  • PR-135 - CC-6599: java.lang.ClassCastException: java.util.Date cannot be cast to java.lang.Long while setting default date value
  • PR-340 - CC-6187: Cache already existing index in JestElasticsearchClient
  • PR-337 - CC-4886: Fix the SecurityIT integration test
HDFS Connector
  • PR-491 - CC-8229: HDFS2 format options are not showing all available options
  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config
  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector
JDBC Connector
  • PR-797 - MINOR: Upgrade to Derby 10.14.2.0
  • PR-793 - CC-7637: change from DEBUG level to TRACE level
  • PR-792 - CC-8090: Validate connection.attempts property in config class
  • PR-791 - CC-8087: Remove plaintext logging of task configs
  • PR-715 - [jdbc-source-connector] Fixed issue in CachedConnectionProvider, to enable retries when a connection is failed
  • PR-787 - Make metadata allFields ordered consistently with the valueSchema.
  • PR-770 - CC-7628 Set conservative timeout for JDBC to stop trying to reach drive
  • PR-785 - CC-5047: connect jdbc url field should have example text
  • PR-775 - Move setFetchSize to GenericDatabaseDialect
  • PR-776 - CC-7637: added logging of query to be executed at DEBUG level
  • PR-758 - Fix Postgres OOM by limiting fetch size
  • PR-739 - MINOR: Add TimeZoneValidator::toString.
  • PR-761 - CC-7414: Fix SqlServer column converter infinite loop & add column converter tests
  • PR-759 - CC-7382: use true/false as default values for boolean type
  • PR-750 - CC-7265: Support SQL Server DateTimeOffset
  • PR-752 - CC-7265: Add support for SQL Server DateTimeOffset type
  • PR-727 - CC-6773: Abort source task if all columns were not converted correctly.
  • PR-728 - MINOR: Expose SinkRecordField schema to database dialects
  • PR-724 - CC-6747: Replace static mocking for the CachedConnectionProvider
  • PR-706 - MINOR: Avoid NPE when columns cannot be described
JMS Connector
  • CC-3439: Update MQ Source Connector to better handle errors and autoreconnect when MQ manager is restarted
  • CC-6503: Capping batch size at 10000 for IBM MQ connector
  • CC-7984: corrected pom version
  • CC-3937: repackaging base classes
  • CC-7926: Corrected logic to prevent infinite loops during poll()
  • CC-7866: Added more and improved existing logging in the base JMS source task and connector
  • CC-6505: Change instances of RetriableException to ConnectException in base source task
  • CC-6267: Fix NPE when using permissive schemas
  • CC-6267: Add permissive schema configuration
  • CC-6200 : Fix NPE due to null property values
Replicator Connector
  • REP-638: Ensuring translatorMonitor is not null before closing it out
  • REP-637: Ensuring timestampsCommitter is not null before closing it out
  • REP-609: Fixing potential null pointer exception
  • REP-24: ZK license manager change
  • REP-47 added arity to boolean cmd line options as they all default true
  • REP-403: Fixed timestamp committer thread leak
  • REP-26: Fixed issue where group.id was not propagated to license manager
  • REP-31: Added code to specify default destination security settings
  • REP-186: Updated replicator to use delivery.timeout.ms instead of retries
  • REP-20: Added a check to ensure TopicMonitorThreadWithZk is shutdown
S3 Connector
  • PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
  • PR-311 - Add AWS IAM Assume Role credentials provider
  • PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
  • PR-306 - MINOR: Upgrade AWS SDK to 1.11.725
  • PR-300 - MINOR: Small fix to the config docs for S3 proxy
  • PR-299 - MINOR: Small fixes to the formatting of config docs
  • PR-281 - CC-6801: Retry effectively on retriable failures while closing files

Storage Common

  • PR-114 - CC-147: logical types support for HiveSchemaConverter
  • PR-112 - MINOR: Improve the locale and timezone config descriptions
  • PR-110 - CC-5963: Improved ability to handle schema evolution with different names

Schema Registry

  • PR-1381 - CC-7027: Revert to legacy serialization of Decimal logical type on incompatible scale/precision
  • PR-1268 - Add null checks
  • PR-1300 - CC-7692 / ISSUE-1197: Fix configuration of basic auth credentials provider
  • PR-1260 - MINOR: Fix warn log during retries
  • PR-1242 - DG-128: Allow DNS lookup policy to be configurable
  • PR-1233 - DG-114: Fix tombstone behavior
  • PR-1026 - CC-3759 - Regenerate ID if in use during registration

Confluent Platform 5.2.3 Release Notes

This is a bugfix release of Confluent Platform that provides Confluent users with Apache Kafka 2.2.2, the latest stable version of Kafka, and additional bug fixes.

You are encouraged to upgrade to Confluent Platform 5.2.3, because it includes important bug fixes. The technical details of this release are summarized below.

Commercial Features

Control Center

  • [MMA-2901] fix file limit issue on 5.2.x

Community Features

Apache Kafka 2.2.2-cp1

  • PR-6283 - KAFKA-7941: Catch TimeoutException in KafkaBasedLog worker thread (#6283)
  • PR-7197 - KAFKA-8774: Regex can be found anywhere in config value (#7197)
  • PR-6959 - KAFKA-8550: Fix plugin loading of aliased converters in Connect (#6959)
  • PR-7164 - KAFKA-8736: Streams performance improvement, use isEmpty() rather than size() == 0 (#7164)
  • PR-7101 - KAFKA-8678; Fix leave group protocol bug in throttling and error response (#7101)
  • PR-7054 - KAFKA-8615: Change to track partition time breaks TimestampExtractor (#7054)
  • PR-7092 - KAFKA-8602: Separate PR for 2.3 branch (#7092)
  • PR-7021 - KAFKA-8620: fix NPE due to race condition during shutdown while rebalancing (#7021)
  • PR-7076 - HOT FIX: close RocksDB objects in correct order (#7076)
  • PR-7070 - KAFKA-7157: Fix handling of nulls in TimestampConverter (#7070)
  • PR-5705 - KAFKA-6605: Fix NPE in Flatten when optional Struct is null (#5705)
  • PR-7030 - KAFKA-5998: fix checkpointableOffsets handling (#7030)
  • PR-6991 - KAFKA-8591; WorkerConfigTransformer NPE on connector configuration reloading (#6991)
  • PR-6974 - KAFKA-8570; Grow buffer to hold down converted records if it was insufficiently sized (#6974)
  • PR-6968 - KAFKA-8564; Fix NPE on deleted partition dir when no segments remain (#6968)
  • PR-6875 - KAFKA-7315 DOCS update TOC internal links serdes all versions (#6875)
  • PR-6898 - KAFKA-8499: ensure java is in PATH for ducker system tests (#6898)
  • PR-6893 - HOTFIX: Close unused ColumnFamilyHandle (#6893)
  • PR-6791 - KAFKA-8404: Add HttpHeader to RestClient HTTP Request and Connector REST API (#6791)
  • PR-6596 - KAFKA-8155: Add 2.1.1 release to system tests (#6596)

Kafka Connect

  • PR-334 - CC-4886: Temporarily ignore SecurityIT
  • PR-335 - MINOR: Upgrade Mockito to work on Java 11
  • PR-327 - Enable Elasticsearch 7 support with adding minimum Integration test (based on TestContainer)
  • PR-324 - CC-5605: Log version conflicts as warnings.
  • PR-312 - CC-4788: Added elastic.security.protocol.
  • PR-450 - CC-5969: Pass extra connector configs to the partitioner
  • PR-686 - MINOR: Upgrade postgress jdbc driver to most recent bugfix release (9.4.1212)
  • PR-663 - MINOR: Remove the final keyword from JdbcSinkConnector
  • PR-79 - MINOR: Revert to using the older version of jackson-databind to fix the build
  • PR-73 - ST-1601: Fix test and framework dependency scopes so we don’t leak unnecessary jars into packages
  • PR-228 - REP-30: MINOR: Fix Jenkins build failures due to the maven-surefire plugin
  • PR-226 - REP-30: MINOR: Fix Jenkins build failures due to the maven-surefire plugin
  • PR-217 - CC-2796: Pass a delta for better than ms precision
  • PR-214 - CC-5541: Set ByteArrayConverter as header converter
  • PR-209 - CC-5012: If topic creation fails, attempt to resize topic
  • PR-215 - REP-22: Handle uncaught exception during offset commit

KSQL

  • PR-3104 - fix: COLLECT_LIST can now be applied to tables
  • PR-3002 - Move docker image to break circular dependency
  • PR-3017 - build: remove npm install from mvn package
  • PR-3008 - build: add commitlint for conventional commits
  • PR-2948 - Using the correct props for the version checker in the headless mode.
  • PR-2947 - Fixing some rest-app module tests.
  • PR-2945 - Added clarification to docs for PARTITION BY
  • PR-2921 - KSQL-2630: Update out-of-order verbiage; remove grace-period section
  • PR-2922 - KSQL-2236: Add paras about the WITHIN clause

Confluent Platform 5.2.2 Release Notes

This is a bugfix release of Confluent Platform that provides Confluent users with Apache Kafka 2.2.1, the latest stable version of Kafka, and additional bug fixes.

You are encouraged to upgrade to Confluent Platform 5.2.2, because it includes important bug fixes. The technical details of this release are summarized below.

Commercial Features

Security Plugins

  • CPKAFKA-2819: Add config for ssl distinguished name mapping (#120)

Control Center

  • MMA-4401: fix unknown cluster exception

Replicator

  • CC-4930: Adjust expected calls in case schedulePeriodicRefreshMetadata gets scheduled by NewReplicatorAdminClient
  • CC-4928: During translation retries, save newer timestamps for retry
  • CC-4536: Adapt ReplicatorApp to upstream changes in AK
  • CC-4576: De-duplicate in timestamp interceptors: add configs for batching in Replicator
  • CC-4474: Use committed offsets even if dest topic doesn’t exist
  • CC-4347: Start REST server when using Replicator app
  • CC-3405: Close admin client if constructed locally
  • CC-4257: Allow consumer offset to be preferred over connect offset

MQTT Proxy

  • CC-3848: Support specifying the maximum allowed MQTT message size in bytes

Community Features

Apache Kafka 2.2.1-cp1

  • PR-6840 - KAFKA-8418: Wait until REST resources are loaded when starting a Connect Worker. (#6840)
  • PR-6818 - KAFKA-8187: Add wait time for other thread in the same jvm to free the locks (#6818)
  • PR-6722 - KAFKA-8351: Cleaner should handle transactions spanning multiple segments (#6722)
  • PR-6579 - KAFKA-8229: Reset WorkerSinkTask offset commit interval after task commit (#6579)
  • PR-6636 - KAFKA-8290: Close producer for zombie task (#6636)
  • PR-6719 - KAFKA-8347: Choose next record to process by timestamp (#6719)
  • PR-6675 - KAFKA-8320: fix retriable exception package for source connectors (#6675)
  • PR-6726 - KAFKA-8363: Fix parsing bug for config providers (#6726)
  • PR-6715 - KAFKA-8335: Clean empty batches when sequence numbers are reused (#6715)
  • PR-5918 - KAFKA-7633: Allow Kafka Connect to access internal topics without cluster ACLs (#5918)
  • PR-6713 - KAFKA-8352: Fix Connect System test failure 404 Not Found (#6713)
  • PR-6707 - KAFKA-8348: Fix KafkaStreams JavaDocs (#6707)
  • PR-5578 - KAFKA-6789: Handle retriable group errors in AdminClient API (#5578)
  • PR-6685 - KAFKA-8240: Fix NPE in Source.equals() (#6685)
  • PR-6651 - KAFKA-8304: Fix registration of Connect REST extensions (#6651)
  • PR-6652 - KAFKA-8306: Initialize log end offset accurately when start offset is non-zero (#6652)
  • PR-6672 - KAFKA-8323: Close RocksDBStore’s BloomFilter (#6672)
  • PR-6670 - KAFKA-8289: Fix Session Expiration and Suppression (#6654) (#6670)
  • PR-6568 - KAFKA-7601: Clear leader epoch cache on downgraded format in append (#6568)
  • PR-6613 - KAFKA-8248: Ensure time updated before sending transactional request (#6613)
  • PR-6402 - KAFKA-8066: Always close the sensors in Selector.close() (#6402)
  • PR-6643 - KAFKA-8298: Fix possible concurrent modification exception (#6643)
  • PR-6602 - KAFKA-8254: Pass Changelog as Topic in Suppress Serdes (#6602)
  • PR-6305 - Fix for KAFKA-7974: Avoid zombie AdminClient when node host isn’t resolvable (#6305)
  • PR-6615 - KAFKA-7895: fix Suppress changelog restore (#6536) (#6615)
  • PR-6555 - KAFKA-8204: fix Streams store flush order (#6555)
  • PR-6550 - KAFKA-8277: Fix NPEs in several methods of ConnectHeaders (#6550)
  • PR-6570 - KAFKA-7866: Ensure no duplicate offsets after txn index append failure (#6570)
  • PR-6585 - KAFKA-8241: Handle configs without truststore for broker keystore update (#6585)
  • PR-6573 - KAFKA-8210: Fix link for streams table duality (#6573)
  • PR-6564 - KAFKA-8209: Wrong link for KStreams DSL in core concepts doc (#6564)
  • PR-6572 - KAFKA-8208: Change paper link directly to ASM (#6572)
  • PR-6581 - KAFKA-8232: Test topic delete completion rather than intermediate state (#6581)
  • PR-6384 - KAFKA-8058: Fix ConnectClusterStateImpl.connectors() method (#6384)
  • PR-6547 - KAFKA-8157: fix the incorrect usage of segment.index.bytes (2.2) (#6547)
  • PR-6539 - KAFKA-8190: Don’t update keystore modification time during validation (#6539)
  • PR-6475 - KAFKA-8126: Flaky Test org.apache.kafka.connect.runtime.WorkerTest.testAddRemoveTask (#6475)

Connect

  • PR-305 - SEC-168: Exclude jackson-core from packaging.
  • PR-302 - CC-4054: Remove the use of null_value (default values) for text and binary based fields
  • PR-425 - CC-4318: Fix off-by-one error for offset reporting to Kafka topic
  • PR-642 - CC-4423: Remove semicolon from Db2 dialect timestamp query
  • PR-66 - CC-4422: JMS Connector Performance Updates
  • PR-243 - CC-4228: Configuration option to override expect-continue in upload protocol

Schema Registry

  • PR-1101 - CC-4513: Fix HTTP header handling in CachedSchemaRegistryClient

Confluent Platform 5.2.1 Release Notes

This is a bugfix release of Confluent Platform that provides Confluent users with Apache Kafka 2.2.0, the latest stable version of Kafka and additional bug fixes.

You are encouraged to upgrade to Confluent Platform 5.2.1, because it includes important bug fixes. The technical details of this release are summarized below.

Commercial Features

Control Center

The Control Center System Health page in Firefox browser now renders correctly. For more information on supported browsers, see Web Browsers.

Community Features

Apache Kafka 2.2.0-cp2

  • PR-6489 - KAFKA-8150: Fix bugs in handling null arrays in generated RPC code (#6489)
  • PR-6342 - KAFKA-8014: Extend Connect integration tests to add and remove workers dynamically (#6342)
  • PR-6484 - KAFKA-8142: Fix NPE for nulls in Headers (#6484)
  • PR-6482 - KAFKA-7989: RequestQuotaTest should wait for quota config change before running tests (#6482)

How to Download

Confluent Platform is available for download at https://www.confluent.io/download/. See the On-Premises Deployments section for detailed information.

To upgrade Confluent Platform to a newer version, check the Upgrade documentation.

Supported Versions and Interoperability

For the supported versions and interoperability of Confluent Platform and its components, see Supported Versions and Interoperability.

Questions?

If you have questions regarding this release, feel free to reach out via the community mailing list or community Slack. Confluent customers are encouraged to contact our support directly.