Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Changelog¶

Version 5.4.0¶

PR-289 - MINOR: Upgrade bugfix versions of dependencies to avoid CVEs
PR-286 - MINOR: Pin hadoop-common version to 2.7.7 and exclude htrace-core dependency
PR-282 - CC-6565: Use parquet.codec and add compression options for Parquet format
PR-281 - CC-6801: Retry effectively on retriable failures while closing files
PR-280 - CC-6766 Handle null-valued records in all three output types
PR-241 - Add Parquet Format
PR-264 - MINOR: New log for SSE
PR-263 - Remove jackson databind version override
PR-250 - CC-4969 MINOR: Remove Unused S3Storage object from TopicPartitionWriter
PR-114 - CC-147: logical types support for HiveSchemaConverter
PR-116 - CC-6565: Add parquet.codec property to enable compression in storage sink connectors
PR-110 - CC-5963: Improved ability to handle schema evolution with different names

PR-244 - CC-4433c: Remove unused dependency on Hive.
PR-243 - CC-4228: Add configuration option to override expect-continue in upload protocol.

PR-220 - CC-3035: Upgrade the S3Mock library so the module builds on Java 8 and Java 11

No changes

PR-181 - MINOR: Update config options to include SSE-C
PR-173 - Extend the existing SSE support to enable Customer (provided) Keys SSE-C
PR-171 - MINOR: Remove MaxPermSize since we dropped support for Java 7
PR-160 - CC-1793. Used full jitter backoff strategy from AWS SDK.
PR-156 - CC-1497. Added logic for passing a subset of configs down to configurable credentials provider
PR-63 - DataUtils::getField returns field

No changes

PR-150 - Make confluent CLI command output shared
PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
PR-149 - Bump Jackson version
PR-140 - Fix for BufferOverflowException in S3OutputStream
PR-106 - Add s3.sse.kms.key.id to specify a particular key in AWS KMS
PR-98 - Add gzip compression support for json and byte array formats
PR-143 - Support for multiple fields in field partitioner
PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
PR-123 - CC-1299: Refactor this code to use the AVRO_CODEC config key from storage-common
PR-114 - Update to PowerMock 2.0.x and Mockito 2
PR-48 - RecordField nested timestamp extraction
PR-57 - Issue-53. Multiple field support for partitioning.
PR-55 - CC-1489: NPE on records with null timestamp.
PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.
PR-52 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-51 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-50 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-46 - CC-1299: Create AVRO_CODEC ConfigKey

PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
PR-149 - Bump Jackson version
PR-140 - Fix for BufferOverflowException in S3OutputStream
PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
PR-55 - CC-1489: NPE on records with null timestamp.
PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.

PR-102 - Allow to write records at the root of the bucket
PR-105 - Add instructions about upstream builds
PR-97 - MINOR: S3 Proxy Settings revisited
PR-103 - Remove unused imports
PR-101 - Remove unused imports
PR-85 - Add ByteArrayFormat
PR-87 - Add s3.canned.acl config
PR-84 - Create CONTRIBUTING.md
PR-92 - CC-1114: Switch to common pom and fix checkstyle issues.
PR-78 - Retry s3 part upload
PR-80 - HOTFIX: Remove refs to schema-generator config.
PR-60 - Add logging when creating a new RecordWriter
PR-44 - HOTFIX: Remove unused parameter in newConfigDef for PartitionerConfig.
PR-41 - HOTFIX: Update dependencies
PR-40 - HOTFIX: Make specific dependencies explicit.
PR-37 - Remove unused imports
PR-35 - Add missing modules to the dependencyManagement pom section so downstream projects will inherit the right version automatically.
PR-31 - Remove schema.generator.class config and have Formats specify their own SchemaGenerator internally

PR-28 - HOTFIX: Explicitly deactivate licensing profiles by default.
PR-34 - CC-524: Reduce the default s3.part.size to 25MB to avoid OOM with current default jvm heap size.
PR-33 - CC-513: Newline is not appended between records when using JsonFormat.
PR-37 - Enable compression of output Avro files.
PR-38 - CC-530: Exclude storage-common jars when packaging S3 connector.
PR-19 - Move to using io.confluent:common for deps.
PR-22 - HOTFIX: Import recent changes from the hdfs connector.
PR-24 - Convert - to _ in Hive table names.
PR-25 - Include causes with ConfigExceptions caused by catching other exceptions.
PR-26 - Add constructors to SchemaGenerators to support no parameters and a Map config.
PR-29 - Timestamp information should be copied to projected SinkRecord.

PR-19 - CC-500: Provide exactly-once time-based partitioning in S3
PR-45 - HOTFIX: S3SinkConnector should extend SinkConnector
PR-51 - Allow custom partitioners to have their own configs
PR-27 - HOTFIX: Include a trailing delimiter when verifying data format for Hive
PR-23 - HOTFIX: Add test schema and record builder for records with timestamp field
PR-18 - CC-497: Add timestamp based partitioners.

PR-33 - Separate JSON records using line separator instead of single white space.
PR-34 - Reduce the default s3.part.size to 25MB to avoid OOM exceptions with the current default java heap size settings for Connect.
PR-32 - Add s3.region property to quickstart config and docs.
PR-25 - flush.size doc fixes.

Initial Version