Changelog¶

Version 10.2.2¶

PR-566 - CCMSG-2233: Fixed compression validation for custom format and compression type none

Version 10.1.3¶

PR-566 - CCMSG-2233: Fixed compression validation for custom format and compression type none

Version 10.1.1¶

PR-522 - CCMSG-1688: CVE fix for Guava dependency
PR-533 - Updated parent pom to 11.0.9

Version 10.1.0¶

PR-538 - Elastic byte buffer for S3OutputStream (base on master)
PR-522 - CCMSG-1688: CVE fix for Guava dependency
PR-530 - CCMSG-2050: Add cross validation for compression and format class
PR-509 - CCMSG-1833: Remove the logging of sink record in TRACE message
PR-504 - CCMSG-1743: Upgraded storage-common to resolve jackson-databind 2.13.2 CVE
PR-497 - Fix regions list being one item of a list
PR-495 - Don’t log full proxy config, don’t log configuration @ info level
PR-494 - Generalize S3 record writes with wrapper to possibly convert to RetriableException
PR-492 - CCMSG-1481: Retriable exception is possible to be thrown from close()
PR-461 - Fix retries for S3 transient errors
PR-485 - CCMSG-1531: Support null items within arrays with Parquet writer
PR-475 - CCMSG-1582: Fixed a build error in testcase happened due to merge of from 5.5.3
PR-474 - CCMSG-1582: Fixed a build error in testcase happened due to merge from upstream
PR-473 - CCMSG-1582: Fix bug that generates a small files when scheduled rotation has expired
PR-470 - MINOR: Remove guava usage and superfluous template arguments.
PR-466 - Allow string match of formatting file extension to exist in the topic name (or elsewhere in the file besides the end)
PR-460 - Disabled concurrent jobs
PR-456 - CCMSG-1420: Exclude unused netty-codec dependency
PR-453 - use lastIndexOf for deriving adjusted filename
PR-448 - CC-15510: Add guava dependency to resolve CVE
PR-445 - CCMSG-1259: Upgrade parent and storage-common dependencies to 10.2.2
PR-442 - CCMSG-1194: Upgrade parent and storage-common dependencies to 10.2.1
PR-441 - CCMSG-1146: Upgrade json-smart to 2.4.7 through parent
PR-394 - Upgrade to hadoop common 2.10.1 for parquet support
PR-431 - CC-14987: Fix serialization when headers.format.class is json
PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4 to resolve CVE
PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
PR-365 - CC-11945: Fix empty header check for storing headers

Version 10.0.14¶

PR-566 - CCMSG-2233: Fixed compression validation for custom format and compression type none

Version 10.0.12¶

PR-542 - CCMSG-2014: CVE fix - Upgrade storage-common version to remove parquet format CVEs
PR-536 - CCMSG-1974: CVE fix - Remove dependency on slf4j-log4j

Version 10.0.11¶

PR-522 - CCMSG-1688: CVE fix for Guava dependency
PR-533 - Update parent pom to 11.0.9

Version 10.0.10¶

PR-530 - CCMSG-2050: Add cross validation for compression and format class

Version 10.0.9¶

No changes

Version 10.0.8¶

PR-518 - CCMSG-1577: Label change to compression level for s3 sink connector

Version 10.0.7¶

PR-504 - CCMSG-1743: Upgraded storage-common to resolve jackson-databind 2.13.2 CVE

Version 10.0.6¶

PR-497 - Fix regions list being one item of a list
PR-495 - Don’t log full proxy config and don’t log configuration at info level
PR-494 - Generalize S3 record writes with wrapper to possibly convert to RetriableException
PR-492 - CCMSG-1481: Retriable exception is possible to be thrown from close()
PR-461 - Fix retries for S3 transient errors
PR-490 - MINOR: Suppress warning which causes IntelliJ to error after Maven import (#489)
PR-485 - CCMSG-1531: Support null items within arrays with Parquet writer

Version 10.0.5¶

PR-473 - CCMSG-1582 Fix bug that generates small files when a scheduled rotation has expired
PR-470 - MINOR: Remove guava usage and superfluous template arguments
PR-451 - MINOR: Remove toString() call in trace log to resolve a performance issue

Version 10.0.4¶

PR-466 - Allow string match of formatting file extension to exist in the topic name (or elsewhere in the file besides the end)
PR-456 - CCMSG-1420 Exclude unused netty-codec dependency

Version 10.0.3¶

PR-453 - CCMSG-1373 use lastIndexOf for deriving adjusted filename when the extension is present in the filename.
PR-450 - Moving to OpenJDK 8 build image.

Version 10.0.2¶

PR-448 - CC-15510: Add guava dependency for standalone functionality
PR-445 - CCMSG-1259: Upgrade commons-compress to 1.21

Version 10.0.1¶

PR-441 - CCMSG-1146: Upgrade json-smart to 2.4.7
PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive
PR-432 - CC-14987: Fix serialization when headers.format.class is json
PR-429 - Upgrade jackson-dataformat-cbor to 2.11.4
PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
PR-394 - Upgrade to hadoop common 2.10.1 for parquet support

Version 10.0.0¶

PR-393 - MINOR: Upgrade storage common to 10.0.5 and catch PartitionException in TopicPartitionWriter’s error handling
PR-392 - CCMSG-861: Catch additional exception to report in DLQ
PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
PR-349 - CC-9723: DLQ support in S3 sink
PR-379 - CC-12772: Upgrade testcontainers to 1.15.0 to address commons-compress CVE
PR-377 - CC-12825: Remove unintentional proprietary dependencies
PR-378 - CC-12695: Stop throwing retriable exceptions on non-retriable S3 errors
PR-325 - CC-7933: Support writing key and headers to S3
PR-262 - MINOR: Restore the complete hierarchy of exceptions during failures in part upload
PR-356 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3
PR-346 - SEC-1034: log4j migration to confluent repackaged version
PR-340 - CC-9753: Validate timezone configuration with rotate.schedule.interval.ms
PR-328 - CC-9198: Move null value handling earlier in SinkTask put
PR-291 - S3 Object Level Tagging for Offsets & Record Counts

Version 5.5.11¶

No changes

Version 5.5.10¶

No changes

Version 5.5.9¶

No changes

Version 5.5.8¶

PR-485 - CCMSG-1531: Support null items within arrays with Parquet writer
PR-473 - CCMSG-1582: Fix bug that generates a small files when scheduled rotation has expired
PR-451 - Minor: Remove toString() call in trace log

Version 5.5.7¶

PR-456 - CCMSG-1420: Exclude unused netty-codec dependency for fix CVE

Version 5.5.6¶

No changes

Version 5.5.5¶

PR-394 - Upgrade to hadoop common 2.10.1 for parquet support
PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor

Version 5.5.4¶

PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common

Version 5.5.3¶

PR-379 - CC-12772: Upgrade testcontainers to 1.15.0 to address commons-compress CVE
PR-377 - CC-12825: Remove unintentional proprietary dependencies
PR-378 - CC-12695: Stop throwing retriable exceptions on non-retriable S3 errors

Version 5.5.2¶

PR-356 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3
PR-340 - CC-9753: Validate timezone configuration with rotate.schedule.interval.ms

Version 5.5.1¶

PR-328 - CC-9198: Move null value handling earlier in SinkTask put

Version 5.5.0¶

PR-318 - CC-8170: Exclude jackson-mapper-asl and commons-beanutils from transient dependencies of hadoop-common
PR-310 - MINOR: Fix log message when tagging an S3 object
PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM AssumeRole credentials provider
PR-305 - CC-7892: Introduce configuration properties for key and secret to easily define credentials per connector instance
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
PR-291 - S3 object level tagging for offsets and record counts
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.4.9¶

No changes

Version 5.4.8¶

No changes

Version 5.4.7¶

PR-473 - CCMSG-1582: Fix bug that generates a small files when scheduled rotation has expired

Version 5.4.6¶

PR-456 - CCMSG-1420: Exclude unused netty-codec dependency

Version 5.4.5¶

PR-394 - Upgrade to hadoop common 2.10.1 for parquet support
PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4

Version 5.4.4¶

PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors

Version 5.4.3¶

PR-356 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3
PR-340 - CC-9753: Validate timezone configuration with rotate.schedule.interval.ms
PR-328 - CC-9198: Move null value handling earlier in SinkTask put

Version 5.4.2¶

PR-318 - CC-8170: Exclude jackson-mapper-asl and commons-beanutils from transient dependencies of hadoop-common
PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM AssumeRole credentials provider
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats

Version 5.4.1¶

PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.4.0¶

PR-289 - MINOR: Upgrade bugfix versions of dependencies to avoid CVEs
PR-286 - MINOR: Pin hadoop-common version to 2.7.7 and exclude htrace-core dependency
PR-282 - CC-6565: Use parquet.codec and add compression options for Parquet format
PR-281 - CC-6801: Retry effectively on retriable failures while closing files
PR-280 - CC-6766: Handle null-valued records in all three output types
PR-241 - Add Parquet Format
PR-264 - MINOR: New log for SSE
PR-263 - MINOR: Remove jackson databind version override
PR-250 - CC-4969: MINOR: Remove Unused S3Storage object from TopicPartitionWriter
PR-114 - CC-147: logical types support for HiveSchemaConverter
PR-116 - CC-6565: Add parquet.codec property to enable compression in storage sink connectors
PR-110 - CC-5963: Improved ability to handle schema evolution with different names

Version 5.3.8¶

No changes

Version 5.3.7¶

No changes

Version 5.3.6¶

No changes

Version 5.3.5¶

PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors

Version 5.3.4¶

PR-357 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3 (back port to 5.0.x)

Version 5.3.3¶

PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM Assume Role credentials provider
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.3.2¶

No changes

Version 5.3.1¶

No changes

Version 5.3.0¶

PR-244 - CC-4433c: Remove unused dependency on Hive.
PR-243 - CC-4228: Add configuration option to override expect-continue in upload protocol.

Version 5.2.7¶

No changes

Version 5.2.6¶

No changes

Version 5.2.5¶

PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors
PR-357 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3 (back port to 5.0.x)

Version 5.2.4¶

PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM AssumeRole credentials provider
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.2.3¶

No changes

Version 5.2.2¶

No changes

Version 5.2.1¶

No changes

Version 5.2.0¶

PR-220 - CC-3035: Upgrade the S3Mock library so the module builds on Java 8 and Java 11

Version 5.1.4¶

PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM AssumeRole credentials provider
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.1.3¶

No changes

Version 5.1.2¶

No changes

Version 5.1.1¶

No changes

Version 5.1.0¶

No changes

Version 5.0.4¶

PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-311 - Add AWS IAM AssumeRole credentials provider
PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 5.0.3¶

No changes

Version 5.0.2¶

No changes

Version 5.0.1¶

PR-214 - CC-2617: Accept timestamp for wallclock instances to assure consistency
PR-200 - CC-2619: Bugfix for append.late.data configuration rotating partitions
PR-196 - CC-2601: Add S3 Append-Late-Data option to configs
PR-187 - CC-2313: Handle late arriving records in storage cloud sink connectors

Version 5.0.0¶

PR-181 - MINOR: Update config options to include SSE-C
PR-173 - Extend the existing SSE support to enable Customer (provided) Keys SSE-C
PR-171 - MINOR: Remove MaxPermSize since we dropped support for Java 7
PR-160 - CC-1793: Used full jitter backoff strategy from AWS SDK.
PR-156 - CC-1497: Added logic for passing a subset of configs down to configurable credentials provider
PR-63 - DataUtils::getField returns field

Version 4.1.4¶

PR-309 - MINOR: Pin httpclient dependency to 4.5.9 to silence warning by the AWS SDK
PR-306 - MINOR: Upgrade AWS SDK to 1.11.725

Version 4.1.3¶

PR-200 - CC-2619: Bugfix for append.late.data configuration rotating partitions
PR-198 - Add autogenerated configs that were missed before
PR-196 - CC-2601: Add S3 Append-Late-Data option to configs
PR-187 - CC-2313: Handle late arriving records in storage cloud sink connectors

Version 4.1.2¶

No changes

Version 4.1.1¶

PR-155 - CC-918: Update build to generate plugin package
PR-159 - CC-1791: Added compression options to raw value example

Version 4.1.0¶

PR-150 - Make confluent CLI command output shared
PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
PR-149 - Bump Jackson version
PR-140 - Fix for BufferOverflowException in S3OutputStream
PR-106 - Add s3.sse.kms.key.id to specify a particular key in AWS KMS
PR-98 - Add gzip compression support for json and byte array formats
PR-143 - Support for multiple fields in field partitioner
PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
PR-123 - CC-1299: Refactor this code to use the AVRO_CODEC config key from storage-common
PR-114 - Update to PowerMock 2.0.x and Mockito 2
PR-48 - RecordField nested timestamp extraction
PR-57 - Issue-53. Multiple field support for partitioning.
PR-55 - CC-1489: NPE on records with null timestamp.
PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.
PR-52 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-51 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-50 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
PR-46 - CC-1299: Create AVRO_CODEC ConfigKey

Version 4.0.3¶

PR-200 - CC-2619: Bugfix for append.late.data configuration rotating partitions
PR-196 - CC-2601: Add S3 Append-Late-Data option to configs
PR-187 - CC-2313: Handle late arriving records in storage cloud sink connectors

Version 4.0.2¶

No changes

Version 4.0.1¶

PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
PR-149 - Bump Jackson version
PR-140 - Fix for BufferOverflowException in S3OutputStream
PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
PR-55 - CC-1489: NPE on records with null timestamp.
PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.

Version 4.0.0¶

PR-102 - Allow to write records at the root of the bucket
PR-105 - Add instructions about upstream builds
PR-97 - MINOR: S3 Proxy Settings revisited
PR-85 - Add ByteArrayFormat
PR-87 - Add s3.canned.acl config
PR-84 - Create CONTRIBUTING.md
PR-92 - CC-1114: Switch to common pom and fix checkstyle issues.
PR-78 - Retry s3 part upload
PR-80 - HOTFIX: Remove refs to schema-generator config.
PR-60 - Add logging when creating a new RecordWriter
PR-44 - HOTFIX: Remove unused parameter in newConfigDef for PartitionerConfig.
PR-41 - HOTFIX: Update dependencies
PR-40 - HOTFIX: Make specific dependencies explicit.
PR-35 - Add missing modules to the dependencyManagement pom section so downstream projects will inherit the right version automatically.
PR-31 - Remove schema.generator.class config and have Formats specify their own SchemaGenerator internally

Version 3.3.3¶

PR-200 - CC-2619: Bugfix for append.late.data configuration rotating partitions
PR-187 - CC-2313: Handle late arriving records in storage cloud sink connectors

Version 3.3.2¶

No changes

Version 3.3.1¶

PR-116 - HOTFIX: Add recommender for schema generator classes only on 3.3.x.
PR-112 - CC-1244: 3.3.1 release notes
PR-108 - HOTFIX: Handle primitive types in AvroFormat
PR-104 - Fix new client creation to cater for URLs and region together
PR-100 - CC-1167: Add recommenders for class type properties
PR-90 - Add upstream project so build are triggered automatically
PR-82 - CC-872: Update default part.size number in S3 docs and more.
PR-77 - Update quickstart to use Confluent CLI.
PR-43 - HOTFIX: Add recommender for schema generator classes only on 3.3.x.
PR-39 - HOTFIX: Check simple class names in recommender.
PR-38 - CC-1153: Isolate config instances and recommenders.
PR-36 - CC-1153: Storage connectors can add recommendations for Class types

Version 3.3.0¶

PR-28 - HOTFIX: Explicitly deactivate licensing profiles by default.
PR-34 - CC-524: Reduce the default s3.part.size to 25MB to avoid OOM with current default jvm heap size.
PR-33 - CC-513: Newline is not appended between records when using JsonFormat.
PR-37 - Enable compression of output Avro files.
PR-38 - CC-530: Exclude storage-common jars when packaging S3 connector.
PR-19 - Move to using io.confluent:common for deps.
PR-22 - HOTFIX: Import recent changes from the hdfs connector.
PR-24 - Convert - to _ in Hive table names.
PR-25 - Include causes with ConfigExceptions caused by catching other exceptions.
PR-26 - Add constructors to SchemaGenerators to support no parameters and a Map config.
PR-29 - Timestamp information should be copied to projected SinkRecord.

Version 3.2.4¶

No changes

Version 3.2.3¶

No changes

Version 3.2.2¶

PR-19 - CC-500: Provide exactly-once time-based partitioning in S3
PR-45 - HOTFIX: S3SinkConnector should extend SinkConnector
PR-51 - Allow custom partitioners to have their own configs
PR-27 - HOTFIX: Include a trailing delimiter when verifying data format for Hive
PR-23 - HOTFIX: Add test schema and record builder for records with timestamp field
PR-18 - CC-497: Add timestamp based partitioners.

Version 3.2.1¶

PR-33 - Separate JSON records using line separator instead of single white space.
PR-34 - Reduce the default s3.part.size to 25MB to avoid OOM exceptions with the current default java heap size settings for Connect.
PR-32 - Add s3.region property to quickstart config and docs.

Version 3.2.0¶

Initial Version