Changelog¶
Version 10.0.9¶
No changes
Version 10.0.6¶
- PR-497 - Fix regions list being one item of a list
- PR-495 - Don’t log full proxy config and don’t log configuration at info level
- PR-494 - Generalize S3 record writes with wrapper to possibly convert to RetriableException
- PR-492 - CCMSG-1481: Retriable exception is possible to be thrown from
close()
- PR-461 - Fix retries for S3 transient errors
- PR-490 - MINOR: Suppress warning which causes IntelliJ to error after Maven import (#489)
- PR-485 - CCMSG-1531: Support null items within arrays with Parquet writer
Version 10.0.5¶
Version 10.0.4¶
Version 10.0.3¶
Version 10.0.2¶
Version 10.0.1¶
- PR-441 - CCMSG-1146: Upgrade json-smart to 2.4.7
- PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive
- PR-432 - CC-14987: Fix serialization when headers.format.class is json
- PR-429 - Upgrade jackson-dataformat-cbor to 2.11.4
- PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
- PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
- PR-394 - Upgrade to hadoop common 2.10.1 for parquet support
Version 10.0.0¶
- PR-393 - MINOR: Upgrade storage common to 10.0.5 and catch PartitionException in TopicPartitionWriter’s error handling
- PR-392 - CCMSG-861: Catch additional exception to report in DLQ
- PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
- PR-349 - CC-9723: DLQ support in S3 sink
- PR-379 - CC-12772: Upgrade testcontainers to 1.15.0 to address commons-compress CVE
- PR-377 - CC-12825: Remove unintentional proprietary dependencies
- PR-378 - CC-12695: Stop throwing retriable exceptions on non-retriable S3 errors
- PR-325 - CC-7933: Support writing key and headers to S3
- PR-262 - MINOR: Restore the complete hierarchy of exceptions during failures in part upload
- PR-356 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3
- PR-346 - SEC-1034: log4j migration to confluent repackaged version
- PR-340 - CC-9753: Validate timezone configuration with rotate.schedule.interval.ms
- PR-328 - CC-9198 Move
null
value handling earlier in SinkTask put - PR-291 - S3 Object Level Tagging for Offsets & Record Counts
Version 5.5.10¶
No changes
Version 5.5.9¶
No changes
Version 5.5.8¶
Version 5.5.6¶
No changes
Version 5.5.5¶
- PR-394 - Upgrade to hadoop common 2.10.1 for parquet support
- PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
- PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
- PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
Version 5.5.4¶
Version 5.5.3¶
Version 5.5.2¶
Version 5.5.0¶
- PR-318 - CC-8170: Exclude
jackson-mapper-asl
andcommons-beanutils
from transient dependencies of hadoop-common - PR-310 - MINOR: Fix log message when tagging an S3 object
- PR-309 - MINOR: Pin
httpclient
dependency to 4.5.9 to silence warning by the AWS SDK - PR-311 - Add AWS IAM AssumeRole credentials provider
- PR-305 - CC-7892: Introduce configuration properties for key and secret to easily define credentials per connector instance
- PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
- PR-291 - S3 object level tagging for offsets and record counts
- PR-306 - MINOR: Upgrade AWS SDK to 1.11.725
Version 5.4.8¶
No changes
Version 5.4.7¶
- PR-473 - CCMSG-1582 Fix bug that generates a small files when scheduled rotation has expired
Version 5.4.5¶
Version 5.4.4¶
- PR-414 - CCMSG-1021: Upgrade AWS SDK to 1.11.1015 and exclude jackson-dataformat-cbor
- PR-399 - SEC-1996: Replace commons-httpclient with apache httpclient
- PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
- PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors
Version 5.4.3¶
Version 5.4.2¶
- PR-318 - CC-8170: Exclude
jackson-mapper-asl
andcommons-beanutils
from transient dependencies of hadoop-common - PR-309 - MINOR: Pin
httpclient
dependency to 4.5.9 to silence warning by the AWS SDK - PR-311 - Add AWS IAM AssumeRole credentials provider
- PR-308 - CC-8085: Allow setting compression level for gzip in Json and ByteArray formats
Version 5.4.0¶
- PR-289 - MINOR: Upgrade bugfix versions of dependencies to avoid CVEs
- PR-286 - MINOR: Pin hadoop-common version to 2.7.7 and exclude htrace-core dependency
- PR-282 - CC-6565: Use parquet.codec and add compression options for Parquet format
- PR-281 - CC-6801: Retry effectively on retriable failures while closing files
- PR-280 - CC-6766: Handle null-valued records in all three output types
- PR-241 - Add Parquet Format
- PR-264 - MINOR: New log for SSE
- PR-263 - MINOR: Remove jackson databind version override
- PR-250 - CC-4969 MINOR: Remove Unused S3Storage object from TopicPartitionWriter
- PR-114 - CC-147: logical types support for HiveSchemaConverter
- PR-116 - CC-6565: Add parquet.codec property to enable compression in storage sink connectors
- PR-110 - CC-5963: Improved ability to handle schema evolution with different names
Version 5.3.8¶
No changes
Version 5.3.7¶
No changes
Version 5.3.6¶
No changes
Version 5.3.5¶
- PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
- PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
- PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
- PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors
Version 5.3.4¶
- PR-357 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3 (back port to 5.0.x)
Version 5.3.3¶
Version 5.3.2¶
No changes
Version 5.3.1¶
No changes
Version 5.3.0¶
Version 5.2.7¶
No changes
Version 5.2.6¶
No changes
Version 5.2.5¶
- PR-432 - MINOR: Remove unused dependency on kafka-connect-storage-hive in the 5.2.x release branch
- PR-429 - Pin version of jackson-dataformat-cbor to 2.11.4
- PR-391 - CC-13830: Upgrade S3 connector to use HttpClient defined in kafka-connect-storage-common
- PR-378 - CC-12695: stop throwing retriable exceptions on non-retriable S3 errors
- PR-357 - CC-11480: Add configuration property to choose between path style and virtual hosted bucket access to S3 (back port to 5.0.x)
Version 5.2.4¶
Version 5.2.3¶
No changes
Version 5.2.2¶
No changes
Version 5.2.1¶
No changes
Version 5.2.0¶
- PR-220 - CC-3035: Upgrade the S3Mock library so the module builds on Java 8 and Java 11
Version 5.1.4¶
Version 5.1.3¶
No changes
Version 5.1.2¶
No changes
Version 5.1.1¶
No changes
Version 5.1.0¶
No changes
Version 5.0.4¶
Version 5.0.3¶
No changes
Version 5.0.2¶
No changes
Version 5.0.1¶
- PR-214 - CC-2617: Accept timestamp for wallclock instances to assure consistency
- PR-200 - CC-2619: Bugfix for
append.late.data
configuration rotating partitions - PR-196 - CC-2601: Add S3 Append-Late-Data option to configs
- PR-187 - CC-2313: Handle late arriving records in storage cloud sink connectors
Version 5.0.0¶
- PR-181 - MINOR: Update config options to include SSE-C
- PR-173 - Extend the existing SSE support to enable Customer (provided) Keys SSE-C
- PR-171 - MINOR: Remove MaxPermSize since we dropped support for Java 7
- PR-160 - CC-1793. Used full jitter backoff strategy from AWS SDK.
- PR-156 - CC-1497. Added logic for passing a subset of configs down to configurable credentials provider
- PR-63 - DataUtils::getField returns field
Version 4.1.4¶
Version 4.1.3¶
Version 4.1.2¶
No changes
Version 4.1.1¶
Version 4.1.0¶
- PR-150 - Make confluent CLI command output shared
- PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
- PR-149 - Bump Jackson version
- PR-140 - Fix for BufferOverflowException in S3OutputStream
- PR-106 - Add s3.sse.kms.key.id to specify a particular key in AWS KMS
- PR-98 - Add gzip compression support for json and byte array formats
- PR-143 - Support for multiple fields in field partitioner
- PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
- PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
- PR-123 - CC-1299: Refactor this code to use the AVRO_CODEC config key from storage-common
- PR-114 - Update to PowerMock 2.0.x and Mockito 2
- PR-48 - RecordField nested timestamp extraction
- PR-57 - Issue-53. Multiple field support for partitioning.
- PR-55 - CC-1489: NPE on records with null timestamp.
- PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.
- PR-52 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
- PR-51 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
- PR-50 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
- PR-46 - CC-1299: Create AVRO_CODEC ConfigKey
Version 4.0.3¶
Version 4.0.2¶
No changes
Version 4.0.1¶
- PR-148 - Rely on jackson.version from common pom instead of defining it ourselves
- PR-149 - Bump Jackson version
- PR-140 - Fix for BufferOverflowException in S3OutputStream
- PR-138 - CC-1490: NPE when rotate.interval.ms is set but partitioner is not instance of TimeBasedPartitioner
- PR-132 - CC-1214: Import all the AvroData configs to storage sink connectors
- PR-55 - CC-1489: NPE on records with null timestamp.
- PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.
Version 4.0.0¶
- PR-102 - Allow to write records at the root of the bucket
- PR-105 - Add instructions about upstream builds
- PR-97 - MINOR: S3 Proxy Settings revisited
- PR-85 - Add ByteArrayFormat
- PR-87 - Add s3.canned.acl config
- PR-84 - Create CONTRIBUTING.md
- PR-92 - CC-1114: Switch to common pom and fix checkstyle issues.
- PR-78 - Retry s3 part upload
- PR-80 - HOTFIX: Remove refs to schema-generator config.
- PR-60 - Add logging when creating a new RecordWriter
- PR-44 - HOTFIX: Remove unused parameter in newConfigDef for PartitionerConfig.
- PR-41 - HOTFIX: Update dependencies
- PR-40 - HOTFIX: Make specific dependencies explicit.
- PR-35 - Add missing modules to the dependencyManagement pom section so downstream projects will inherit the right version automatically.
- PR-31 - Remove
schema.generator.class
config and have Formats specify their own SchemaGenerator internally
Version 3.3.3¶
Version 3.3.2¶
No changes
Version 3.3.1¶
- PR-116 - HOTFIX: Add recommender for schema generator classes only on 3.3.x.
- PR-112 - CC-1244: 3.3.1 release notes
- PR-108 - HOTFIX: Handle primitive types in AvroFormat
- PR-104 - Fix new client creation to cater for URLs and region together
- PR-100 - CC-1167: Add recommenders for class type properties
- PR-90 - Add upstream project so build are triggered automatically
- PR-82 - CC-872: Update default part.size number in S3 docs and more.
- PR-77 - Update quickstart to use Confluent CLI.
- PR-43 - HOTFIX: Add recommender for schema generator classes only on 3.3.x.
- PR-39 - HOTFIX: Check simple class names in recommender.
- PR-38 - CC-1153: Isolate config instances and recommenders.
- PR-36 - CC-1153: Storage connectors can add recommendations for Class types
Version 3.3.0¶
- PR-28 - HOTFIX: Explicitly deactivate licensing profiles by default.
- PR-34 - CC-524: Reduce the default
s3.part.size
to 25MB to avoid OOM with current default jvm heap size. - PR-33 - CC-513: Newline is not appended between records when using JsonFormat.
- PR-37 - Enable compression of output Avro files.
- PR-38 - CC-530: Exclude storage-common jars when packaging S3 connector.
- PR-19 - Move to using io.confluent:common for deps.
- PR-22 - HOTFIX: Import recent changes from the hdfs connector.
- PR-24 - Convert - to _ in Hive table names.
- PR-25 - Include causes with ConfigExceptions caused by catching other exceptions.
- PR-26 - Add constructors to SchemaGenerators to support no parameters and a Map config.
- PR-29 - Timestamp information should be copied to projected SinkRecord.
Version 3.2.4¶
No changes
Version 3.2.3¶
No changes
Version 3.2.2¶
- PR-19 - CC-500: Provide exactly-once time-based partitioning in S3
- PR-45 - HOTFIX: S3SinkConnector should extend SinkConnector
- PR-51 - Allow custom partitioners to have their own configs
- PR-27 - HOTFIX: Include a trailing delimiter when verifying data format for Hive
- PR-23 - HOTFIX: Add test schema and record builder for records with timestamp field
- PR-18 - CC-497: Add timestamp based partitioners.
Version 3.2.1¶
Version 3.2.0¶
Initial Version