Changelog

Version 10.1.0

  • PR-574 - CCMSG-1133: Make hive table name configurable

Version 10.0.5

  • PR-568 - CC-14408: Fix empty context assignment in start()
  • PR-567 - CC-14520: Upgrade jackson-databind to 2.10.5.1 and hadoop to 2.10.1 and resolve CVEs

Version 10.0.4

  • PR-561 - CCMSG-1004: fix incorrect TopicPartitionWriter internal offset tracking
  • PR-558 - CCMSG-953: work around wal CannotObtainBlockLengthException

Version 10.0.3

  • PR-556 - CCMSG-670: Use WAL file as primary method for offset recovery

Version 10.0.2

  • PR-550 - CCMSG-852: Swallow exceptions and continue recovery when attempting to apply a corrupt WAL
  • PR-548 - CCMSG-857: Ensure consistent lease acquisition log level

Version 10.0.1

  • PR-542 - CCMSG-779: Update kafka-connect-storage-common-parent to 10.0.4 to resolve CVE
  • PR-541 - CCMSG-768: Fix concurrent modification exception when closing topic partition writers
  • PR-534 - CC-13111: Fix Kerberos tests interfering with non-Kerberos tests

Version 10.0.0

  • PR-529 - Import CVE-free htrace-core4 and avatica from kafka-connect-storage-common
  • PR-527 - Update transitive dependencies to fix CVEs
  • PR-526 - CC-12567: Fix duplicate columns in Hive for Parquet and Avro formats
  • PR-522 - Properly initialize AvroData by using config’s avroDataConfig method
  • PR-510 - Resolving netty CVEs
  • PR-507 - Upgrade hadoop version to address beanutils CVE
  • PR-501 - CC-7360: Recover gracefully if closing a temporary file fails
  • PR-506 - CC-9755: Validate timezone configuration with rotate.schedule.interval.ms
  • PR-503 - CC-9710: Retry effectively on exceptions from time-based commits
  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet
  • PR-497 - MINOR: add orc format to configs
  • PR-484 - CC-7648: add ORC format with Hive
  • PR-162 - CC-1360: Enable enhanced avro schema support by default in storage sink connectors
  • PR-151 - Correct SR schemas cache size property name for AvroDataConfig
  • PR-156 - Update commons codec to 1.15 to resolve CVE and exclude zookeeper test jar
  • PR-155 - Upgrade hadoop to 2.10.1 and hive to 2.3.7
  • PR-138 - CC-12005: Exclude vulnerable dependency jackson mapper asl
  • PR-149 - Upgrade parquet.version to 1.11.1
  • PR-139 - CC-8704: Exclude jetty from hcatalog - CVE
  • PR-136 - CC-9779: PartitionerConfig should perform basic timezone validation
  • PR-132 - CC-8816: Document more accurate timezone restrictions

Version 5.5.3

  • PR-530 - Fix typo in Javadoc
  • PR-522 - Fix #326 by using config’s avroDataConfig method

Version 5.5.2

  • PR-510 - Resolving netty CVEs
  • PR-507 - Upgrade hadoop version to address beanutils CVE
  • PR-501 - CC-7360: Recover gracefully if closing a temporary file fails
  • PR-506 - CC-9755: Validate timezone configuration with rotate.schedule.interval.ms
  • PR-503 - CC-9710: Retry effectively on exceptions from time-based commits

Version 5.5.1

  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet

Version 5.5.0

  • PR-491 - CC-8229: HDFS2 format options are not showing all available options
  • PR-477 - CC-7247: allow topics.dir config to be more configurable

Version 5.4.3

  • PR-510 - Resolving netty CVEs
  • PR-507 - Upgrade hadoop version to address beanutils CVE
  • PR-501 - CC-7360: Recover gracefully if closing a temporary file fails
  • PR-506 - CC-9755: Validate timezone configuration with rotate.schedule.interval.ms
  • PR-503 - CC-9710: Retry effectively on exceptions from time-based commits

Version 5.4.2

  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet
  • PR-491 - CC-8229: HDFS2 format options are not showing all available options

Version 5.4.1

No changes

Version 5.4.0

  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config
  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector
  • PR-450 - CC-5969: Pass extra connector configs to the partitioner
  • PR-442 - Adding debug logs for WAL recovery
  • PR-440 - Remove jackson databind version override
  • PR-114 - CC-147: logical types support for HiveSchemaConverter
  • PR-116 - CC-6565: Add parquet.codec property to enable compression in storage sink connectors
  • PR-110 - CC-5963: Improved ability to handle schema evolution with different names

Version 5.3.4

  • PR-501 - CC-7360: Recover gracefully if closing a temporary file fails
  • PR-506 - CC-9755: Validate timezone configuration with rotate.schedule.interval.ms
  • PR-503 - CC-9710: Retry effectively on exceptions from time-based commits
  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet
  • PR-491 - CC-8229: HDFS2 format options are not showing all available options

Version 5.3.3

No changes

Version 5.3.2

  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config

Version 5.3.1

  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector

Version 5.3.0

No changes

Version 5.2.4

  • PR-491 - CC-8229: HDFS2 format options are not showing all available options
  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config
  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector

Version 5.2.3

No changes

Version 5.2.2

No changes

Version 5.2.1

No changes

Version 5.2.0

  • PR-422 - CC-4124: Clear partition assignment on close
  • PR-417 - CC-3950: Change the HDFS sink task to return to Connect offsets for records committed to HDFS

Version 5.1.4

  • PR-491 - CC-8229: HDFS2 format options are not showing all available options
  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config
  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector

Version 5.1.3

No changes

Version 5.1.2

No changes

Version 5.1.1

No changes

Version 5.1.0

No changes

Version 5.0.4

  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet
  • PR-491 - CC-8229: HDFS2 format options are not showing all available options
  • PR-460 - CC-6423: Disable FileSystem Object Caching
  • PR-456 - CC-6196: fix NPE for store.url config
  • PR-451 - CC-6089: Improve fault tolerance of HDFS sink connector

Version 5.0.3

No changes

Version 5.0.2

No changes

Version 5.0.1

No changes

Version 5.0.0

  • PR-63 - DataUtils::getField returns field

Version 4.1.4

  • PR-496 - CC-8809: Replace assignment with topicPartitionWriters.keySet
  • PR-456 - CC-6196: fix NPE for store.url config

Version 4.1.3

No changes

Version 4.1.2

  • PR-335 - MINOR: Fix the secure HDFS test

Version 4.1.1

  • PR-307 - CC-918 Add Kafka Connect Maven plugin to build

Version 4.1.0

  • PR-299 - CC-1669: Set no offset when no offset was found for topic partition
  • PR-293 - Explicitly specify Jackson version instead of relying on transitive dependency version.
  • PR-283 - Adding StringFormat to available formatters
  • PR-286 - Support for multiple fields in field partitioner
  • PR-269 - Fix or suppress findbugs errors
  • PR-255 - CC-1299: Add configuration setting for Avro compression codec
  • PR-48 - RecordField nested timestamp extraction
  • PR-57 - Issue-53. Multiple field support for partitioning.
  • PR-55 - CC-1489: NPE on records with null timestamp.
  • PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.
  • PR-52 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
  • PR-51 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
  • PR-50 - CC-1333: Remove dependency on hive-exec and instead use hive-exec:core
  • PR-46 - CC-1299: Create AVRO_CODEC ConfigKey

Version 4.0.3

No changes

Version 4.0.2

No changes

Version 4.0.1

  • PR-305 - CC-1669: Disable commit of consumer offsets
  • PR-299 - CC-1669: Set no offset when no offset was found for topic partition
  • PR-293 - Explicitly specify Jackson version instead of relying on transitive dependency version.
  • PR-55 - CC-1489: NPE on records with null timestamp.
  • PR-45 - HOTFIX: Consider all the properties in AvroDataConfig.

Version 4.0.0

  • PR-251 - CC-1270: Reinstantiate HiveConfig properties in getConfig.
  • PR-247 - MINOR: Exclude storage-common dependencies.
  • PR-246 - CC-1213: Enable record based timebased partitioners in HDFS connector
  • PR-245 - CC-1212: Accept partitioners implementing the new partitioner interface
  • PR-243 - CC-1168: Recommenders for HDFS connector’s class type properties
  • PR-237 - Remove unused imports
  • PR-196 - CC-492: Add JsonFormat to HDFS Connector
  • PR-142 - not trying to hold leases on WAL files if we are holding them already.
  • PR-220 - HOTFIX: Enable default for format and storage classes in hdfs connector
  • PR-219 - Move to common pom
  • PR-188 - CC-397: Refactoring on top of storage-common
  • PR-214 - fix TimeBasedPartitionerTest by defining the expected locale
  • PR-212 - Remove invalid offset check.
  • PR-44 - HOTFIX: Remove unused parameter in newConfigDef for PartitionerConfig.
  • PR-41 - HOTFIX: Update dependencies
  • PR-40 - HOTFIX: Make specific dependencies explicit.
  • PR-37 - Remove unused imports
  • PR-35 - Add missing modules to the dependencyManagement pom section so downstream projects will inherit the right version automatically.
  • PR-31 - Remove schema.generator.class config and have Formats specify their own SchemaGenerator internally

Version 3.3.3

No changes

Version 3.3.2

No changes

Version 3.3.1

  • PR-235 - CC-1172: Fix memory leak in TopicPartitionerWriter
  • PR-227 - Specify a nodeLabel for Jenkins that doesn’t die during the HDFS connector tests.
  • PR-226 - Add upstream project so builds will be triggered automatically
  • PR-217 - Update quickstart to use Confluent CLI

Version 3.3.0

  • PR-187 - CC-491: Consolidate and simplify unit tests of HDFS connector.
  • PR-205 - Upgrade avro to 1.8.2.

Version 3.2.4

No changes

Version 3.2.3

No changes

Version 3.2.2

  • PR-194 - Fix HdfsSinkConnector to extend from SinkConnector instead of Connector.
  • PR-200 - Fix incorrect licensing and webpage info.

Version 3.2.1

No changes

Version 3.2.0

  • PR-135 - Fix typos
  • PR-164 - Issue 136 - Support topic with dots in hive.
  • PR-170 - MINOR: Upgrade Hadoop version to 2.7.3 and joda-time to 2.9.7

Version 3.1.2

No changes

Version 3.1.1

No changes

Version 3.1.0

  • PR-134 - Flush the last partial file when incoming stream is paused.
  • PR-133 - CC-331: Update config options docs
  • PR-126 - Fix TimeBasedPartitioner config validation
  • PR-112 - Lint change to avoid compiler error in Oracle JDK 1.7 using jenv.
  • PR-94 - Fix lint annoyances
  • PR-108 - Revert “support multi partition fields.”
  • PR-105 - support multi partition fields.
  • PR-101 - Added link to Confluent documentation for the connector.
  • PR-92 - Start a new WAL file after truncate instead of appending to log.1
  • PR-87 - Scheduled rotation implementation
  • PR-90 - Use configured Hadoop configuration object for Parquet writer
  • PR-91 - Upgrade to Hadoop 2.6.1
  • PR-70 - Fix handling of topics with periods
  • PR-68 - prints details of HDFS exceptions
  • PR-67 - clean up hive metastore artifacts from testing
  • PR-64 - cleaned up .gitignore. Now ignores Eclipse files

Version 3.0.1

HDFS Connector

  • PR-82 - add version.txt to share/doc

Version 3.0.0

HDFS Connector

  • PR-62 - Update doc for Confluent Platform 3.0.
  • PR-60 - Remove HDFS connectivity check.
  • PR-55 - Removing retry logic from HiveMetaStore to fix the metastore connection bloat.
  • PR-50 - Remove close of topic partition writers in DataWriter close.
  • PR-42 - Using new config validation.
  • PR-41 - Bump version to 3.0.0-SNAPSHOT and Apache Kafka® dependency to 0.10.0.0-SNAPSHOT.
  • PR-35 - Minor doc typo fix TimeBasedPartitioner.
  • PR-33 - Minor doc fix.

Version 2.0.1

No changes

Version 2.0.0

Initial version