Confluent Platform Component Changelogs

This topic provides changelogs for the individual Confluent Platform components.

Control Center Changelog

Version 5.3.0

  • Redesigned the Control Center UI, including KSQL and Connect enhancements. New pages show cluster overview, topics pages, and consumers index.
  • RBAC enforcement.
  • Topic message browser enhancements, including jump to timestamp and download results in JSON format.
  • Alert enhancements that added Slack and PagerDuty webhook notifications, a new Consumer lead alert, pause and resume global alerts actions, and fast navigation to Alerts features from the Alerts bell icon.

Version 5.2.0

  • Added multiple Connect support.
  • Added multiple KSQL support.
  • Added broker configs editing.
  • Added Schema Registry editing.
  • Added license management UI.
  • Improved message browsing.
  • Improved KSQL UI.
  • Improved scalability.

Version 5.1.0

  • Browser support for Chrome, Firefox, and Safari. For more information, see Web Browsers.
  • A new, more readable light theme.
  • Many bug fixes and UX improvements.
  • Added configuration confluent.controlcenter.license.manager=_confluent-controlcenter-license-manager-5-1-0. (It was confluent.controlcenter..license.manager = _confluent-controlcenter-license-manager5-1-0 in first 5.1.x release, but it is fixed now.) No need for customers to override this configuration.

Version 5.0.0

  • Added KSQL interface to create/view streams, tables, and queries.
  • Added topic inspection to read messages from a topic.
  • Added consumer group message lag viewing and alerting.
  • Added Schema Registry viewing.
  • Added read only view of broker configurations.
  • Added feature flags for variety of user accessible features.

Version 4.1.0

  • Added ability to create and edit topics in Control Center.
  • Added topic filtering and ability to hide Apache Kafka®/Confluent Platform internal topics from the UI.
  • Added new alert to warn when Control Center stops receiving data from a Kafka cluster.
  • Added a restricted user type that prevents editing or creating of topics, alerts, or connectors.

Version 4.0.1

  • Made calls to Connect more resilient.
  • Fixed an issue that would prevent chart highlighting on System Health charts.
  • Fixed an issue that could cause Control Center to improperly warn about improper log dirs on brokers.
  • Fixed an issue that could cause Control Center to improperly alert about active controller count.

Version 4.0.0

  • Fixed an issue that could cause long request times if a cluster had brokers with noncontiguous IDs.
  • Additional logging during startup.
  • Fixed an issue that could cause Control Center to show blank graphs when data was present.
  • Set and enforce TimestampType for internal Control Center topics at startup.
  • Fixed an issue that caused us to link to an invalid consumer group on Streams Monitoring.
  • Fixed an issue where Broker Monitoring could check for data last update time on wrong cluster.
  • Allow overriding ConfluentMetricsReporter default producer settings.
  • Allow using a file for confluent.license config.
  • Docker image: allow passing license using CONTROL_CENTER_CONFLUENT_LICENSE environment variable.
  • Fixed an issue that resulted in Offline and Under Replicated Partitions data to be swapped.
  • Fixed an issue that could cause Control Center to shutdown if it lost connection to Kafka.
  • Fixed an issue that could cause Control Center to hang if it encountered an error creating topics on startup.

Version 3.3.1 (EOSL)

  • Allow text input in date picker.
  • Fixed an issue that could cause unsubmittable forms.
  • Show renamed cluster value rather than base clusterid on alerts page.
  • Added alert banner on Broker Monitoring to indicate a cluster may be down.
  • Scalability increase for Broker Monitoring to reduce number of messages sent to Kafka.
  • Fixed an issue where 100% Network Pool Usage was displayed instead of 0%.
  • Allow graceful shutdown of Control Center during startup.
  • Fixed a race condition that could happen if you start Control Center and your Kafka broker at the same time.

Version 3.3.0 (EOSL)

  • Sped up restore time.
  • Reduced the size of old data over long periods of time.
  • Upgraded to use new admin client (KIP-117).
  • Alert UI should render better alert information.
  • Triggers are createable directly though the UI on System Health.
  • Massive performance improvements to Stream Monitoring UI.
  • Fixed various time range display/selection bugs; persist time ranges though navigation.
  • Nonconfigurable form values being sent to Connect via UI should remain and be visible prior to submit.

Version 3.2.2 (EOSL)

  • Added shutdown script.
  • Force faster compaction and cleanup on changelog topics.
  • Several Streams improvements to reduce state-store restoration time after unclean shutdown.
  • Reduce the volume of data to be restored to state-stores on startup.
  • Fixed possible concurrency problem in setting RocksDB configuration options.
  • Fixed possible deadlock on exception after streams startup.

Version 3.2.1 (EOSL)

  • Enabled HTTP compression by default.
  • Fixed possible deadlock on streams exception.
  • Fixed fd leak when communicating with Kafka.
  • Update defaults for monitoring producers to be more resilient.
  • Prevent inadvertently creating misconfigured topics during startup during a failure.
  • Check Kafka version at startup.
  • Reduce minimum startup time.
  • More efficient duplicate checking to reduce write amplification.
  • Handle ‘hidden’ connect configs. This allows Control Center to manage Connectors that do not fully enumerate their configs.
  • Ensure users only see invalid license warning if their license is invalid.
  • Fixed topic display bug that would prevent topic details from showing.
  • Reduced logging in js.
  • Allow users to close missing data modal dialogs.
  • Better rendering of System Health metrics especially when some data is missing.
  • Better validation of triggers, actions, and connector configs.
  • Faster processing of trigger and action creation and update.
  • Corrected some misleading labels on System Health.
  • Better error handling around Connect communications.
  • Change default timespan in UI to 4 hours.
  • Check and warn about errors on Topic Management page.

Version 3.2.0 (EOSL)

  • Add capabilities to monitor brokers instrumented with Confluent Metrics Reporter.
  • Enable alerting on broker and topic metrics.
  • Add UI for viewing topic information.
  • Add status UI for tracking internal progress.
  • Fixed bug where we could inadvertently show license expired message.
  • Fixed bugs with alert trigger creation in UI.
  • Fixed issue with parsing certain error conditions from Connect.

Version 3.1.0 (EOSL)

  • Enable gathering stream monitoring data from multiple clusters.
  • Support alerting on stream monitoring conditions.
  • Ability to restrict UI access via user login.

Version 3.0.1 (EOSL)

  • Improved data fetching from the browser.
  • Handle authenticated/authorized Kafka clusters.
  • Bug fix for running in daemon mode.
  • More efficient topic creation at startup.
  • Reduced number of necessary topics/rocksdbs for aggregations.
  • Reduced total amount of stored data.
  • Fixed timing issues when receiving delayed data.
  • Configs updates to prefix Kafka/rest configs.
  • Added rate limit for logging to prevent logsize blowup.
  • Allow deleting connector in UI.

Confluent REST Proxy Changelog

Version 5.1.0

  • PR-511 - CPKAFKA-1876: Log cause of failure to produce messages in tests
  • PR-512 - Fix race condition when expiration thread iterates over partially-initialized consumer
  • PR-503 - Fix location of jars for dev testing
  • PR-481 - CPKAFKA-1681: Add long polling functionality
  • PR-508 - CPKAFKA-1868: Add more consumer tests
  • PR-507 - CPKAFKA-1871: Fix problem where ReadTaskSchedulerThread would loop too often
  • PR-504 - CPKAFKA-1865: Performance degrades drastically after running for extended periods
  • PR-495 - CPKAFKA-1794: Assign leader ID -1 on offline partitions in the topic describe API
  • PR-477 - Fix SimpleConsumerManager properly return requested offsets for compressed topic
  • PR-474 - MINOR: Fix checkstyle path
  • PR-466 - Change Consumer threading model to better handle rebalances
  • PR-459 - MINOR: Add utility createConsumerProperties() test method since it was removed from AK
  • PR-452 - Fix checkstyle to pass on 8.5 after upgrade from 6.19
  • PR-439 - Temporarily pin Kafka and Confluent versions
  • PR-438 - Don’t use deprecated RecordMetadata constructor

Version 5.0.0

  • PR-439 - Temporarily pin Kafka and Confluent versions
  • PR-438 - Don’t use deprecated RecordMetadata constructor
  • PR-434 - Fix usage of internal Jersey utilities that no longer exist
  • PR-428 - Use ExplicitGCInvokesConcurrent instead of disabling explicit GC

Version 3.3.0

  • PR-305 - Use max.block.ms instead of metadata.fetch.timeout.ms since the latter was deprecated and now removed in KAFKA-3763
  • Ability to add resources like filter through a custom extension and provide a custom context
  • PR-296 - Interceptor configuration options

Version 3.2.2

  • PR-303 - CLIENTS-304: Fix AvroConsumerRecord constructors and handling of topic for v1 consumer API.

Version 3.2.1

  • PR-285 - CLIENTS-281: Fix KafkaConsumerReadTask’s locking, handling of multiple polls, invalid handling of old ConsumerTimeout exceptions, and incorrect generic type parameters.
  • PR-277 - Documentation fixes

Version 3.2.0

  • PR-256 - Update ClusterTestHarness to use o.a.k.common.utils.Time.
  • PR-255 - clarified the key/value format limitations based on user questions
  • PR-264 - Fix build to work post KIP-103
  • PR-265 - Follow up for KIP-103 changes that fixes ProducerPool’s extraction of endpoints to use the security protocol in the URL instead of the listener name.
  • PR-268 - modified readme to use v2. also bumped installation up for usability
  • PR-271 - Updated quickstart and configuration docs for V2 and for security
  • PR-258 - REST Proxy security
  • PR-274 - added field ‘topic’ to tools/ConsumerPerformance.java for system test
  • PR-272 - added description of JVM parameters for SASL configuration

Version 3.1.2

No changes

Version 3.1.1

No changes

Version 3.1.0

  • PR-239 - Require bash since we use some bashisms and fix a copyright.
  • PR-235 - fix typo command line options -help check
  • PR-222 - fixing issue #91
  • PR-230 - Issue #229: Add blurb on Jetty jmx metrics
  • PR-214 - Small readme.md fix
  • PR-127 - Fix to ConsumerManager tests AND then handling of max bytes
  • PR-203 - Fix a typo that cause non-corresponding logger name
  • PR-202 - fix the implementation of topicExists

Version 3.0.0

  • PR-164 - 2.x merge to master
  • PR-166 - Bump version to 3.0.0-SNAPSHOT and Kafka dependency to 0.10.0.0-SNAPSHOT
  • PR-171 - Fix build to handle rack aware changes in Kafka.
  • PR-174 - Update CoreUtils usage to match kafka updates
  • PR-189 - Minor fixes for compatibility with newest 0.10.0 branch.
  • PR-192 - Minor fixes for compatibility with newest 0.10.0 branch.
  • PR-202 - fix the implementation of topicExists
  • PR-205 - Rearrange quickstart and use topics from earlier steps in requests for metadata so the example output will exactly match real output when starting from an empty cluster.

Version 2.0.1

  • PR-144 - Fix for Race condition when consuming while topic is being created (Issue #105)
  • PR-158 - Update Kafka version to 0.9.0.1-cp1

Version 2.0.0

  • PR-64 - Reduce integration test time.
  • PR-66 - Add support for SimpleConsumer-like access (Issue #26)
  • PR-67 - Handle conflicting IDs and separate IDs used in the REST Proxy and by Kafka’s consumer implementation.
  • PR-78 - Remove kafka from list of production directories to include in CLASSPATH.
  • PR-89 - JSON message support
  • PR-96 - Fixed log4j and daemon flag bugs in kafka-rest-run-class based on fix from schema-registry.
  • PR-99 - Require Java 7
  • PR-101 - rest-utils updates
  • PR-103 - Issue 94 rename main
  • PR-108 - Clarify partitioning behavior for produce requests
  • PR-117 - Update to Kafka 0.9.0.0-SNAPSHOT and make adjustments to work with updated ZkUtils.
  • PR-122 - Use x.y.z versioning scheme (i.e. 2.0.0-SNAPSHOT)
  • PR-123 - Updated args for JaasUtils.isZkSecurityEnabled()
  • PR-125 - Use Kafka compiled with Scala 2.11

KSQL Changelog

Version 5.3.0

KSQL 5.3.0 includes new features, including:

  • Drop the requirement that CREATE TABLE statements must have a KEY set in their WITH clause. This is now an optional optimization to avoid unnecessary repartition steps. See Github issue #2745 for more info.
  • Improved handling of KEY fields. The KEY field is an optional copy of the Kafka record’s key held within the record’s value. Users can supply the name of the field that holds the copy of the key within the WITH clause. The improved handling may eliminate unnecessary repartition steps in certain queries. Please note that preexisting persistent queries, e.g. those created via CREATE TABLE AS SELECT ... or CREATE STREAM AS SELECT ... or INSERT INTO ..., will continue to have the unnecessary repartition step. This is required to avoid the potential for data loss should this step be dropped. See Github issue #2636 for more info.
  • INSERT INTO ... VALUES is now supported, with standard SQL syntax to insert rows to existing KSQL streams/tables. To disable this functionality, set ksql.insert.into.values.enabled to false in the server properties.
  • CREATE STREAM and CREATE TABLE will now allow you to create the topic if it is missing. To do this, specify the PARTITIONS and optionally REPLICAS in the WITH clause.
  • CREATE STREAM AS SELECT ... and CREATE TABLE AS SELECT ... statements now use the source topic partitions and replica counts for the sink topic if the config properties are not set in the WITH clause. In case of JOIN, the left hand side topic is used. This deprecates ksql.sink.partitions and ksql.sink.replicas config properties.
  • A new config variable, ksql.internal.topic.replicas, was introduced to set the replica count for the internal topics created by KSQL Server. The internal topics include the command topic and the config topic.
  • A new KSQL testing tool was added. The tool is a command line utility that enables testing KSQL statements without requiring any infrastructure, like Apache Kafka and KSQL clusters.

KSQL 5.3.0 includes bug fixes, including:

  • The ROWTIME of the row generated when a JOIN encounters late data was previous the ROWTIME of the late event, where as now it is the max of ROWTIME of the rows involved in the join. This provides more deterministic join semantics.

  • Return values of UDF and UDAFs are now correctly marked as optional, where previously there was potential for non-optional fields, which would result in serialization issues in the presence of null values.

    This is a forward compatible change in Avro, i.e. after upgrading, KSQL will be able to read old values using the new schema. However, it is important to ensure downstream consumers of the data are using the updated schema before upgrading KSQL, as otherwise deserialization may fail. The updated schema is best obtained from running the query in another KSQL cluster, running version 5.3.

    See Github issue #2769 for more info.

KSQL 5.3 includes bug fixes, including:

  • Fixed issues with using AS keyword when aliasing sources. See #2732 for more info.

Version 5.2.0

KSQL 5.2 includes new features, including:

  • Support for HTTPS.
  • Support for CASE expression: KSQL now supports CASE conditional expression in Searched form where KSQL evaluates each condition from left to right. It returns the result for the first condition that evaluates to true. If no condition evaluates to true, the result for the ELSE clause will be returned. If there is no ELSE clause, null is returned.
  • A new family of UDFs for improved handling of URIs (e.g. extracting information/decoding information), see UDF table for all URL functions
  • LIMIT keyword support for PRINT (#1316)
  • Support for read-after-write consistency: new commands don’t execute until previous commands have finished executing. This feature is enabled by default in the CLI (#2280) and can be implemented by the user for the REST API (Coordinate Multiple Requests).
  • A log of record processing events to help users debug their KSQL queries. The log can be configured to log to Kafka to be consumed as a KSQL stream. See KSQL processing log for more details.
  • Aggregation functionality has been extended. KSQL now supports:
    • GROUP BY more than just simple columns, including fields within structs, arithmetic results, functions, string concatenations and literals.
    • literals in the projection, (a.k.a the SELECT clause).
    • Multiple HAVING clauses, including the use of aggregate functions and literals.
  • Automatic compatibility management for queries in headless mode across versions. Starting with 5.2, KSQL will automatically take care of ensuring query compatibility when upgrading. This means you won’t need to worry about setting properties correctly during upgrade, as has been required for previous upgrades. Refer to the architecture documentation for details. Note that it is still up to the user to set properties correctly before upgrading to 5.2. The upgrade doc has details about the properties required to safely upgrade to 5.2.

KSQL 5.2 includes bug fixes, including:

  • Improved support for multi-line requests in interactive mode deployments. Prior to version 5.2 KSQL parsed the full request before attempting to execute any statements. Requests that contained later statements that were dependent the execution of prior statements may have failed. In version 5.2 and later, this is no longer an issue.
  • Improved support for non-interactive, “headless” mode deployments. Prior to version 5.2 KSQL parsed the full script before attempting to execute any statements. The full parse would often fail when later statements relied on the execution of earlier statements. In version 5.2 and later, this is no longer an issue.

KSQL 5.2 deprecates some features, including:

  • The use of the RUN SCRIPT statement via the REST API is now deprecated and will be removed in the next major release. (Github issue 2179). The feature circumnavigates certain correctness checks and is unnecessary, given the script content can be supplied in the main body of the request. If you are using the RUN SCRIPT functionality from the KSQL CLI you will not be affected, as this will continue to be supported. If you are using the RUN SCRIPT functionality directly against the REST API your requests will work with the 5.2 server, but will be rejected after the next major version release. Instead, include the contents of the script in the main body of your request.

Version 5.1.0

KSQL 5.1 includes new features, including:

  • WindowStart() and WindowEnd() UDFs
  • StringToDate() and DateToString() UDFs

Detailed Changelog

  • PR-2265 - MINOR: Fix bug encountered when restoring RUN SCRIPT
  • PR-2240 - Bring version checker improvements to 5.1.x
  • PR-2242 - KSQL-1795: First draft of STRUCT topic
  • PR-2235 - KSQL-1794: First draft of query with arrays and maps topic
  • PR-2239 - KSQL-1975: Fix munged Docker commands for kafkacat examples
  • PR-2232 - KSQL-1912: Fix munged scalar functions table
  • PR-2229 - KSQL-1912: Remove extraneous newline
  • PR-2227 - KSQL-1912: Add IFNULL to Scalar Functions table
  • PR-2219 - KSQL-1912: Add IFNULL function to functions table
  • PR-2223 - KSQL-1958: Fix munged CSAS properties table YET AGAIN
  • PR-2222 - KSQL-1957: Add links to new topics; also restore missing CSAS and CTAS text
  • PR-2221 - DOCS-960: Add link to partitioning topic in key requirements section
  • PR-2220 - DOCS-960: Add note about the KEY property
  • PR-2134 - KSQL-1787: First draft of Time and Windows topic
  • PR-2201 - KSQL-1930: Fix a typo in the new Transform a Stream topic
  • PR-2180 - KSQL-1797: First draft of Transform a Stream topic
  • PR-2181 - KSQL-1796: First draft of aggregation topic
  • PR-2136 - Add reference about compatibility breaking configs in upgrade docs
  • PR-2193 - Fix flaky json format test
  • PR-2195 - 5.0.x fix flaky
  • PR-2174 - DOCS-1006: Fix munged :: block
  • PR-2170 - DOCS-911: Fix typos and grammatical errors
  • PR-2169 - DOCS-911: Fix typos and grammatical errors
  • PR-2142 - KSQL-1786: First draft of KSQL and KStreams topic
  • PR-2165 - KSQL-1854: Merge partition sections
  • PR-2143 - Fix some bugs in recovery logic
  • PR-2156 - KSQL-1864: Remove ksql> prompt from example commands
  • PR-2155 - KSQL-1864: Remove ksql> prompt from example commands
  • PR-2152 - KSQL-1864: Remove $ chars prompts for example commands
  • PR-2150 - Currently we don’t support AS for aliasing stream/table.
  • PR-2149 - Using ksql topic name instead of Kafka topic name in topic map in metastore.
  • PR-2137 - Clarify the description of SUBSTRING and its legacy mode setting.
  • PR-2120 - KSQL-1789: First draft of Create a KSQL Table topic
  • PR-2132 - KSQL-1853: Fix heading levels in join and partition topics
  • PR-2130 - DOCS-950: Reworked partitions topic per feedback
  • PR-2122 - Bringing back the commit that was lost because of bad merge.
  • PR-2109 - KSQL-1799: New topic: Troubleshoot KSQL
  • PR-2092 - Window’s UDF doc changes.
  • PR-2090 - Add WindowStart and WindowEnd UDFs (#1993)
  • PR-2075 - Disable optimizations for 5.1.x
  • PR-2051 - Preserve originals when merging configs
  • PR-2080 - Fixed the test.
  • PR-2079 - Fix deprecation issues.
  • PR-2031 - Fix deprecated issues in the build
  • PR-2066 - Minor: Fix bug involving filters with NOT keyword.
  • PR-2056 - Added stringtodate and datetostring UDFs for 5.1.x
  • PR-2048 - Minor: Fix bug involving LIKE patterns without wildcards.
  • PR-2045 - List UDAFs for 5.1.x
  • PR-2043 - Bump airline version to 2.6.0
  • PR-2023 - MINOR: Cause ‘ksql help’ and ‘ksql -help’ to behave the same as ‘ksql -h’ and ‘ksql –help’
  • PR-1979 - Metrics refactor + fix a couple issues
  • PR-2018 - Display stats timestamps in unambiguous format.
  • PR-2017 - KSQL-1725: Fix tables and build warnings
  • PR-1997 - MINOR: Remove duplicate junit dependency in ksql-examples
  • PR-2014 - KSQL-1722: Fix broken inline literal
  • PR-2007 - KSQL-1722: Fix build error in changelog.rst
  • PR-1991 - Minor: Switch tests to use mock Kafka clients.
  • PR-1992 - Minor: Improve test output for QueryTranslationTest
  • PR-1999 - KSQL-1717: Fix build warning in faq.rst
  • PR-1981 - ST-1153: Switch to use cp-base-new and bash-config to hide passwords by default
  • PR-1977 - Use version 5.0.0 for KSQL server image
  • PR-1955 - Hide ssl configs and refactor KsqlResourceTest

Schema Registry Changelog

Version 5.3.0

  • PR-1009 - CC-3741: Remove use of IdentityHashMap
  • PR-1020 - CC-3803: Corrected support for handling defaults of logical schema types
  • PR-1018 - DOCS-319: Add reference to SR security plugin
  • PR-1064 - DEVX-768: 5.2 C3 feature focus: new Schema Registry capabilities
  • PR-1039 - DOCS-900: use compatibilityLevel for GET response examples
  • PR-1067 - DOCS-1437: Change sr-short to sr for proper substitution
  • PR-1073 - DOCS-1440: Remove CP install from SR lefthand TOC
  • PR-1065 - DOCS-1333: Add Schema Migration docs
  • PR-1055 - DOCS-1094: Schema Registry ACLs
  • PR-1060 - MINOR: Backport of: Fix checkstyle to pass on 8.5 after upgrade from 6.19 (#857)
  • PR-1074 - MINOR: Fix exception string formatting in Avro converter
  • PR-1022 - MINOR: improve err message during duplicate ID detection
  • PR-1021 - MINOR: add defensive check for tombstoning

Version 5.2.0

  • PR-1062 - CC-4091: Enable reregistration of schema with different ID
  • PR-1056 - CC-4062: Fix NPE caused after _schemas topic is compacted
  • PR-1054 - CC-4056: Pass id/version when secondary forwards register requests
  • PR-1045 - CC-4015 : Clean up tombstones from internal caches
  • PR-1043 - Support explicit names for non-string map schemas.
  • PR-1035 - CC-3956: Revert “Merge pull request #687 from sunbit-dev/canonical-schemas
  • PR-687 - Fix #28: Normalize schemas to canonical form
  • PR-1031 - CC-3930: Only send version/id if not null
  • PR-1023 - MINOR: swap deleted w/non-deleted before clearing; fix javadoc
  • PR-953 - CC-2669 Allow id and version to be passed to register; add modes
  • PR-992 - CC-3635 - Add config to specify location of static resources
  • PR-994 - Ability to pass additional topic configs to the schemas topic
  • PR-948 - Make CachedSchemaRegistryClient thread-safe
  • PR-930 - CC-3003 : Ability to inject custom headers to CachedSchemaRegistryClient
  • PR-939 - MINOR: ensure store is initialized for getAllKeys

Version 5.1.0

  • PR-933 - DOCS-989 - Fix legacy SR doc issues
  • PR-936 - DOCS-929 - Fix confusing anchor links
  • PR-938 - Duplicate Heading Update (#937)
  • PR-921 - Pass ClientDnsLookup to NetworkClient and parseAndValidateAddresses
  • PR-873 - Refactor SR to pull lookup datastructure to its own interface and Id Generation to its own interface
  • PR-888 - CC-2674: Point to the right checkstyle suppression file
  • PR-846 - CC-2367 : Tombstone deleted schemakey if the same schema is registered again in the subject
  • PR-853 - CC-2441 : Add Cyclic schema support
  • PR-861 - MINOR: Add Timer class. Change from Kafka (commit fc5f6b0e46ff81302b3e445fed0cdf454c942792)
  • PR-857 - Fix checkstyle to pass on 8.5 after upgrade from 6.19

Version 5.0.0

  • PR-837 - CC-2185: Deprecate schema.registry. prefixes on commonly modified configs and allow unprefixed versions
  • PR-834 - DOCS-381 - Rename quick start and improve navigation
  • PR-813 - Update maven-plugin-plugin to fix out of bounds error in build
  • PR-811 - Fix SchemaRegistryCoordinator for upstream changes
  • PR-810 - Remove KafkaJsonEncoder since Scala producers have been removed from Kafka
  • PR-809 - Remove KafkaAvroEncoder since the Scala producers have been removed from Kafka
  • PR-808 - Use java.util.Base64 instead of javax.xml.bind.DatatypeConverter
  • PR-807 - Fix usage of internal Jersey utilities that no longer exist
  • PR-806 - Inherit maven compiler plugin definition from common
  • PR-804 - Ability to plugin an implementation of BasicAuthCredentialProvider
  • PR-803 - CC-1958: Verify configs if createTopic fails because topic already exists
  • PR-792 - Improve GC options
  • PR-784 - Docs for Subject Name Strategy and Basic Auth
  • PR-774 - Revert “Remove ZooKeeper from title”
  • PR-773 - Remove ZooKeeper from title
  • PR-756 - CLIENTS-986 Forward Auth headers
  • PR-748 - confluent.maven.repo: use ‘https’
  • PR-731 - Issue #704 - Make Serde constructors public

Version 4.1.2

  • PR-840 - CC-2201: Handle NPE exception when tombstone messages are encountered
  • PR-839 - CC-2024 : Fix schema not found issue
  • PR-801 - CC-1902 : pass all configs to underlying Serializer/deserializer

Version 4.1.1

  • PR-792 - Improve GC options
  • PR-787 - Docs for Subject Name Strategy and Basic Auth

Version 4.1.0

  • PR-757 - Add WantedBy dependency to ensure startup order
  • PR-741 - CC-1511 : Fix for subject strategy when used with new SubjectNameStrategy.
  • PR-742 - Added delete to the client.
  • PR-738 - CLIENTS-896: Fix path to patch/series
  • PR-735 - Change to use updated Kafka internal API for JaasContext
  • PR-718 - Log properly when Exception raised on SchemaRegistryRestApplication
  • PR-723 - Merge packaging branches into master and add systemd init scripts
  • PR-545 - #544: Add null check for schema doc while adding the property “connect.enum.doc” for schema conversion of enum types
  • PR-680 - New (key|value).multi.type option for Avro serialization
  • PR-710 - Support Multiple SchemaRegistryResourceExtension
  • PR-702 - Set metrics tag for SR metrics
  • PR-694 - check kafkaTopicReader and producer not null before closing
  • PR-686 - issue#629: endpoint to return schema literal
  • PR-696 - Fix sasl inherit
  • PR-683 - Systemd
  • PR-681 - Add getAllVersions to SchemaRegistryClient
  • PR-673 - CLIENTS-729 : ability to inject BasicAuth credentials from producer/consumer SASL
  • PR-676 - CLIENTS-774 : ability to forward authorization header
  • PR-664 - CLIENTS-708 & CLIENTS-720 Fix enum type in fromConnectData
  • PR-571 - Support basic auth credentials embedded in the schema.registry.url
  • PR-657 - Add maxPollTimeout parameter to ConsumerNetworkClient constructor

Version 4.0.1

This release upgrades Jackson 2.5.4 to 2.9.4, and fixes handling of Avro enums.

  • PR-753 - Bump Jackson version
  • PR-692 - MINOR: Backport enum fixes to 4.0.x

Version 4.0.0

  • PR-686 <https://github.com/confluentinc/schema-registry/pull/686> - Added REST endpoint to return the unescaped Avro schema for a specific subject and version.

Upgrade Notes

In 4.0.0, initial creation or validation of the topic used to store schemas has been reimplemented to use native Kafka protocol requests instead of accessing ZooKeeper directly. This means that you are no longer required to have direct access to the ZooKeeper cluster backing your Kafka cluster. However, note that this also requires appropriate permissions to create topics (on first execution of Schema Registry) or describe topics and configs (on subsequent executions to validate the topic is configured correctly). If you have authentication and authorization enabled on your Kafka cluster, you must ensure your principal has the correct permissions before upgrading Schema Registry cluster. Your principal must have the following permissions:

Create Schemas Topic

Operation Resource Reason
Describe Topic: kafkastore.topic Check existence of topic
Create Cluster Create the schemas topic, set compaction policy

Validate Schemas Topic

Operation Resource Reason
Describe Topic: kafkastore.topic Check existence of topic
DescribeConfigs Topic: kafkastore.topic Validate correct compaction policy on topic

Version 3.3.0

  • Upgrade avro to 1.8.2
  • PR-561 - Use TO_AVRO_LOGICAL_CONVERTERS to convert default values that are Kafka connect logical data types to internal format which correspond to schema type. Logic copied from AvroData fromConnectData
  • PR-549 - Replace usage of deprecated ZkUtils.DefaultAcls()
  • Allow for some retries when validating that all nodes have the same primary
  • Relocate Avro serdes under a new avro package
  • Increment Magic Byte for SchemaKey and add compatibility tests
  • Add Delete Schema support
  • Added avro-serde for Kafka Streams. Pulled from the example project.
  • #506 - The AvroMessageFormatter passes byte[] to an Avro encoder, but Avro only likes ByteBuffer. So we need to ByteBuffer.wrap() instead.
  • Added optional kafkastore.group.id config to override the one automatically created by Schema Registry
  • PR-476 - Adapt to KAFKA-4636 changes: Per listener security settings overrides (KIP-103)

Version 3.2.1

  • PR-503 - CLIENTS-244: Update 3.2.0 changelog
  • PR-499 - making sure Schema Registry doesn’t start with uncompacted topic
  • PR-497 - CLIENTS-103: Fix ArrayIndexOutOfBoundsException in SchemaRegistryPerformance by counting the registration attempt even if it failed.
  • PR-493 - Fixes for CLIENTS-257
  • PR-494 - MINOR: Add compact schemas topic doc
  • PR-458 - CLIENTS-104: Add a few retries during startup to allow for slow metadata propagation after creating the _schemas topic.

Version 3.2.0

  • PR-428 - Maven Checkstyle
  • PR-425 - Logical Type support
  • PR-440 - Documentation changes to reflect pull request 415
  • PR-451 - Generalize schema incompatibility message.
  • PR-457 - Update ClusterTestHarness to use o.a.k.common.utils.Time.
  • PR-458 - CLIENTS-104: Add a few retries during startup to allow for slow metadata propagation after creating the _schemas topic.
  • PR-464 - Improve request URL building in client
  • PR-465 - Add topic to error string to make debugging easier.
  • PR-448 - Fixes the following avro issues (complex union, document preservation, output schema != input schema)
  • PR-468 - CC-443: AvroData’s caches should be synchronized for thread-safety
  • PR-473 - Fix build to work post KIP-103.
  • PR-474 - Fix broker endpoint extraction to correctly translate to the non-ListenerName version that clients use to initiate broker connections.
  • PR-472 - Handle primitive types when specific.avro.reader is true
  • PR-477 - Docchangefor3.2
  • PR-488 - Don’t re-invoke SchemaBuilder.version() and SchemaBuilder.name() if the value has already been set.
  • PR-494 - MINOR: Add compact schemas topic doc

Version 3.1.1

No changes

Version 3.1.0

Schema Registry

  • PR-429 - Moving licenses and notices to a new format, generated by an internal script.
  • PR-415 - Option to apply fully transitive schema compatibility checking
  • PR-412 - Require bash since we use some bashisms and fix a copyright.
  • PR-396 - ZooKeeper and Kafka SASL support.
  • PR-384 - Update the link to Google Java code Style
  • PR-372 - Update KafkaStore to use moved TopicExistsException class.
  • PR-373 - Added get all subjects.
  • PR-364 - Increase testing timeouts from 5000ms to 15000ms
  • PR-346 - configured log4j to write to log file

Serializers, Formatters, and Converters

  • PR-403 - CLIENTS-101: Document wire format for Avro.
  • PR-379 - Maven plugin
  • PR-355 - Adding support for Fixed data types
  • PR-352 - Fix schemas.cache.config can’t be overrided

Version 3.0.1

Schema Registry

  • PR-369 - Introducing a new config: kafkastore.bootstrap.servers
  • PR-371 - Fixing a bug where the listener port wasn’t used in ZooKeeper
  • PR-390 - Include cURL output in quickstart
  • PR-392 - Fix schemes used for listeners

Serializers, Formatters, and Converters

  • PR-363 - 3.0.x duplicate schema support
  • PR-365 - Fix API compatibility regression from #363
  • PR-378 - Support null namespace for Array of Records

Version 3.0.0

Schema Registry

  • PR-212 - change the documentation on port to have a high priority and list it higher up in the docs
  • PR-298 - Bump version to 3.0.0-SNAPSHOT and Kafka dependency to 0.10.0.0-SNAPSHOT
  • PR-300 - Using the new 0.9 Kafka consumer.
  • PR-302 - Fix build to handle rack aware changes in Kafka.
  • PR-305 - Update to match changed methods in CoreUtils
  • PR-317 - Change the ‘host.name’ importance to high
  • PR-319 - KafkaStore SSL support.
  • PR-320 - API reference uses ‘integer’ Avro type which isn’t supported. ‘int’ is supported.
  • PR-329 - https support.

Serializers, Formatters, and Converters

  • PR-264 - Add null checks to json serializer/deserializer
  • PR-274 - Add support for Avro projections in decoders
  • PR-275 - Fixed references to confluent common version
  • PR-276 - Unit tests and bugfix for NPE when using nested optional fields
  • PR-278 - Test cases for optional nested structs
  • PR-280 - Fix fromConnectData for optional complex types
  • PR-290 - Issue #284 Cannot set max.schemas.per.subject due to cast exception
  • PR-297 - Allows any CharSequence implementation to be considered a string
  • PR-318 - Minor cleanup
  • PR-323 - Fix #142 - handle parsing non-json responses in the RestService
  • PR-332 - Add status storage topic to Connect Avro sample config.

Version 2.0.1

Schema Registry

  • PR-286 - Update Kafka version to 0.9.0.1-cp1.

Version 2.0.0

Schema Registry

  • PR-141 - Incorrect path to log4j.properties file for simple zip file layout
  • PR-143 - schema-registry-start does not work with -daemon argument
  • PR-152 - Added point about Google code style to Readme.
  • PR-163 - Expose more information when registry fails to start
  • PR-165 - Add compatibility support for SchemaRegistryClient
  • PR-167 - Update the versionCache with a new schemaVersionMap for a subject
  • PR-169 - Use correct URL to update compatibility setting of a subject
  • PR-180 - GH-177: Remove unneeded content-type headers for GET ops in quickstart and README
  • PR-184 - Support multiple registry URLs in client
  • PR-186 - Rename LocalSchemaRegistryClient to make it clear it is only intended to be used as a mock in tests. Fixes #185.
  • PR-187 - Correct example response for POST /subjects/(string: subject)/versions
  • PR-188 - Address GH-168; enable unit testing of CachedSchemaRegistryClient
  • PR-195 - Fixed typo in Exception message
  • PR-196 - Require Java 7
  • PR-197 - Enable test code sharing
  • PR-198 - Update jersey, jackson and junit versions to match rest-utils
  • PR-202 - Correct minor docs error in example response
  • PR-203 - Issue 194 rename main
  • PR-207 - Issue #170: PUT /config/(string: subject) should return 4xx for unknown subjects
  • PR-210 - Issue #208: Several RestApiTest test cases don’t test proper exception behavior
  • PR-219 - Update Kafka version to 0.8.3-SNAPSHOT so we can start developing using upcoming 0.8.3 features.
  • PR-237 - Update uses of ZkUtils to match changes made in KAFKA-2639.
  • PR-240 - Fixes Typo in the docs. ‘actsas’ -> ‘acts as’
  • PR-242 - Fixed Schema Registry build against kafka trunk
  • PR-243 - Use x.y.z versioning scheme (i.e. 2.0.0-SNAPSHOT)
  • PR-245 - Fix mvn assembly setup
  • PR-252 - Use Kafka compiled with Scala 2.11
  • PR-257 - Updated classpath in schema-registry-run-class to reflect changes in pom.xml
  • PR-258 - CC-53: Add worker configs for Avro Kafka Connect that integrates with Schema Registry.

Serializers, Formatters, and Converters

  • PR-146 - Added decoder to return a specific Avro record from bytes.
  • PR-162 - Add implementation of new Deserializer interface.
  • PR-192 - Add new module for Kafka JSON serialization stuff
  • PR-193 - Add more JSON codec support
  • PR-200 - Switch serializer config classes to use AbstractConfig from confluent-common instead of from Kafka.
  • PR-222 - Add AvroConverter in new copycat-avro-converter jar to convert Copycat and Avro data.
  • PR-234 - Add AvroConverter support for Decimal, Date, Time, and Timestamp logical types.
  • PR-235 - Add caching of schema conversions in AvroData and AvroConverter.
  • PR-247 - Update for Copycat -> Kafka Connect renaming.
  • PR-251 - Add tests of conversion of null values from Kafka Connect and fix handling of null for int8 and int16 types.
  • PR-254 - Ensure AvroConverter passes through null values without adding schemas and that deserialized null values are converted to SchemaAndValue.NULL.
  • PR-255 - CC-44: Include version when deserializing Kafka Connect data.
  • PR-256 - Encode null values for schemaless array entries and map keys and values as an Anything record.