Confluent Platform Component Changelogs¶
This topic provides changelogs for the individual Confluent Platform components.
- Redesigned the Control Center UI, including KSQL and Connect enhancements. New pages show cluster overview, topics pages, and consumers index.
- RBAC enforcement.
- Topic message browser enhancements, including jump to timestamp and download results in JSON format.
- Alert enhancements that added Slack and PagerDuty webhook notifications, a new Consumer lead alert, pause and resume global alerts actions, and fast navigation to Alerts features from the Alerts bell icon.
- Added multiple Connect support.
- Added multiple KSQL support.
- Added broker configs editing.
- Added Schema Registry editing.
- Added license management UI.
- Improved message browsing.
- Improved KSQL UI.
- Improved scalability.
- Browser support for Chrome, Firefox, and Safari. For more information, see Web Browsers.
- A new, more readable light theme.
- Many bug fixes and UX improvements.
- Added configuration confluent.controlcenter.license.manager=_confluent-controlcenter-license-manager-5-1-0. (It was confluent.controlcenter..license.manager = _confluent-controlcenter-license-manager5-1-0 in first 5.1.x release, but it is fixed now.) No need for customers to override this configuration.
- Added KSQL interface to create/view streams, tables, and queries.
- Added topic inspection to read messages from a topic.
- Added consumer group message lag viewing and alerting.
- Added Schema Registry viewing.
- Added read only view of broker configurations.
- Added feature flags for variety of user accessible features.
- Added ability to create and edit topics in Control Center.
- Added topic filtering and ability to hide Apache Kafka®/Confluent Platform internal topics from the UI.
- Added new alert to warn when Control Center stops receiving data from a Kafka cluster.
- Added a restricted user type that prevents editing or creating of topics, alerts, or connectors.
- Made calls to Connect more resilient.
- Fixed an issue that would prevent chart highlighting on System Health charts.
- Fixed an issue that could cause Control Center to improperly warn about improper log dirs on brokers.
- Fixed an issue that could cause Control Center to improperly alert about active controller count.
- Fixed an issue that could cause long request times if a cluster had brokers with noncontiguous IDs.
- Additional logging during startup.
- Fixed an issue that could cause Control Center to show blank graphs when data was present.
- Set and enforce TimestampType for internal Control Center topics at startup.
- Fixed an issue that caused us to link to an invalid consumer group on Streams Monitoring.
- Fixed an issue where Broker Monitoring could check for data last update time on wrong cluster.
- Allow overriding ConfluentMetricsReporter default producer settings.
- Allow using a file for
- Docker image: allow passing license using CONTROL_CENTER_CONFLUENT_LICENSE environment variable.
- Fixed an issue that resulted in Offline and Under Replicated Partitions data to be swapped.
- Fixed an issue that could cause Control Center to shutdown if it lost connection to Kafka.
- Fixed an issue that could cause Control Center to hang if it encountered an error creating topics on startup.
Version 3.3.1 (EOSL)¶
- Allow text input in date picker.
- Fixed an issue that could cause unsubmittable forms.
- Show renamed cluster value rather than base clusterid on alerts page.
- Added alert banner on Broker Monitoring to indicate a cluster may be down.
- Scalability increase for Broker Monitoring to reduce number of messages sent to Kafka.
- Fixed an issue where 100% Network Pool Usage was displayed instead of 0%.
- Allow graceful shutdown of Control Center during startup.
- Fixed a race condition that could happen if you start Control Center and your Kafka broker at the same time.
Version 3.3.0 (EOSL)¶
- Sped up restore time.
- Reduced the size of old data over long periods of time.
- Upgraded to use new admin client (KIP-117).
- Alert UI should render better alert information.
- Triggers are createable directly though the UI on System Health.
- Massive performance improvements to Stream Monitoring UI.
- Fixed various time range display/selection bugs; persist time ranges though navigation.
- Nonconfigurable form values being sent to Connect via UI should remain and be visible prior to submit.
Version 3.2.2 (EOSL)¶
- Added shutdown script.
- Force faster compaction and cleanup on changelog topics.
- Several Streams improvements to reduce state-store restoration time after unclean shutdown.
- Reduce the volume of data to be restored to state-stores on startup.
- Fixed possible concurrency problem in setting RocksDB configuration options.
- Fixed possible deadlock on exception after streams startup.
Version 3.2.1 (EOSL)¶
- Enabled HTTP compression by default.
- Fixed possible deadlock on streams exception.
- Fixed fd leak when communicating with Kafka.
- Update defaults for monitoring producers to be more resilient.
- Prevent inadvertently creating misconfigured topics during startup during a failure.
- Check Kafka version at startup.
- Reduce minimum startup time.
- More efficient duplicate checking to reduce write amplification.
- Handle ‘hidden’ connect configs. This allows Control Center to manage Connectors that do not fully enumerate their configs.
- Ensure users only see invalid license warning if their license is invalid.
- Fixed topic display bug that would prevent topic details from showing.
- Reduced logging in js.
- Allow users to close missing data modal dialogs.
- Better rendering of System Health metrics especially when some data is missing.
- Better validation of triggers, actions, and connector configs.
- Faster processing of trigger and action creation and update.
- Corrected some misleading labels on System Health.
- Better error handling around Connect communications.
- Change default timespan in UI to 4 hours.
- Check and warn about errors on Topic Management page.
Version 3.2.0 (EOSL)¶
- Add capabilities to monitor brokers instrumented with Confluent Metrics Reporter.
- Enable alerting on broker and topic metrics.
- Add UI for viewing topic information.
- Add status UI for tracking internal progress.
- Fixed bug where we could inadvertently show license expired message.
- Fixed bugs with alert trigger creation in UI.
- Fixed issue with parsing certain error conditions from Connect.
Version 3.1.0 (EOSL)¶
- Enable gathering stream monitoring data from multiple clusters.
- Support alerting on stream monitoring conditions.
- Ability to restrict UI access via user login.
Version 3.0.1 (EOSL)¶
- Improved data fetching from the browser.
- Handle authenticated/authorized Kafka clusters.
- Bug fix for running in daemon mode.
- More efficient topic creation at startup.
- Reduced number of necessary topics/rocksdbs for aggregations.
- Reduced total amount of stored data.
- Fixed timing issues when receiving delayed data.
- Configs updates to prefix Kafka/rest configs.
- Added rate limit for logging to prevent logsize blowup.
- Allow deleting connector in UI.
- PR-511 - CPKAFKA-1876: Log cause of failure to produce messages in tests
- PR-512 - Fix race condition when expiration thread iterates over partially-initialized consumer
- PR-503 - Fix location of jars for dev testing
- PR-481 - CPKAFKA-1681: Add long polling functionality
- PR-508 - CPKAFKA-1868: Add more consumer tests
- PR-507 - CPKAFKA-1871: Fix problem where ReadTaskSchedulerThread would loop too often
- PR-504 - CPKAFKA-1865: Performance degrades drastically after running for extended periods
- PR-495 - CPKAFKA-1794: Assign leader ID -1 on offline partitions in the topic describe API
- PR-477 - Fix SimpleConsumerManager properly return requested offsets for compressed topic
- PR-474 - MINOR: Fix checkstyle path
- PR-466 - Change Consumer threading model to better handle rebalances
- PR-459 - MINOR: Add utility createConsumerProperties() test method since it was removed from AK
- PR-452 - Fix checkstyle to pass on 8.5 after upgrade from 6.19
- PR-439 - Temporarily pin Kafka and Confluent versions
- PR-438 - Don’t use deprecated RecordMetadata constructor
- PR-303 - CLIENTS-304: Fix AvroConsumerRecord constructors and handling of topic for v1 consumer API.
- PR-256 - Update ClusterTestHarness to use o.a.k.common.utils.Time.
- PR-255 - clarified the key/value format limitations based on user questions
- PR-264 - Fix build to work post KIP-103
- PR-265 - Follow up for KIP-103 changes that fixes ProducerPool’s extraction of endpoints to use the security protocol in the URL instead of the listener name.
- PR-268 - modified readme to use v2. also bumped installation up for usability
- PR-271 - Updated quickstart and configuration docs for V2 and for security
- PR-258 - REST Proxy security
- PR-274 - added field ‘topic’ to tools/ConsumerPerformance.java for system test
- PR-272 - added description of JVM parameters for SASL configuration
- PR-239 - Require bash since we use some bashisms and fix a copyright.
- PR-235 - fix typo command line options
- PR-222 - fixing issue #91
- PR-230 - Issue #229: Add blurb on Jetty jmx metrics
- PR-214 - Small readme.md fix
- PR-127 - Fix to ConsumerManager tests AND then handling of max bytes
- PR-203 - Fix a typo that cause non-corresponding logger name
- PR-202 - fix the implementation of topicExists
- PR-164 - 2.x merge to master
- PR-166 - Bump version to 3.0.0-SNAPSHOT and Kafka dependency to 0.10.0.0-SNAPSHOT
- PR-171 - Fix build to handle rack aware changes in Kafka.
- PR-174 - Update CoreUtils usage to match kafka updates
- PR-189 - Minor fixes for compatibility with newest 0.10.0 branch.
- PR-192 - Minor fixes for compatibility with newest 0.10.0 branch.
- PR-202 - fix the implementation of topicExists
- PR-205 - Rearrange quickstart and use topics from earlier steps in requests for metadata so the example output will exactly match real output when starting from an empty cluster.
- PR-64 - Reduce integration test time.
- PR-66 - Add support for SimpleConsumer-like access (Issue #26)
- PR-67 - Handle conflicting IDs and separate IDs used in the REST Proxy and by Kafka’s consumer implementation.
- PR-78 - Remove kafka from list of production directories to include in CLASSPATH.
- PR-89 - JSON message support
- PR-96 - Fixed log4j and daemon flag bugs in kafka-rest-run-class based on fix from schema-registry.
- PR-99 - Require Java 7
- PR-101 - rest-utils updates
- PR-103 - Issue 94 rename main
- PR-108 - Clarify partitioning behavior for produce requests
- PR-117 - Update to Kafka 0.9.0.0-SNAPSHOT and make adjustments to work with updated ZkUtils.
- PR-122 - Use x.y.z versioning scheme (i.e. 2.0.0-SNAPSHOT)
- PR-123 - Updated args for JaasUtils.isZkSecurityEnabled()
- PR-125 - Use Kafka compiled with Scala 2.11
KSQL 5.3.0 includes new features, including:
- Drop the requirement that
CREATE TABLEstatements must have a
KEYset in their
WITHclause. This is now an optional optimization to avoid unnecessary repartition steps. See Github issue #2745 for more info.
- Improved handling of
KEYfield is an optional copy of the Kafka record’s key held within the record’s value. Users can supply the name of the field that holds the copy of the key within the
WITHclause. The improved handling may eliminate unnecessary repartition steps in certain queries. Please note that preexisting persistent queries, e.g. those created via
CREATE TABLE AS SELECT ...or
CREATE STREAM AS SELECT ...or
INSERT INTO ..., will continue to have the unnecessary repartition step. This is required to avoid the potential for data loss should this step be dropped. See Github issue #2636 for more info.
INSERT INTO ... VALUESis now supported, with standard SQL syntax to insert rows to existing KSQL streams/tables. To disable this functionality, set
falsein the server properties.
CREATE TABLEwill now allow you to create the topic if it is missing. To do this, specify the
CREATE STREAM AS SELECT ...and
CREATE TABLE AS SELECT ...statements now use the source topic partitions and replica counts for the sink topic if the config properties are not set in the
WITHclause. In case of JOIN, the left hand side topic is used. This deprecates
- A new config variable,
ksql.internal.topic.replicas, was introduced to set the replica count for the internal topics created by KSQL Server. The internal topics include the command topic and the config topic.
- A new KSQL testing tool was added. The tool is a command line utility that enables testing KSQL statements without requiring any infrastructure, like Apache Kafka and KSQL clusters.
KSQL 5.3.0 includes bug fixes, including:
ROWTIMEof the row generated when a
JOINencounters late data was previous the
ROWTIMEof the late event, where as now it is the max of
ROWTIMEof the rows involved in the join. This provides more deterministic join semantics.
Return values of UDF and UDAFs are now correctly marked as optional, where previously there was potential for non-optional fields, which would result in serialization issues in the presence of
This is a forward compatible change in Avro, i.e. after upgrading, KSQL will be able to read old values using the new schema. However, it is important to ensure downstream consumers of the data are using the updated schema before upgrading KSQL, as otherwise deserialization may fail. The updated schema is best obtained from running the query in another KSQL cluster, running version 5.3.
See Github issue #2769 for more info.
KSQL 5.3 includes bug fixes, including:
- Fixed issues with using
ASkeyword when aliasing sources. See #2732 for more info.
KSQL 5.2 includes new features, including:
- Support for HTTPS.
- Support for CASE expression: KSQL now supports CASE conditional expression in Searched form where KSQL evaluates each condition from left to right. It returns the result for the first condition that evaluates to true. If no condition evaluates to true, the result for the ELSE clause will be returned. If there is no ELSE clause, null is returned.
- A new family of UDFs for improved handling of URIs (e.g. extracting information/decoding information), see UDF table for all URL functions
LIMITkeyword support for
- Support for read-after-write consistency: new commands don’t execute until previous commands have finished executing. This feature is enabled by default in the CLI (#2280) and can be implemented by the user for the REST API (Coordinate Multiple Requests).
- A log of record processing events to help users debug their KSQL queries. The log can be configured to log to Kafka to be consumed as a KSQL stream. See KSQL processing log for more details.
- Aggregation functionality has been extended. KSQL now supports:
GROUP BYmore than just simple columns, including fields within structs, arithmetic results, functions, string concatenations and literals.
- literals in the projection, (a.k.a the
HAVINGclauses, including the use of aggregate functions and literals.
- Automatic compatibility management for queries in headless mode across versions. Starting with 5.2, KSQL will automatically take care of ensuring query compatibility when upgrading. This means you won’t need to worry about setting properties correctly during upgrade, as has been required for previous upgrades. Refer to the architecture documentation for details. Note that it is still up to the user to set properties correctly before upgrading to 5.2. The upgrade doc has details about the properties required to safely upgrade to 5.2.
KSQL 5.2 includes bug fixes, including:
- Improved support for multi-line requests in interactive mode deployments. Prior to version 5.2 KSQL parsed the full request before attempting to execute any statements. Requests that contained later statements that were dependent the execution of prior statements may have failed. In version 5.2 and later, this is no longer an issue.
- Improved support for non-interactive, “headless” mode deployments. Prior to version 5.2 KSQL parsed the full script before attempting to execute any statements. The full parse would often fail when later statements relied on the execution of earlier statements. In version 5.2 and later, this is no longer an issue.
KSQL 5.2 deprecates some features, including:
- The use of the
RUN SCRIPTstatement via the REST API is now deprecated and will be removed in the next major release. (Github issue 2179). The feature circumnavigates certain correctness checks and is unnecessary, given the script content can be supplied in the main body of the request. If you are using the
RUN SCRIPTfunctionality from the KSQL CLI you will not be affected, as this will continue to be supported. If you are using the
RUN SCRIPTfunctionality directly against the REST API your requests will work with the 5.2 server, but will be rejected after the next major version release. Instead, include the contents of the script in the main body of your request.
KSQL 5.1 includes new features, including:
- PR-2265 - MINOR: Fix bug encountered when restoring RUN SCRIPT
- PR-2240 - Bring version checker improvements to 5.1.x
- PR-2242 - KSQL-1795: First draft of STRUCT topic
- PR-2235 - KSQL-1794: First draft of query with arrays and maps topic
- PR-2239 - KSQL-1975: Fix munged Docker commands for kafkacat examples
- PR-2232 - KSQL-1912: Fix munged scalar functions table
- PR-2229 - KSQL-1912: Remove extraneous newline
- PR-2227 - KSQL-1912: Add IFNULL to Scalar Functions table
- PR-2219 - KSQL-1912: Add IFNULL function to functions table
- PR-2223 - KSQL-1958: Fix munged CSAS properties table YET AGAIN
- PR-2222 - KSQL-1957: Add links to new topics; also restore missing CSAS and CTAS text
- PR-2221 - DOCS-960: Add link to partitioning topic in key requirements section
- PR-2220 - DOCS-960: Add note about the KEY property
- PR-2134 - KSQL-1787: First draft of Time and Windows topic
- PR-2201 - KSQL-1930: Fix a typo in the new Transform a Stream topic
- PR-2180 - KSQL-1797: First draft of Transform a Stream topic
- PR-2181 - KSQL-1796: First draft of aggregation topic
- PR-2136 - Add reference about compatibility breaking configs in upgrade docs
- PR-2193 - Fix flaky json format test
- PR-2195 - 5.0.x fix flaky
- PR-2174 - DOCS-1006: Fix munged :: block
- PR-2170 - DOCS-911: Fix typos and grammatical errors
- PR-2169 - DOCS-911: Fix typos and grammatical errors
- PR-2142 - KSQL-1786: First draft of KSQL and KStreams topic
- PR-2165 - KSQL-1854: Merge partition sections
- PR-2143 - Fix some bugs in recovery logic
- PR-2156 - KSQL-1864: Remove ksql> prompt from example commands
- PR-2155 - KSQL-1864: Remove ksql> prompt from example commands
- PR-2152 - KSQL-1864: Remove $ chars prompts for example commands
- PR-2150 - Currently we don’t support AS for aliasing stream/table.
- PR-2149 - Using ksql topic name instead of Kafka topic name in topic map in metastore.
- PR-2137 - Clarify the description of SUBSTRING and its legacy mode setting.
- PR-2120 - KSQL-1789: First draft of Create a KSQL Table topic
- PR-2132 - KSQL-1853: Fix heading levels in join and partition topics
- PR-2130 - DOCS-950: Reworked partitions topic per feedback
- PR-2122 - Bringing back the commit that was lost because of bad merge.
- PR-2109 - KSQL-1799: New topic: Troubleshoot KSQL
- PR-2092 - Window’s UDF doc changes.
- PR-2090 - Add WindowStart and WindowEnd UDFs (#1993)
- PR-2075 - Disable optimizations for 5.1.x
- PR-2051 - Preserve originals when merging configs
- PR-2080 - Fixed the test.
- PR-2079 - Fix deprecation issues.
- PR-2031 - Fix deprecated issues in the build
- PR-2066 - Minor: Fix bug involving filters with NOT keyword.
- PR-2056 - Added stringtodate and datetostring UDFs for 5.1.x
- PR-2048 - Minor: Fix bug involving LIKE patterns without wildcards.
- PR-2045 - List UDAFs for 5.1.x
- PR-2043 - Bump airline version to 2.6.0
- PR-2023 - MINOR: Cause ‘ksql help’ and ‘ksql -help’ to behave the same as ‘ksql -h’ and ‘ksql –help’
- PR-1979 - Metrics refactor + fix a couple issues
- PR-2018 - Display stats timestamps in unambiguous format.
- PR-2017 - KSQL-1725: Fix tables and build warnings
- PR-1997 - MINOR: Remove duplicate junit dependency in ksql-examples
- PR-2014 - KSQL-1722: Fix broken inline literal
- PR-2007 - KSQL-1722: Fix build error in changelog.rst
- PR-1991 - Minor: Switch tests to use mock Kafka clients.
- PR-1992 - Minor: Improve test output for QueryTranslationTest
- PR-1999 - KSQL-1717: Fix build warning in faq.rst
- PR-1981 - ST-1153: Switch to use cp-base-new and bash-config to hide passwords by default
- PR-1977 - Use version 5.0.0 for KSQL server image
- PR-1955 - Hide ssl configs and refactor KsqlResourceTest
- PR-1009 - CC-3741: Remove use of IdentityHashMap
- PR-1020 - CC-3803: Corrected support for handling defaults of logical schema types
- PR-1018 - DOCS-319: Add reference to SR security plugin
- PR-1064 - DEVX-768: 5.2 C3 feature focus: new Schema Registry capabilities
- PR-1039 - DOCS-900: use compatibilityLevel for GET response examples
- PR-1067 - DOCS-1437: Change sr-short to sr for proper substitution
- PR-1073 - DOCS-1440: Remove CP install from SR lefthand TOC
- PR-1065 - DOCS-1333: Add Schema Migration docs
- PR-1055 - DOCS-1094: Schema Registry ACLs
- PR-1060 - MINOR: Backport of: Fix checkstyle to pass on 8.5 after upgrade from 6.19 (#857)
- PR-1074 - MINOR: Fix exception string formatting in Avro converter
- PR-1022 - MINOR: improve err message during duplicate ID detection
- PR-1021 - MINOR: add defensive check for tombstoning
- PR-1062 - CC-4091: Enable reregistration of schema with different ID
- PR-1056 - CC-4062: Fix NPE caused after _schemas topic is compacted
- PR-1054 - CC-4056: Pass id/version when secondary forwards register requests
- PR-1045 - CC-4015 : Clean up tombstones from internal caches
- PR-1043 - Support explicit names for non-string map schemas.
- PR-1035 - CC-3956: Revert “Merge pull request #687 from sunbit-dev/canonical-schemas
- PR-687 - Fix #28: Normalize schemas to canonical form
- PR-1031 - CC-3930: Only send version/id if not null
- PR-1023 - MINOR: swap deleted w/non-deleted before clearing; fix javadoc
- PR-953 - CC-2669 Allow id and version to be passed to register; add modes
- PR-992 - CC-3635 - Add config to specify location of static resources
- PR-994 - Ability to pass additional topic configs to the schemas topic
- PR-948 - Make CachedSchemaRegistryClient thread-safe
- PR-930 - CC-3003 : Ability to inject custom headers to CachedSchemaRegistryClient
- PR-939 - MINOR: ensure store is initialized for getAllKeys
- PR-933 - DOCS-989 - Fix legacy SR doc issues
- PR-936 - DOCS-929 - Fix confusing anchor links
- PR-938 - Duplicate Heading Update (#937)
- PR-921 - Pass ClientDnsLookup to NetworkClient and parseAndValidateAddresses
- PR-873 - Refactor SR to pull lookup datastructure to its own interface and Id Generation to its own interface
- PR-888 - CC-2674: Point to the right checkstyle suppression file
- PR-846 - CC-2367 : Tombstone deleted schemakey if the same schema is registered again in the subject
- PR-853 - CC-2441 : Add Cyclic schema support
- PR-861 - MINOR: Add Timer class. Change from Kafka (commit fc5f6b0e46ff81302b3e445fed0cdf454c942792)
- PR-857 - Fix checkstyle to pass on 8.5 after upgrade from 6.19
- PR-837 - CC-2185: Deprecate schema.registry. prefixes on commonly modified configs and allow unprefixed versions
- PR-834 - DOCS-381 - Rename quick start and improve navigation
- PR-813 - Update maven-plugin-plugin to fix out of bounds error in build
- PR-811 - Fix SchemaRegistryCoordinator for upstream changes
- PR-810 - Remove KafkaJsonEncoder since Scala producers have been removed from Kafka
- PR-809 - Remove KafkaAvroEncoder since the Scala producers have been removed from Kafka
- PR-808 - Use java.util.Base64 instead of javax.xml.bind.DatatypeConverter
- PR-807 - Fix usage of internal Jersey utilities that no longer exist
- PR-806 - Inherit maven compiler plugin definition from common
- PR-804 - Ability to plugin an implementation of BasicAuthCredentialProvider
- PR-803 - CC-1958: Verify configs if createTopic fails because topic already exists
- PR-792 - Improve GC options
- PR-784 - Docs for Subject Name Strategy and Basic Auth
- PR-774 - Revert “Remove ZooKeeper from title”
- PR-773 - Remove ZooKeeper from title
- PR-756 - CLIENTS-986 Forward Auth headers
- PR-748 - confluent.maven.repo: use ‘https’
- PR-731 - Issue #704 - Make Serde constructors public
- PR-757 - Add WantedBy dependency to ensure startup order
- PR-741 - CC-1511 : Fix for subject strategy when used with new SubjectNameStrategy.
- PR-742 - Added delete to the client.
- PR-738 - CLIENTS-896: Fix path to patch/series
- PR-735 - Change to use updated Kafka internal API for JaasContext
- PR-718 - Log properly when Exception raised on SchemaRegistryRestApplication
- PR-723 - Merge packaging branches into master and add systemd init scripts
- PR-545 - #544: Add null check for schema doc while adding the property “connect.enum.doc” for schema conversion of enum types
- PR-680 - New (key|value).multi.type option for Avro serialization
- PR-710 - Support Multiple SchemaRegistryResourceExtension
- PR-702 - Set metrics tag for SR metrics
- PR-694 - check kafkaTopicReader and producer not null before closing
- PR-686 - issue#629: endpoint to return schema literal
- PR-696 - Fix sasl inherit
- PR-683 - Systemd
- PR-681 - Add getAllVersions to SchemaRegistryClient
- PR-673 - CLIENTS-729 : ability to inject BasicAuth credentials from producer/consumer SASL
- PR-676 - CLIENTS-774 : ability to forward authorization header
- PR-664 - CLIENTS-708 & CLIENTS-720 Fix enum type in fromConnectData
- PR-571 - Support basic auth credentials embedded in the schema.registry.url
- PR-657 - Add maxPollTimeout parameter to ConsumerNetworkClient constructor
This release upgrades Jackson 2.5.4 to 2.9.4, and fixes handling of Avro enums.
- PR-686 <https://github.com/confluentinc/schema-registry/pull/686> - Added REST endpoint to return the unescaped Avro schema for a specific subject and version.
In 4.0.0, initial creation or validation of the topic used to store schemas has been reimplemented to use native Kafka protocol requests instead of accessing ZooKeeper directly. This means that you are no longer required to have direct access to the ZooKeeper cluster backing your Kafka cluster. However, note that this also requires appropriate permissions to create topics (on first execution of Schema Registry) or describe topics and configs (on subsequent executions to validate the topic is configured correctly). If you have authentication and authorization enabled on your Kafka cluster, you must ensure your principal has the correct permissions before upgrading Schema Registry cluster. Your principal must have the following permissions:
Create Schemas Topic
||Check existence of topic|
|Create||Cluster||Create the schemas topic, set compaction policy|
Validate Schemas Topic
||Check existence of topic|
||Validate correct compaction policy on topic|
- Upgrade avro to 1.8.2
- PR-561 - Use TO_AVRO_LOGICAL_CONVERTERS to convert default values that are Kafka connect logical data types to internal format which correspond to schema type. Logic copied from AvroData fromConnectData
- PR-549 - Replace usage of deprecated ZkUtils.DefaultAcls()
- Allow for some retries when validating that all nodes have the same primary
- Relocate Avro serdes under a new avro package
- Increment Magic Byte for SchemaKey and add compatibility tests
- Add Delete Schema support
- Added avro-serde for Kafka Streams. Pulled from the example project.
- #506 - The AvroMessageFormatter passes byte to an Avro encoder, but Avro only likes ByteBuffer. So we need to ByteBuffer.wrap() instead.
- Added optional kafkastore.group.id config to override the one automatically created by Schema Registry
- PR-476 - Adapt to KAFKA-4636 changes: Per listener security settings overrides (KIP-103)
- PR-503 - CLIENTS-244: Update 3.2.0 changelog
- PR-499 - making sure Schema Registry doesn’t start with uncompacted topic
- PR-497 - CLIENTS-103: Fix ArrayIndexOutOfBoundsException in SchemaRegistryPerformance by counting the registration attempt even if it failed.
- PR-493 - Fixes for CLIENTS-257
- PR-494 - MINOR: Add compact schemas topic doc
- PR-458 - CLIENTS-104: Add a few retries during startup to allow for slow metadata propagation after creating the _schemas topic.
- PR-428 - Maven Checkstyle
- PR-425 - Logical Type support
- PR-440 - Documentation changes to reflect pull request 415
- PR-451 - Generalize schema incompatibility message.
- PR-457 - Update ClusterTestHarness to use o.a.k.common.utils.Time.
- PR-458 - CLIENTS-104: Add a few retries during startup to allow for slow metadata propagation after creating the _schemas topic.
- PR-464 - Improve request URL building in client
- PR-465 - Add topic to error string to make debugging easier.
- PR-448 - Fixes the following avro issues (complex union, document preservation, output schema != input schema)
- PR-468 - CC-443: AvroData’s caches should be synchronized for thread-safety
- PR-473 - Fix build to work post KIP-103.
- PR-474 - Fix broker endpoint extraction to correctly translate to the non-ListenerName version that clients use to initiate broker connections.
- PR-472 - Handle primitive types when specific.avro.reader is true
- PR-477 - Docchangefor3.2
- PR-488 - Don’t re-invoke SchemaBuilder.version() and SchemaBuilder.name() if the value has already been set.
- PR-494 - MINOR: Add compact schemas topic doc
- PR-429 - Moving licenses and notices to a new format, generated by an internal script.
- PR-415 - Option to apply fully transitive schema compatibility checking
- PR-412 - Require bash since we use some bashisms and fix a copyright.
- PR-396 - ZooKeeper and Kafka SASL support.
- PR-384 - Update the link to Google Java code Style
- PR-372 - Update KafkaStore to use moved TopicExistsException class.
- PR-373 - Added get all subjects.
- PR-364 - Increase testing timeouts from 5000ms to 15000ms
- PR-346 - configured log4j to write to log file
- PR-212 - change the documentation on port to have a high priority and list it higher up in the docs
- PR-298 - Bump version to 3.0.0-SNAPSHOT and Kafka dependency to 0.10.0.0-SNAPSHOT
- PR-300 - Using the new 0.9 Kafka consumer.
- PR-302 - Fix build to handle rack aware changes in Kafka.
- PR-305 - Update to match changed methods in CoreUtils
- PR-317 - Change the ‘host.name’ importance to high
- PR-319 - KafkaStore SSL support.
- PR-320 - API reference uses ‘integer’ Avro type which isn’t supported. ‘int’ is supported.
- PR-329 - https support.
Serializers, Formatters, and Converters¶
- PR-264 - Add null checks to json serializer/deserializer
- PR-274 - Add support for Avro projections in decoders
- PR-275 - Fixed references to confluent common version
- PR-276 - Unit tests and bugfix for NPE when using nested optional fields
- PR-278 - Test cases for optional nested structs
- PR-280 - Fix fromConnectData for optional complex types
- PR-290 - Issue #284 Cannot set max.schemas.per.subject due to cast exception
- PR-297 - Allows any CharSequence implementation to be considered a string
- PR-318 - Minor cleanup
- PR-323 - Fix #142 - handle parsing non-json responses in the RestService
- PR-332 - Add status storage topic to Connect Avro sample config.
- PR-141 - Incorrect path to log4j.properties file for simple zip file layout
- PR-143 - schema-registry-start does not work with -daemon argument
- PR-152 - Added point about Google code style to Readme.
- PR-163 - Expose more information when registry fails to start
- PR-165 - Add compatibility support for SchemaRegistryClient
- PR-167 - Update the versionCache with a new schemaVersionMap for a subject
- PR-169 - Use correct URL to update compatibility setting of a subject
- PR-180 - GH-177: Remove unneeded content-type headers for GET ops in quickstart and README
- PR-184 - Support multiple registry URLs in client
- PR-186 - Rename LocalSchemaRegistryClient to make it clear it is only intended to be used as a mock in tests. Fixes #185.
- PR-187 - Correct example response for POST /subjects/(string: subject)/versions
- PR-188 - Address GH-168; enable unit testing of CachedSchemaRegistryClient
- PR-195 - Fixed typo in Exception message
- PR-196 - Require Java 7
- PR-197 - Enable test code sharing
- PR-198 - Update jersey, jackson and junit versions to match rest-utils
- PR-202 - Correct minor docs error in example response
- PR-203 - Issue 194 rename main
- PR-207 - Issue #170: PUT /config/(string: subject) should return 4xx for unknown subjects
- PR-210 - Issue #208: Several RestApiTest test cases don’t test proper exception behavior
- PR-219 - Update Kafka version to 0.8.3-SNAPSHOT so we can start developing using upcoming 0.8.3 features.
- PR-237 - Update uses of ZkUtils to match changes made in KAFKA-2639.
- PR-240 - Fixes Typo in the docs. ‘actsas’ -> ‘acts as’
- PR-242 - Fixed Schema Registry build against kafka trunk
- PR-243 - Use x.y.z versioning scheme (i.e. 2.0.0-SNAPSHOT)
- PR-245 - Fix mvn assembly setup
- PR-252 - Use Kafka compiled with Scala 2.11
- PR-257 - Updated classpath in schema-registry-run-class to reflect changes in pom.xml
- PR-258 - CC-53: Add worker configs for Avro Kafka Connect that integrates with Schema Registry.
Serializers, Formatters, and Converters¶
- PR-146 - Added decoder to return a specific Avro record from bytes.
- PR-162 - Add implementation of new Deserializer interface.
- PR-192 - Add new module for Kafka JSON serialization stuff
- PR-193 - Add more JSON codec support
- PR-200 - Switch serializer config classes to use AbstractConfig from confluent-common instead of from Kafka.
- PR-222 - Add AvroConverter in new copycat-avro-converter jar to convert Copycat and Avro data.
- PR-234 - Add AvroConverter support for Decimal, Date, Time, and Timestamp logical types.
- PR-235 - Add caching of schema conversions in AvroData and AvroConverter.
- PR-247 - Update for Copycat -> Kafka Connect renaming.
- PR-251 - Add tests of conversion of null values from Kafka Connect and fix handling of null for int8 and int16 types.
- PR-254 - Ensure AvroConverter passes through null values without adding schemas and that deserialized null values are converted to SchemaAndValue.NULL.
- PR-255 - CC-44: Include version when deserializing Kafka Connect data.
- PR-256 - Encode null values for schemaless array entries and map keys and values as an Anything record.