# Limits for Fully Managed Connectors for Confluent Cloud Fully managed connectors for Confluent Cloud have usage limits that apply across all connectors, plus connector-specific limits. The following table is a quick reference to the general limits. Each row links to the section below that describes the limit in full. Each connector also has its own limits, listed in [Supported connector limitations](#connect-ccloud-supported-limits). | Limit | Value | Scope | |------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|-------------------| | [Maximum source connector request size](#cc-limits-source-connectors) | 8 MB on Basic, Standard, Enterprise, and Freight clusters; 20 MB on Dedicated clusters | Source connectors | | [Backslash in passwords or secret keys](#cc-limits-connector-configurations) | Not supported (`\` can cause connection failures) | All connectors | | [Sink connector dead letter queue (DLQ) topic](#cc-limits-sink-connectors) | Cannot be in the same connector’s consumed topics (prevents an infinite loop) | Sink connectors | | [Additional connector properties](#cc-limits-common-property-configuration) | Configurable through the Confluent REST API and Confluent CLI only, not the Confluent Cloud Console | All connectors | ## Connector Configurations Do not use `\` in passwords or secret keys when setting up the connector. It may lead to connection failures. ## Sink connectors You cannot add a sink connector’s [dead letter queue (DLQ)](dead-letter-queue.md#ccloud-dlq-topics) topic to the list of topics consumed by the same sink connector (to prevent an infinite loop). ## Source connectors An internal Apache Kafka® configuration property (`max.request.size`) controls the maximum producer request size. This size is set at 8 MB maximum for Basic, Standard, Enterprise, and Freight clusters, and 20 MB for Dedicated clusters. If you want to run a source connector making requests larger than 8 MB, you must run the connector in a [Dedicated cluster](../clusters/cluster-types.md#dedicated-cluster). ## Automatic topic creation If the Kafka broker configuration is set to `auto.create.topics.enable=false` and you delete a topic that was automatically created, the connector does not automatically create a new topic, even if the Connect worker has the property `topic.creation.enable=true`. To work around this issue, make a minor modification to the connector configuration so the connector can restart. The connector creates the new topic when it restarts after the configuration change. ## Common property configuration You can add additional properties only using the Confluent REST API and Confluent CLI. Configuring the additional properties using Confluent Cloud Console is not currently supported. #### NOTE You can configure the following properties for sink connectors though Confluent REST API or Confluent CLI. These properties are not supported on the Confluent Cloud Console. - `errors.deadletterqueue.topic.name` - The name of the topic to use as the Dead Letter Queue (DLQ). Messages are sent to the DLQ if they cause an error during processing by the sink connector, its transformations, or its converters. If you don’t set this property, it defaults to `dlq-${connector}`. The `${connector}` placeholder is automatically replaced with the connector’s name. The DLQ topic will be created automatically if it doesn’t already exist. - `reporter.result.topic.name` - The name of the topic where a success report is sent after a sink record is processed successfully. If you do not set this, it defaults to `success-${connector}`. The `${connector}` placeholder is automatically replaced with the connector’s name. - `reporter.error.topic.name` - The name of the topic where an error report is sent after a sink record fails to be processed. If this property is not set, it defaults to `error-${connector}`. The `${connector}` placeholder is automatically replaced with the connector’s name. - `topics.regex` - A regular expression (regex) used to match the names of topics to consume from. This property allows the connector to dynamically subscribe to any matching topics without requiring you to list each one individually. For example, the value `confluent.*` would match topics such as `confluent-1`, `confluent.abc`, and `confluentprod`. In Confluent Cloud, a broad `topics.regex` pattern can inadvertently match Confluent Cloud for Apache Flink® virtual topics, causing `UnknownTopicOrPartition` errors, timeouts, or failed connector tasks. For guidance on avoiding this, see [Topic subscription best practices](../client-apps/consumer.md#kafka-consumer-topic-subscription-best-practices). Before the connector accepts the configuration, Confluent Cloud also checks the pattern for ReDoS (regular expression denial of service) vulnerabilities with an internally defined time limit. If the pattern can’t be evaluated safely within the time limit, the configuration is rejected with the `The topic regex '' could not be validated within the time limit; simplify it for more efficient evaluation` error message. If Confluent Cloud evaluates the pattern and finds it unsafe or invalid, the configuration is rejected with the `The topic regex '' is unsafe or invalid` error message. To avoid this, write efficient regular expressions and avoid constructs prone to catastrophic backtracking, such as nested or overlapping quantifiers. ## Connector-Specific Limitations ### Supported connector limitations See the following limitations for supported connectors. #### ActiveMQ Source Connector The [ActiveMQ Source Connector for Confluent Cloud](cc-activemq-source.md#cc-activemq-source) has the following limitation: - The connector does not support Advanced Message Queuing Protocol (AMQP). #### AlloyDB Sink Connector The [AlloyDB Sink Connector for Confluent Cloud](cc-alloydb-sink.md#cc-alloydb-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - AlloyDB only provides a private IP, hence cannot be accessed publicly. Setup the [AlloyDB Auth Proxy](https://cloud.google.com/alloydb/docs/auth-proxy/connect) and ensure that it is accessible to the connector. - The database and Kafka cluster should be in the same region. - For tombstone records, set `delete.enabled` to `true` #### Amazon CloudWatch Logs Source Connector The [Amazon CloudWatch Logs Source Connector for Confluent Cloud](cc-amazon-cloudwatch-logs-source.md#cc-amazon-cloudwatch-logs-source) has the following limitations: * The connector does not support Protobuf. * **AWS API rate limits**: The connector is subject to AWS CloudWatch API rate limits that can affect performance: - **DescribeLogStreams API**: Limited to 25 requests per second per account per region, with a maximum of 50 log streams per request. During connector startup, discovering log streams takes time and grows with the number of log streams (approximately 4 seconds for every 5,000 streams). - **FilterLogEvents API**: Limited to 10 requests per second per account per region during normal operation. For deployments with large numbers of log streams, poll intervals may need to be increased to avoid rate limiting. * **Dynamic stream discovery**: The connector performs log stream discovery only at startup. New log streams added after connector startup will not be automatically detected and require a connector restart. #### Amazon CloudWatch Metrics Sink Connector The [Amazon CloudWatch Metrics Sink Connector for Confluent Cloud](cc-amazon-cloudwatch-metrics-sink.md#cc-amazon-cloudwatch-metrics-sink) has the following limitation: - The Amazon CloudWatch Metrics region must be in the same region where your Confluent Cloud cluster is, and where you are running the [Amazon CloudWatch Metrics Sink Connector for Confluent Cloud](cc-amazon-cloudwatch-metrics-sink.md#cc-amazon-cloudwatch-metrics-sink). #### Amazon DocumentDB Sink Connector The [Amazon DocumentDB Sink Connector for Confluent Cloud](cc-amazon-document-db-sink/cc-document-db-sink.md#cc-document-db-sink) has the following limitations: - If your DocumentDB database username or password includes any of the following characters: `$ : / ? # [ ] @`, you must convert the characters using [percent encoding](https://tools.ietf.org/html/rfc3986#section-2.1). - To use a schema-based message format, like AVRO, a valid schema must be available in Confluent Cloud Schema Registry. - The connector is compatible with Amazon DocumentDB engine versions 4.0, 5.0, and 8.0. It does not support connections to Amazon DocumentDB elastic clusters. - IAM role based authentication is only supported by DocumentDB engine version 5.0. This is an AWS limitation. - Amazon DocumentDB Elastic clusters are not yet supported. - For networking considerations, see [Private networking setup](cc-amazon-document-db-sink/cc-document-db-sink.md#cc-document-db-sink-connect-to-cluster). - You cannot use a dot in a field name (for example, `Client.Email`). If a field name includes a dot, the connector displays the following error. Do not use the `$` in a field name. For additional information, see [Field Names](https://docs.mongodb.com/manual/core/document/#field-names). ```none Your record has an invalid BSON field name. ``` - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Amazon DynamoDB Sink Connector The [Amazon DynamoDB Sink Connector for Confluent Cloud](cc-amazon-dynamo-db-sink.md#cc-amazon-dynamodb-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Amazon DynamoDB database and Kafka cluster should be in the same region. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Amazon DynamoDB CDC Source Connector The [Amazon DynamoDB CDC Source Connector for Confluent Cloud](cc-amazon-dynamodb-cdc-source/cc-amazon-dynamodb-cdc-source.md#cc-amazon-dynamodb-cdc-source) has the following limitations: 1. A table can only be processed by one task at a given time. Thus configuring more tasks than tables results in some tasks remaining idle. The maximum possible number of tasks is equal to the total number of configured tables. 2. The `dynamodb.snapshot.max.poll.records` configuration property tunes the maximum number of records that can be returned in a single DynamoDB read operation. Each read operation has a 1 MB size limit. 3. The `dynamodb.cdc.max.poll.records` configuration property tunes the maximum number of records that can be returned in single DynamoDB Streams [GetRecords](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_streams_GetRecords.html) operation. Each GetRecords operation has a 1 MB size limit. 4. You can only run up two connectors (or consumers) on a single DynamoDB table stream. This is not supported by AWS DynamoDB. 5. During `SNAPSHOT` mode, if the table’s pricing mode is provisioned with auto-scaling off, the connector throughout is limited by RCU constraints defined on the table. 6. A task might fail if a `SNAPSHOT` for a table takes longer than 24 hours–since CDC items are removed after 24 hours. To workaround this, ensure there are not too many tables assigned to each task, and their table RCU is adequate. 7. The connector requires setting `StreamViewType` on the stream to either `NEW_IMAGE` or `NEW_AND_OLD_IMAGES` to source data from DynamoDB Streams. 8. You cannot set the `cdc.start.position.timestamp` configuration property to a timestamp older than 24 hours, because DynamoDB Streams data expires after 24 hours. #### Amazon Kinesis Source Connector - The [Amazon Kinesis Source Connector for Confluent Cloud](cc-kinesis-source.md#cc-kinesis-source) does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.ValueToKey` - `org.apache.kafka.connect.transforms.HoistField$Value` - `org.apache.kafka.connect.transforms.ReplaceField$Value` - `org.apache.kafka.connect.transforms.ExtractField$Value` - `io.confluent.connect.transforms.ExtractTopic` #### Amazon Redshift Sink Connector The [Amazon Redshift Sink Connector for Confluent Cloud](cc-amazon-redshift-sink.md#cc-amazon-redshift-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Confluent Cloud cluster and the target Redshift cluster must be in the same AWS region. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector cannot consume data containing nested structs. - The connector does not support the Array data type. For supported data types, see [Data types](https://docs.aws.amazon.com/redshift/latest/dg/c_Supported_data_types.html). - The connector does not support Avro schemas that contain `decimal` logical types. For a better understanding of numeric data types, see this blog post: [Bytes, Decimals, Numerics and oh my](https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector/#bytes-decimals-numerics). - Provider integration with AWS IAM role-based authorization only supports provisioned Redshift clusters that use the format, `...redshift.amazonaws.com`. #### Amazon SQS Source Connector There are no current limitations for the [Amazon SQS Source Connector for Confluent Cloud](cc-amazon-sqs-source.md#cc-amazon-sqs-source). #### Amazon S3 Sink Connector The [Amazon S3 Sink Connector for Confluent Cloud](cc-s3-sink/cc-s3-sink.md#cc-s3-connect-sink) has the following limitations: - One task can handle up to 100 partitions, but is limited to 1 partition when using FieldPartitioner. - FieldPartitioner only supports top-level fields for partitioning. Nested fields in JSON structures are not supported for the `partition.field.name` property. - Partitioning (hourly or daily) is based on Kafka record time. - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. Performing a compatible schema change may cause the connector to flush data prior to whatever is configured for `flush.size`. - When the connector observes a schema change in the value field of an enum type, the schema compatibility check fails in all modes except when `schema.compatibility` is set to `NONE`. In the event of this failure, the connector writes the record to the DLQ. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The S3 Sink connector does not allow recursive schema types. Writing to Parquet output format with a recursive schema type results in a StackOverflowError. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Amazon S3 Source Connector The [Amazon S3 Source Connector for Confluent Cloud](cc-s3-source.md#cc-s3-connect-source) has the following limitations: - For a new bucket, you need to create a new connector with an unused name. If you reconfigure an existing connector to source from the new bucket, or create a connector with a name that is used for another connector, the connector will not source from the beginning of data stored in the bucket. This is because the connector will maintain offsets tied to the connector name. - The connector limits the number of objects it can index in a bucket to 10,000,000, as set by the `bucket.listing.max.objects.threshold` property. This limit helps manage large directories effectively. You must keep the object count under 10,000,000 to comply with bucket limitation. Confluent recommends creating **more top-level folders** to help process files with less delays and scale better in the long term. - The connector does not support `zstd` compression type. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.HoistField$Value` - `org.apache.kafka.connect.transforms.HoistField$Key` #### AWS Lambda Sink Connector The [AWS Lambda Sink Connector for Confluent Cloud](cc-aws-lambda-sink.md#cc-aws-lambda-sink) has the following limitation: - The Confluent Cloud cluster and your AWS Lambda project must be in the same AWS region. #### Azure Blob Storage Sink Connector The [Azure Blob Storage Sink Connector for Confluent Cloud](cc-azure-blob-sink/cc-azure-blob-sink.md#cc-azure-blob-sink) has the following limitations: - The Azure Blob Storage Container should be in the same region as your Confluent Cloud cluster. If you use a different region, be aware that you may incur additional data transfer charges. Contact [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and Azure Blob storage in different regions. - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - One task can handle up to 100 partitions, but is limited to 1 partition when using FieldPartitioner. - FieldPartitioner only supports top-level fields for partitioning. Nested fields in JSON structures are not supported for the `partition.field.name` property. - Partitioning (hourly or daily) is based on Kafka record time. - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. - `schema.compatibility` is set to `NONE`. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Azure Blob Storage Source Connector The [Azure Blob Storage Source Connector for Confluent Cloud](cc-azure-blob-source.md#cc-azure-blob-source) has the following limitations: - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - The connector ignores any object with a name that does not start with the configured `topics.dir` directory. This name is `topics/` by default. - The connector uses the connector name to store offsets that identify how much of the container it has processed. If you delete a connector and then use the same connector name for a new connector, the new connector will not reprocess data from the beginning of the container. The progress for the deleted connector is saved and the new connector starts from where the original connector’s processing ended. The connector can start processing earlier container data if the corresponding entry in the offset topic is cleared. - The connector limits the number of objects it can index in a bucket to 10,000,000, as set by the `bucket.listing.max.objects.threshold` property. This limit helps manage large directories effectively. You must keep the object count under 10,000,000 to comply with bucket limitation. Confluent recommends creating **more top-level folders** to help process files with less delays and scale better in the long term. - For a new container, you need to create a new connector. If you reconfigure an existing connector to source from the new container, the connector will not source from the beginning of data stored in the container. - The connector will not reload data during the following scenarios: * Renaming a file that the connector has already read. * Uploading a newer version of an existing file with a new record. - If a shared access signature (SAS) token is used, the connector requires an account-level SAS token. A service-level (container) SAS token will not work. - There are compatibility constraints for certain input data formats. | Output data format | Supported input formats | |----------------------|---------------------------| | PROTOBUF, JSON_SR | BYTES, AVRO | | JSON, AVRO, STRING | AVRO, JSON, BYTES, STRING | | BYTES | STRING, BYTES | - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.HoistField$Value` - `org.apache.kafka.connect.transforms.HoistField$Key` #### Azure Cognitive Search Sink Connector The [Azure Cognitive Search Sink Connector for Confluent Cloud](cc-azure-cognitive-search-sink.md#cc-azure-cognitive-search-sink) has the following limitations: - Batching multiple metrics: The connector tries to batch metrics in a single payload. The maximum payload size is 16 megabytes for each API request. For additional details, refer to [Size limits per API call](https://docs.microsoft.com/en-us/azure/search/search-limits-quotas-capacity#document-size-limits-per-api-call). - The Azure Cognitive Search service must be in the same region as your Confluent Cloud cluster. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Azure Cosmos DB Sink Connector The [Azure Cosmos DB Sink Connector [Deprecated] for Confluent Cloud](cc-azure-cosmos-sink.md#cc-azure-cosmos-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Azure Cosmos DB must be in the same region as your Confluent Cloud cluster. - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - The Kafka topic must not contain tombstone records. The connector does not handle tombstone or null values. #### Azure Cosmos DB Source Connector The [Azure Cosmos DB Source Connector [Deprecated] for Confluent Cloud](cc-azure-cosmos-source.md#cc-azure-cosmos-source) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - [Azure Cosmos DB serverless](https://learn.microsoft.com/en-us/azure/cosmos-db/serverless) mode is not supported. - The Kafka record key is serialized by StringConverter. - The task-to-container ratio is 1:1, and only a single task processes data for any given container. #### Azure Data Lake Storage Gen2 Sink Connector The [Azure Data Lake Storage Gen2 Sink Connector for Confluent Cloud](cc-azure-datalakeGen2-storage-sink.md#cc-azure-datalakegen2storage-sink) has the following limitations: - Azure Data Lake storage should be in the same region as your Confluent Cloud cluster. If you use a different region, be aware that you may incur additional data transfer charges. Contact [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and Azure Data Lake storage in different regions. - Input format JSON to output format AVRO does not work for the preview connector. - One task can handle up to 100 partitions. - Partitioning (hourly or daily) is based on Kafka record time. - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. - `schema.compatibility` is set to `NONE`. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Using a recursive schema type is not allowed and will result in a StackOverflowError. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - When write once, read many (WORM) state is enabled on the container, the connector only supports at-least-once delivery semantics. #### Azure Event Hubs Source Connector The [Azure Event Hubs Source Connector for Confluent Cloud](cc-azure-event-hubs-source.md#cc-azure-event-hubs-source) has the following limitations: - `max.events`: `499` is the maximum number of events allowed. Defaults to `50`. - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.ValueToKey` - `org.apache.kafka.connect.transforms.HoistField$Value` #### Azure Functions Sink Connector The [Azure Functions Sink Connector for Confluent Cloud](cc-azure-functions-sink.md#cc-azure-functions-sink) has the following limitations: - The target Azure Function must be in the same region as your Confluent Cloud cluster. - The target Azure Function must be on a host using a public IP address. For more information, see [Manage Networking for Confluent Cloud Connectors](networking/internet-resource.md#clusters-connect-cloud). #### Azure Log Analytics Sink Connector The [Azure Log Analytics Sink Connector for Confluent Cloud](cc-azure-log-analytics-sink.md#cc-azure-log-analytics-sink) has the following limitations: - There is a 30 MB per post size limit when posting to the Azure Monitor Data Collector API. This size limit is for a single post. - There is a 32 KB field value size limit. If the field value is greater than 32 KB data will be truncated. - The recommended maximum number of fields for a given type is 50. This is a practical limit based on usability testing. - Tables in Azure Log Analytics workspaces can support 500 columns maximum. - Column names can have a maximum of 45 characters. - Table names can have a maximum of 100 characters. Table names must start with letters and can only contain letters, numbers, and the underscore character (_). #### Azure Log Analytics Sink V2 Connector The [Azure Log Analytics Sink V2 Connector for Confluent Cloud](cc-azure-log-analytics-sink-v2.md#cc-azure-log-analytics-sink-v2) has the following limitations: - The maximum number of tables per connector is 20, matching the Azure limit of 20 streams per Data Collection Rule (DCR). - The maximum size of an API call to the Azure Logs Ingestion API is 1 MB. Because batching is record-count based (controlled by `batch.size`, default `500`), tune `batch.size` downward if you have large records. - A `413 Payload Too Large` response is not retried. An oversize batch fails on the first attempt and the connector routes records to the error topic based on the `behavior.on.error` setting. - The maximum size for field values is 64 KB. - The maximum data ingested per minute per DCR is 2 GB. - The maximum number of requests per minute per DCR is `12,000`. For the full list of Azure Monitor service limits, see [Azure Monitor service limits](https://learn.microsoft.com/en-us/azure/azure-monitor/service-limits). - Azure Log Analytics Auxiliary-plan tables enforce a 30-minute `TimeGenerated` window. Azure silently drops records with timestamps outside this window (returns HTTP 204, but does not load any row). Use Analytics-plan tables for streaming Kafka data. For more information, see [Log data plans](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs#log-data-plans). - Azure drops fields not declared in the DCR’s stream schema at ingestion. Azure returns HTTP 204 (success), but does not index the field or row. To verify ingestion, run KQL queries against the target table. #### Azure Service Bus Source Connector The [Azure Service Bus Source Connector for Confluent Cloud](cc-azure-service-bus-source.md#cc-azure-service-bus-source) has the following limitations: - For JSON, JSON_SR, AVRO and PROTOBUF, the message body (`messageBody`) produced by the connector contains JSON or text in base64 encoded format. - For using a private DNS zone/server to configure the Azure Service Bus Source connector to a private endpoint, see **DNS Support** in [Manage Networking for Confluent Cloud Connectors](networking/internet-resource.md#clusters-connect-cloud). - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - This connector does not support egress IP addresses when your Confluent Cloud cluster is running on Azure. Public inbound traffic access (0.0.0.0/0) must be allowed for this connector if connecting via public endpoint. #### Azure Synapse Analytics Sink Connector The [Azure Synapse Analytics Sink Connector for Confluent Cloud](cc-azure-synapse-analytics-sink.md#cc-azure-synapse-analytics-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - This connector can only insert data into an Azure SQL data warehouse database. Azure Synapse Analytics does not support primary keys. Since updates, upserts, and deletes are all performed on the primary keys, these queries are not supported for this connector. - When `auto.evolve` is enabled, if a new column with a default value is added, that default value is only used for new records. Existing records will have `"null"` as the value for the new column. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### ClickHouse Sink Connector The [Get Started with the ClickHouse Sink Connector for Confluent Cloud](cc-clickhouse-sink-connector/cc-clickhouse-sink.md#cc-clickhouse-sink) has the following limitations: - The connector does not support private networking on Azure. - The connector does not support delete operations. #### Couchbase Sink Connector The [Couchbase Sink Connector for Confluent Cloud](cc-couchbase-db-sink/cc-couchbase-db-sink.md#cc-couchbase-sink) has the following limitations: - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like AVRO. - The data system the sink connector is connecting to should be in the same region as your Confluent Cloud cluster. If you use a different region or cloud platform, be aware that you may incur additional data transfer charges. Contact your Confluent account team or [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and connect to a data system that is in a different region or on a different cloud platform. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The connector does not support private networking on GCP. #### Couchbase Source Connector The [Get Started with the Couchbase Source Connector for Confluent Cloud](cc-couchbase-db-source/cc-couchbase-db-source.md#cc-couchbase-source) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The connector does not support private networking on GCP. - The connector does not support CSFLE. - The connector does not support schema. #### Databricks Delta Lake Sink The [Databricks Delta Lake Sink Connector for Confluent Cloud](cc-databricks-delta-lake-sink/cc-databricks-delta-lake-sink.md#cc-databaricks-delta-lake-sink-configure-connector) has the following limitations: - The connector is available only on Amazon Web Services (AWS). - The Amazon S3 bucket (where data is staged), the Delta Lake instance, and the Kafka cluster must be in the same region. - You cannot configure multiple connectors that consume from the same topic and that use the same Amazon S3 staging bucket. - Exactly-once semantics functionality is not currently supported. - To use multiple tasks, your Databricks workspace must be using Databricks Runtime version 10.5 or later. For earlier Databricks Runtime versions, the connector is limited to a single task per connector. - The connector does not support Array, Map, or Struct field schemas. - The connector appends data only. - The connector uses the UTC timezone. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Datadog Metrics Sink Connector The [Datadog Metrics Sink for Confluent Cloud](cc-datadog-metrics-sink.md#cc-datadog-metrics-sink) has the following limitations: - Batching multiple metrics: The connector tries to batch metrics in a single payload. The maximum payload size is 3.2 megabytes for each API request. For additional details, refer to [Post timeseries points](https://docs.datadoghq.com/api/latest/metrics/#submit-metrics). - Metrics Rate Limiting: The API endpoints are [rate limited](https://docs.datadoghq.com/api/latest/rate-limits/). The rate limit for metrics retrieval is 100 per hour, per organization. These limits can be modified by contacting Datadog support. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Elasticsearch Service Sink Connector The [Elasticsearch Service Sink Connector [Deprecated] for Confluent Cloud](cc-elasticsearch-service-sink.md#cc-elasticsearch-service-sink) has the following limitations: - The connector works with Elastic Cloud and the official Elasticsearch distribution from [Elastic](https://www.elastic.co/). Open source versions and derivatives like Amazon OpenSearch are not supported. - The connector does not support Amazon Elasticsearch Service (now Amazon OpenSearch Service). For Elasticsearch or OpenSearch deployments on AWS, use the [OpenSearch Sink connector](cc-opensearch-sink.md#cc-opensearch-sink) instead. - The connector has been tested with versions up to Elasticsearch 8.x. - The Confluent Cloud cluster and the target Elasticsearch deployment must be in the same region. - The `batch.size` property is limited to 4,000 records. - Topic Mutating SMTs won’t work when using external resources and doing so would cause connector failures. #### Elasticsearch Service Sink V2 Connector The [Elasticsearch Sink V2 Connector for Confluent Cloud](cc-elasticsearch-sink-v2/cc-elasticsearch-sink-v2.md#cc-elasticsearch-sink-v2) has the following limitations: - The connector is tested to work with Elastic Cloud and the official Elasticsearch distribution from [Elastic](https://www.elastic.co/). Open source versions and derivatives like AWS OpenSearch are not supported. - The connector has been tested with Elasticsearch `8.x` & `9.x` server versions. - Topic Mutating SMTs won’t work when using external resources and doing so would cause connector failures. - Elasticsearch serverless currently does not support PrivateLink connectivity. Thus, PrivateLink cannot be used when connecting the Elasticsearch Sink connector to serverless deployments. #### GitHub Source Connector The [GitHub Source Connector for Confluent Cloud](cc-github-source.md#cc-github-source) has the following limitation: * Because of a GitHub API limitation, only one task per connector is supported. #### Google BigTable Sink Connector The [Google Cloud BigTable Sink Connector for Confluent Cloud](cc-gcp-bigtable-sink.md#cc-gcp-bigtable-sink) has the following limitations: * Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). * The database and the Kafka cluster should be in the same region. #### Google BigQuery Sink (Legacy) Connector The [Google Cloud BigQuery Sink Connector [End of Life] for Confluent Cloud](cc-gcp-bigquery-sink.md#cc-gcp-bigquery-sink) has the following limitations: - The data system the sink connector is connecting to should be in the same region as your Confluent Cloud cluster. If you use a different region or cloud platform, be aware that you may incur additional data transfer charges. Contact your Confluent account team or [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and connect to a data system that is in a different region or on a different cloud platform. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - Source topic names must comply with BigQuery naming conventions even if `sanitizeTopics` is set to `true` in the connector configuration. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector does not support schemas with recursion. - Auto schema update does not support column removal. - Auto schema update does not support recursive schemas. - DLQ routing does not work if **Auto update schemas** (`auto.update.schemas`) is enabled and the connector detects that the failure is due to schema mismatch. - Topic names are mapped to BigQuery table names. For example, if you have a topic named `pageviews`, a topic named `visitors`, and a dataset named `website`, the result is two tables in BigQuery; one named `pageviews` and one named `visitors` under the `website` dataset. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Google BigQuery Sink V2 Connector The [Google BigQuery Sink V2 Connector for Confluent Cloud](cc-gcp-bigquery-storage-sink.md#cc-gcp-bigquery-storage-api-sink) has the following limitations: - Primary keys are required to run the connector in `UPSERT` or `UPSERT_DELETE` mode. You must ensure that there aren’t any null primary keys as ingesting new and existing records with null primary keys may result in unreliable behavior. For more details, see the [user responsibilities](https://cloud.google.com/blog/products/data-analytics/join-optimizations-with-bigquery-primary-and-foreign-keys#:~:text=User%20responsibilities) for primary and foreign keys with BigQuery. Note that only Kafka record keys are treated as primary keys, and [BigQuery CDC limitations](https://cloud.google.com/bigquery/docs/change-data-capture#limitations) still apply. - When the primary key is configured as the record key, the connector automatically excludes fields with unsupported data types (such as STRUCT, FLOAT, and BYTES) from the primary key definition to ensure compatibility with BigQuery. These excluded fields are still included in the table as regular columns. This behavior occurs because BigQuery does not support these data types as primary keys. For more information about supported data types for primary keys, see [BigQuery Primary Key Limitations](https://cloud.google.com/bigquery/docs/primary-foreign-keys#limitations). - The data system the connector is connecting to should be in the same region as your Confluent Cloud cluster. If you use a different region or cloud platform, you may incur additional data transfer charges. Contact your Confluent account team or [Confluent Support](https://support.confluent.io/) if you need to use Confluent Cloud and connect to a data system that is in a different region or on a different cloud platform. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - Source topic names must comply with BigQuery naming conventions even if `sanitize.topics` is set to `true` in the connector configuration. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector does not support schemas with recursion. - Auto schema update does not support column removal. - Auto schema update does not support recursive schemas. - DLQ routing does not work if **Auto update schemas** (`auto.update.schemas`) is enabled and the connector detects that the failure is due to schema mismatch. - Topic names are mapped to BigQuery table names. For example, if you have a topic named `pageviews`, a topic named `visitors`, and a dataset named `website`, the result is two tables in BigQuery; one named `pageviews` and one named `visitors` under the `website` dataset. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Google Cloud Firestore Sink Connector The [Google Cloud Firestore Sink Connector for Confluent Cloud](cc-gcp-firestore-sink.md#cc-gcp-firestore-sink) has the following limitations: * The connector does not support private networking. * The connector does not support Firestore in native mode. #### Google Functions Sink Connector The [Google Cloud Functions Sink Connector [End of Life] for Confluent Cloud](cc-google-functions-sink.md#cc-google-functions-sink) has the following limitations: - The target Google Function should be in the same region as your Confluent Cloud cluster. - The connector does not currently support [Google Cloud Functions (2nd gen)](https://cloud.google.com/functions/docs/concepts/version-comparison). - The connector does not support private networking (for example, Private Service Connect egress). This is because the destination URL uses a `cloudfunctions.net` domain, and Google does not provide the necessary TLS certificates for customer-provisioned load balancers. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Google Pub/Sub Source Connector There are no current limitations for the [Google Cloud Pub/Sub Source Connector for Confluent Cloud](cc-google-pubsub-source.md#cc-google-pubsub-source). #### Google Cloud Spanner CDC Source (Debezium) Connector The [Google Cloud Spanner CDC Source (Debezium) Connector for Confluent Cloud](cc-google-spanner-cdc-source-debezium.md#cc-google-spanner-cdc-source-debezium) has the following limitations: - The connector does not support streaming [snapshot events](https://debezium.io/blog/2021/10/07/incremental-snapshots/). The first time the connector connects to a Spanner database, it streams changes from either the configured start timestamp or the current timestamp if you do not provide a start timestamp. - The connector does not support task autoscaling. It runs with a fixed number of tasks and cannot automatically add or remove tasks when a change stream splits or merges partitions. Because automatically scaling tasks up and down triggers frequent rebalances, and every rebalance moves partitions between tasks and briefly pauses change streaming. You must configure an adequate number of tasks during initial setup as allocating too few tasks increases the number of partitions per task, leading to higher consumer lag. - The connector does not support the PostgreSQL interface for Cloud Spanner. - The connector does not create change streams automatically. You must create a Spanner change stream on the database before you configure the connector. - When you enable low watermark, you cannot use [Debezium topic routing transformations](https://debezium.io/documentation/reference/stable/transformations/topic-routing.html). - Depending on the service environment, certain network access limitations can exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. #### Google Cloud Spanner Sink Connector The [Google Cloud Spanner Sink Connector for Confluent Cloud](cc-gcp-spanner-sink.md#cc-gcp-spanner-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Confluent Cloud cluster and the target Google Spanner cluster must be in the same Google Cloud region. - A valid schema must be available in Confluent Cloud Schema Registry to use Avro, JSON Schema, or Protobuf. - The connector does not support PostgreSQL dialect. - The connector does not support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Google Cloud Storage Sink Connector The [Google Cloud Storage Sink Connector for Confluent Cloud](cc-gcs-sink.md#cc-gcs-connect-sink) has the following limitations: - The data system the sink connector is connecting to should be in the same region as your Confluent Cloud cluster. If you use a different region or cloud platform, be aware that you may incur additional data transfer charges. Contact your Confluent account team or [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and connect to a data system that is in a different region or on a different cloud platform. - One task can handle up to 100 partitions, but is limited to 1 partition when using FieldPartitioner. - FieldPartitioner only supports top-level fields for partitioning. Nested fields in JSON structures are not supported for the `partition.field.name` property. - Partitioning (hourly or daily) is based on Kafka record time. - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. - `schema.compatibility` is set to `NONE`. - If output format BYTES is selected, the input message format must also be BYTES. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Using a recursive schema type is not allowed and will result in a StackOverflowError. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Google Cloud Storage Source Connector The [Google Cloud Storage Source Connector for Confluent Cloud](cc-gcs-source.md#cc-gcs-connect-source) has the following limitations: - The connector ignores any GCS object with a name that does not start with the configured `topics.dir` directory. This name is `topics/` by default. - The connector uses the connector name to store offsets that identify how much of the bucket it has processed. If you delete a connector and then use the same connector name for a new connector, the new connector will not reprocess data from the beginning of the bucket. The progress for the deleted connector is saved and the new connector starts from where the original connector’s processing ended. The connector can start processing earlier bucket data if the corresponding entry in the offset topic is cleared. - The connector limits the number of objects it can index in a bucket to 10,000,000, as set by the `bucket.listing.max.objects.threshold` property. This limit helps manage large directories effectively. You must keep the object count under 10,000,000 to comply with bucket limitation. Confluent recommends creating **more top-level folders** to help process files with less delays and scale better in the long term. - For a new bucket, you need to create a new connector. If you reconfigure an existing connector to source from the new bucket, the connector will not source from the beginning of data stored in the bucket. - The connector will not reload data during the following scenarios: * Renaming a file that the connector has already read. * Uploading a newer version of an existing file with a new record. - There are compatibility constraints for certain input data formats. | Output data format | Supported input formats | |----------------------|---------------------------| | PROTOBUF, JSON_SR | BYTES, AVRO | | JSON, AVRO, STRING | AVRO, JSON, BYTES, STRING | | BYTES | STRING, BYTES | - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.HoistField$Value` - `org.apache.kafka.connect.transforms.HoistField$Key` #### HTTP Sink Connector The [HTTP Sink Connector for Confluent Cloud](cc-http-sink.md#cc-http-sink) has the following limitations: - The Confluent Cloud Kafka consumer configuration property `max.poll.interval.ms` is set to `300000` milliseconds (5 minutes). This is a hard-coded property. If the sink connector takes longer than five minutes to complete processing and overshoots the poll interval, the connector is kicked out of the consumer group. This results in a failure to commit offsets and duplicate messages. - When using self-signed certificates, the connector only validates the server if the hostname is in the Subject Common Name (CN) field. Validation fails if the hostname is only in the Subject Alternative Name (SAN) field. #### HTTP Source Connector The [HTTP Source Connector for Confluent Cloud](cc-http-source.md#cc-http-source) has the following limitations: - The connector does not support APIs that rely on timestamp range-based queries. - The connector cannot parse responses in any format other than JSON. - When using self-signed certificates, the connector only validates the server if the hostname is in the Subject Common Name (CN) field. Validation fails if the hostname is only in the Subject Alternative Name (SAN) field. #### HTTP Sink V2 Connector The [HTTP Sink V2 Connector for Confluent Cloud](cc-http-sink-v2.md#cc-http-sink-v2) has the following limitations: - The Confluent Cloud user interface currently does not support adding, removing, or updating APIs for the OpenAPI specification. In this case, you must create a new connector. To workaround this limitation, use the **Switch to JSON** feature to use the raw JSON to configure the connector. - The connector only supports HTTP/1.x protocol and RESTful APIs. - The connector’s **Test API** feature is only available for publicly accessible endpoints (with no IP restrictions). - If CSFLE is enabled, the actual payload won’t be available in the headers of the error topic. - Each topic can be routed to only one API path. If the same topic is included in the topics list for more than one API path, the connector routes records from that topic to the last-configured API path only. - When using self-signed certificates, the connector only validates the server if the hostname is in the Subject Common Name (CN) field. Validation fails if the hostname is only in the Subject Alternative Name (SAN) field. #### HTTP Source V2 Connector The [HTTP Source V2 Connector for Confluent Cloud](cc-http-source-v2.md#cc-http-source-v2) has the following limitations: - The Confluent Cloud user interface currently does not support adding, removing, or updating APIs for the OpenAPI specification. In this case, you must create a new connector. To workaround this limitation, use the **Switch to JSON** feature to use the raw JSON to configure the connector. - The connector only supports HTTP/1.x protocol and RESTful APIs. - When using self-signed certificates, the connector only validates the server if the hostname is in the Subject Common Name (CN) field. Validation fails if the hostname is only in the Subject Alternative Name (SAN) field. #### IBM MQ Source Connector There are no current limitations for the [IBM MQ Source Connector for Confluent Cloud](cc-ibmmq-source.md#cc-ibmmq-source). #### IBM MQ Sink Connector The [IBM MQ Sink Connector for Confluent Cloud](cc-ibm-mq-sink.md#cc-ibm-mq-sink) has the following limitations: - The connector does not support Custom Credentials Provider on Confluent Cloud. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### IBM Db2 Source JDBC Connector The [IBM Db2 Source (JDBC) Connector for Confluent Cloud](cc-ibmdb2-source.md#cc-ibmdb2-source) has the following limitations: - The connector supports only LUW-based IBM Db2 databases, which includes Linux, UNIX, and Windows. - Connector startup time increases with the size of the source data. Task processing startup time increases proportionally with the number of tasks due to database query execution overhead, resulting in delayed record processing. - IBM Db2 transaction log size limitations cause extended startup delays when parallel tasks execute `SELECT` queries on large tables, frequently resulting in query execution timeouts. - When `output.data.format` is set to `STRING`, row values are emitted as a Kafka Connect `Struct` string representation (`Struct{COL1=value1,COL2=value2,...}`) without an accompanying schema. Downstream consumers must parse this representation manually. - When `output.data.format` is set to `JSON`, set `value.converter.schemas.enable` to `true` so that each emitted record includes an inline `{schema, payload}` envelope. The embedded schema preserves the type of every field and allows downstream systems to consume the records without having to infer the schema from the payload. - If you plan to use one or more Single Message Transformations (SMTs), see [SMT Limitations](single-message-transforms.md#cc-single-message-transforms-limitations). #### Jira Source Connector The [Jira Source Connector for Confluent Cloud](cc-jira-source.md#cc-jira-source) has the following limitations: - To use a schema-based output format, you must set schema compatibility to `NONE` in Schema Registry. - For Schema Registry-based output formats, the connector attempts to deduce the schema based on the source API response returned. The connector registers a new schema for every NULL and NOT NULL value of an optional field in the API response. For this reason, the connector may register schema versions at a much higher rate than expected. - Resources which do not support fetching records by datetime will have duplicate records and will be fetched repeatedly at a duration specified by the `request.interval.ms` configuration property. - The connector is not able to detect data deletion on Jira. - The connector does not guarantee accurate record order in the Apache Kafka® topic. - The timezone set by the user (defined in the `jira.username` configuration property) must match the general setting Jira timezone used for the connector. #### InfluxDB 2 Sink Connector The [InfluxDB 2 Sink Connector for Confluent Cloud](cc-influxdb2-sink.md#cc-influxdb2-sink) has the following limitation: - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### InfluxDB 3 Sink Connector The [InfluxDB 3 Sink Connector for Confluent Cloud](cc-influx-db3-sink.md#cc-influxdb3-sink) has the following limitations: - The connector does not support processing complex data types like Maps and arrays. - The connector does not support Private Link connection for InfluxDB cloud-hosted database. It support Private Link connection for self-hosted InfluxDB 3 database. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### InfluxDB Source Connector There are no current limitations for the [InfluxDB 2 Source Connector for Confluent Cloud](cc-influxdb2-source.md#cc-influxdb2-source). #### MariaDB CDC Source (Debezium) Connector The [MariaDB CDC Source (Debezium) Connector for Confluent Cloud](cc-mariadb-cdc-source-debezium.md#cc-mariadb-source-cdc-debezium-configure-connector) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Organizations can run multiple connectors with a limit of one task per connector (`"tasks.max": "1"`). - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. #### Microsoft SQL Server Sink Connector The [Microsoft SQL Server Sink (JDBC) Connector for Confluent Cloud](cc-microsoft-sql-server-sink.md#cc-microsoft-sql-server-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - Active Directory authentication is not currently supported. - The database and Kafka cluster should be in the same region. If you use a different region, you may incur additional data transfer charges. - For tombstone records, set `delete.enabled` to `true`. - When `input.data.format` is set to `STRING`, the entire Kafka record value is written into a single column specified by `string.output.value.column.name` (default `record_value`). The connector does not support splitting a `STRING` value across multiple columns. - When `input.data.format` is set to `STRING`, the value of `string.output.value.column.name` must not match any column listed in `pk.fields` for the target table. - When `input.data.format` is set to `STRING`, `pk.mode=record_value` is not supported. Only `pk.mode=none`, `pk.mode=kafka`, and `pk.mode=record_key` are valid with `STRING` values. - When `input.data.format` is set to `STRING` and `auto.create` is set to `true`, the column specified by `string.output.value.column.name` is created using the default string data type of the target database. - When `input.data.format` is set to `JSON`, schemaless JSON with (`value.converter.schemas.enable=false`) is not supported. Records must include an inline `{schema, payload}` envelope with (`value.converter.schemas.enable=true`), because the connector requires a value schema to map fields onto database columns. #### Microsoft SQL Server CDC Source Connector (Debezium) [Legacy] The [Microsoft SQL Server CDC Source (Debezium) Connector [End of Life] for Confluent Cloud](cc-microsoft-sql-server-source-cdc-debezium.md#cc-microsoft-sql-server-source-cdc-debezium) has the following limitations: - Change data capture (CDC) is only available in the Enterprise, Developer, Enterprise Evaluation, and Standard editions. - Active Directory authentication is not currently supported. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). #### Microsoft SQL Server CDC Source V2 (Debezium) Connector The [Microsoft SQL Server CDC Source Connector V2 (Debezium) for Confluent Cloud](cc-microsoft-sql-server-source-cdc-v2-debezium/index.md#cc-microsoft-sql-server-source-cdc-v2-debezium) has the following limitations: - Change data capture (CDC) is only available in the Enterprise, Developer, Enterprise Evaluation, and Standard editions. - Active Directory authentication is not currently supported. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The signaling does not work when the connector is configured with multiple databases. In such cases, the connector can not execute incremental snapshots, as they require signals to be produced into the signaling table. - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. - The connector supports only the millisecond-precision timestamp field `ts_ms` in the change event envelope, which indicates the time at which the connector processed the event. Higher-precision fields such as `ts_us` and `ts_ns` are not supported. - Large transactions or a high volume of change events during the streaming phase can fill the connector’s underlying JDBC driver’s result buffer, causing memory spikes and repeated connector restarts with little or no streaming progress. When possible, split large transactions into smaller batches to avoid this. If you can’t avoid large transactions, mitigate the issue as follows: - Lower `max.iteration.transactions` from its default of `500` to as low as `1` to reduce the connector’s memory footprint. This temporary mitigation can reduce throughput. - If lowering `max.iteration.transactions` alone doesn’t resolve the issue, particularly when you include multiple tables, contact [Confluent Support](https://support.confluent.io/) to request that `database.selectMethod=cursor` be applied to the connector. With this override, the source database streams the result set to the connector using a server-side cursor instead of returning it all at once. This bounds the connector’s memory use when processing large transactions. But it can increase `tempdb` usage on the source database during such large transactions. - Lowering `max.iteration.transactions` trades some performance for the connector’s memory stability, and applying `database.selectMethod=cursor` trades server-side memory for the same. Revert `max.iteration.transactions` and `database.selectMethod=cursor` to their default values after the large transactions have fully streamed through. #### Microsoft SQL Server Source (JDBC) Connector The [Microsoft SQL Server Source (JDBC) Connector for Confluent Cloud](cc-microsoft-sql-server-source.md#cc-microsoft-sql-server-source) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - You cannot use [public egress IP addresses](static-egress-ip.md#cc-static-egress-ips) (IP address allowlisting) for the connector. Azure provides secure and direct private service endpoints to Azure services. For more information, see [Service and gateway endpoints](/cloud/current/connectors/internet-resources.html#service-and-gateway-endpoints). - Active Directory authentication is not currently supported. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - A timestamp column must not be nullable for the *timestamp* or *timestamp+incrementing* mode and should be [datetime2](https://docs.microsoft.com/en-us/sql/t-sql/data-types/datetime2-transact-sql?view=sql-server-ver15). - If the connector is making numerous parallel insert operations in a large source table, insert transactions can commit out of order (this is typical). What this means is that a “greater” auto_increment ID (for example, 101) is committed earlier and a “smaller” ID (for example, 100) is committed later. The time difference here may only be a few milliseconds, but the commits are out of order nevertheless. Note that using *incrementing* mode to load data from such tables always results in some data loss. This happens because when the source connector worker reads (polls) the table, the connector gets the row with a greater offset value (with the smaller offset row remaining uncommitted). In the next iteration, although the uncommitted row is committed, the offset position has moved beyond that value, so the row is skipped. Using *timestamp+incrementing* mode is not a good choice either, because the tables may be very large (five to eight million rows added daily) and there is a high cost for any indexing approach, with the exception of PK indexing. - When `output.data.format` is set to `STRING`, row values are emitted as a Kafka Connect `Struct` string representation (`Struct{COL1=value1,COL2=value2,...}`) without an accompanying schema. Downstream consumers must parse this representation manually. - When `output.data.format` is set to `JSON`, set `value.converter.schemas.enable` to `true` so that each emitted record includes an inline `{schema, payload}` envelope. The embedded schema preserves the type of every field and allows downstream systems to consume the records without having to infer the schema from the payload. - The connector doesn’t allow the use of special characters in table names (for example, `$`), which are not allowed in Kafka topic names. To work around this limitation, change the name of the target topic using the [ExtractTopic Single Message Transformation (SMT)](/platform/current/connect/transforms/extracttopic.html). Note that this SMT helps you extract the topic name from the key or value of the message; therefore, you must include the desired topic name in the payload. - The connector does not accept zero date values, for example, `0000-00-00` when using Date fields. You can convert them to NULL by using the driver property `tableName?zeroDateTimeBehavior=convertToNull` in the connection string. - The connector can skip records in `timestamp+incrementing` mode when multiple transactions write to the source database table, and some transactions commit later. If a transaction with a higher timestamp and incrementing ID commits first and is read by the connector, the connector will ignore any later transactions with older timestamps and incrementing IDs. As a workaround, set the `timestamp.delay.interval.ms` in the connector to match the transaction timeout on the database side. This ensures the connector considers records that are committed later, as long as they are committed within the `timestamp.delay.interval.ms` timeframe. #### MongoDB Atlas Sink Connector The [MongoDB Atlas Sink Connector for Confluent Cloud](cc-mongo-db-sink/cc-mongo-db-sink.md#cc-mongo-db-sink) has the following limitations: - If your MongoDB database username or password includes any of the following characters: `$ : / ? # [ ] @`, you must convert the characters using [percent encoding](https://tools.ietf.org/html/rfc3986#section-2.1). - Document post processing configuration properties are not supported. These include: - `post.processor.chain` - `key.projection.type` - `value.projection.type` - `field.renamer.mapping` - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The data system the sink connector is connecting to should be in the same region as your Confluent Cloud cluster. If you use a different region or cloud platform, be aware that you may incur additional data transfer charges. Contact your Confluent account team or [Confluent Support](https://support.confluent.io) if you need to use Confluent Cloud and connect to a data system that is in a different region or on a different cloud platform. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - You cannot use a dot in a field name (for example, `Client.Email`). The error shown below is displayed if a field name includes a dot. You should also not use `$` in a field name. For additional information, see [Field Names](https://docs.mongodb.com/manual/core/document/#field-names). ```none Your record has an invalid BSON field name. You can check the MongoDB documentation for details. ``` - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### MongoDB Atlas Source Connector The [MongoDB Atlas Source Connector for Confluent Cloud](cc-mongo-db-source.md#cc-mongo-db-source) has the following limitations: - [MongoDB Time Series Collections](https://www.mongodb.com/docs/manual/core/timeseries-collections/#time-series-collections) are not supported with this connector. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The connector supports running a single task. - When `output.schema.infer.value` is set to `true`, the connector infers the schema from each incoming MongoDB document independently. Pre-registered schemas on the topic aren’t used, and documents with varying structures may result in many schema versions being registered in Confluent Cloud Schema Registry. For more information, see [Schema FAQ](cc-mongo-db-source.md#cc-mongodb-source-schema-infer-faq). #### MongoDB CDC Source (Debezium) Connector The [MongoDB CDC Source (Debezium) Connector for Confluent Cloud](cc-mongodb-cdc-source/cc-mongodb-cdc-source-debezium.md#cc-mongodb-source-cdc-debezium-configure-connector) has the following limitations: - Depending on the service environment, certain network access limitations can exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format like Avro. - You can run several connectors with a limit of one task per connector using `"tasks.max": "1"`. - The connector is compatible with the MongoDB versions 4.4 through 8.0. - Setting the `capture.scope` to `collection` disables the `source` signaling channel. Use the `kafka` signal channel for ad-hoc operations in this mode. - The connector does not currently support the following Single Message Transformations (SMTs): - `io.debezium.transforms.ByLogicalTableRouter` - `io.debezium.transforms.ExtractNewRecordState`, use `io.debezium.connector.v2.mongodb.transforms.ExtractNewDocumentState` instead. - `io.debezium.transforms.outbox.EventRouter`, use `io.debezium.connector.v2.mongodb.transforms.outbox.MongoEventRouter` instead. - `io.confluent.connect.smt.envelop.RestoreEnvelopedRecord` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.GzipDecompress$Key/$Value` - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. #### MQTT Sink Connector There are no current limitations for the [MQTT Sink Connector for Confluent Cloud](cc-mqtt-sink.md#cc-mqtt-sink). #### MQTT Source Connector - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.ValueToKey` - `org.apache.kafka.connect.transforms.HoistField$Value` #### MySQL Sink Connector The [MySQL Sink (JDBC) Connector for Confluent Cloud](cc-mysql-sink.md#cc-mysql-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The database and Kafka cluster should be in the same region. - For tombstone records, set `delete.enabled` to `true`. - When `input.data.format` is set to `STRING`, the entire Kafka record value is written into a single column specified by `string.output.value.column.name` (default `record_value`). The connector does not support splitting a `STRING` value across multiple columns. - When `input.data.format` is set to `STRING`, the value of `string.output.value.column.name` must not match any column listed in `pk.fields` for the target table. - When `input.data.format` is set to `STRING`, `pk.mode=record_value` is not supported. Only `pk.mode=none`, `pk.mode=kafka`, and `pk.mode=record_key` are valid with `STRING` values. - When `input.data.format` is set to `STRING` and `auto.create` is set to `true`, the column specified by `string.output.value.column.name` is created using the default string data type of the target database. - When `input.data.format` is set to `JSON`, schemaless JSON with (`value.converter.schemas.enable=false`) is not supported. Records must include an inline `{schema, payload}` envelope with (`value.converter.schemas.enable=true`), because the connector requires a value schema to map fields onto database columns. #### MySQL CDC Source Connector (Debezium) [Legacy] The [MySQL CDC Source (Debezium) Connector [End of Life] for Confluent Cloud](cc-mysql-source-cdc-debezium.md#cc-mysql-source-cdc-debezium) has the following limitations: - MariaDB is not currently supported. See [the Debezium docs](https://debezium.io/documentation/reference/1.6/connectors/mysql.html#how-the-mysql-connector-works) for more information. - Amazon Aurora doesn’t support binary logging using a multi-master cluster as the binlog master or worker. You can’t use binlog-based CDC tools with [multi-master clusters](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-multi-master.html). - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - The `additional-condition` option for the [signaling feature](https://debezium.io/documentation/reference/1.9/configuration/signalling.html#sending-signals-to-a-debezium-connector) of incremental snapshots is not supported in v1. If you are using a legacy version of the connector and need this option, [upgrade](cc-mysql-source-cdc-v2-debezium/cc-mysql-source-cdc-v2-debezium.md#cc-mysql-cdc-source-v2-debezium-migrating) to the latest version of the connector. #### MySQL CDC Source V2 (Debezium) Connector The [MySQL CDC Source Connector V2 (Debezium) for Confluent Cloud](cc-mysql-source-cdc-v2-debezium/index.md#cc-mysql-source-cdc-v2-debezium) has the following limitations: - MariaDB is now supported. For more information on connecting to MariaDB, see [MariaDB CDC Source connector](cc-mariadb-cdc-source-debezium.md#cc-mariadb-source-cdc-debezium-configure-connector) documentation. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like AVRO. - Organizations can run multiple connectors with a limit of one task per connector (`"tasks.max": "1"`). - The connector supports only the millisecond-precision timestamp field `ts_ms` in the change event envelope, which indicates the time at which the connector processed the event. Higher-precision fields such as `ts_us` and `ts_ns` are not supported. - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. #### MySQL Source (JDBC) Connector The [MySQL Source (JDBC) Connector for Confluent Cloud](cc-mysql-source.md#cc-mysql-source) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A timestamp column must not be nullable for the *timestamp* or *timestamp+incrementing* mode. - A query produced by the connector may take a very long time to execute when using *timestamp+incrementing*, as opposed to the query speed when using *incrementing*. - If the connector is making numerous parallel insert operations in a large source table, insert transactions can commit out of order (this is typical). What this means is that a “greater” auto_increment ID (for example, 101) is committed earlier and a “smaller” ID (for example, 100) is committed later. The time difference here may only be a few milliseconds, but the commits are out of order nevertheless. Note that using *incrementing* mode to load data from such tables always results in some data loss. This happens because when the source connector worker reads (polls) the table, the connector gets the row with a greater offset value (with the smaller offset row remaining uncommitted). In the next iteration, although the uncommitted row is committed, the offset position has moved beyond that value, so the row is skipped. Using *timestamp+incrementing* mode is not a good choice either, because the tables may be very large (five to eight million rows added daily) and there is a high cost for any indexing approach, with the exception of PK indexing. - The connector doesn’t allow the use of special characters in table names (for example, `$`), which are not allowed in Kafka topic names. To work around this limitation, change the name of the target topic using the [ExtractTopic Single Message Transformation (SMT)](/platform/current/connect/transforms/extracttopic.html). Note that this SMT helps you extract the topic name from the key or value of the message; therefore, you must include the desired topic name in the payload. - The connector cannot capture tables with the same name across different schemas, even if you configure the schema pattern with the correct schema name or use fully qualified names (FQNs) for tables. As a workaround, you should create the connector with the required table names and restrict user access at the schema level. - If the records contain zero date values, for example, `0000-00-00` when using Date fields, you can convert them to NULL by using the driver property `tableName?zeroDateTimeBehavior=CONVERT_TO_NULL` in the connection string. - When `output.data.format` is set to `STRING`, row values are emitted as a Kafka Connect `Struct` string representation (`Struct{COL1=value1,COL2=value2,...}`) without an accompanying schema. Downstream consumers must parse this representation manually. - When `output.data.format` is set to `JSON`, set `value.converter.schemas.enable` to `true` so that each emitted record includes an inline `{schema, payload}` envelope. The embedded schema preserves the type of every field and allows downstream systems to consume the records without having to infer the schema from the payload. - The connector can skip records in `timestamp+incrementing` mode when multiple transactions write to the source database table, and some transactions commit later. If a transaction with a higher timestamp and incrementing ID commits first and is read by the connector, the connector will ignore any later transactions with older timestamps and incrementing IDs. As a workaround, set the `timestamp.delay.interval.ms` in the connector to match the transaction timeout on the database side. This ensures the connector considers records that are committed later, as long as they are committed within the `timestamp.delay.interval.ms` timeframe. #### Neo4j Sink Connector The [Neo4j Sink Connector for Confluent Cloud](cc-neo4j-sink/cc-neo4j-sink.md#cc-neo4j-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The connector supports private networking only for self-managed Neo4j instances on Microsoft Azure (Azure). - The connector supports private networking only for self-managed Neo4j instances on Google Cloud. #### New Relic Metrics Sink Connector The [New Relic Metrics Sink Connector for Confluent Cloud](cc-new-relic-metrics-sink.md#cc-new-relic-metrics-sink) has the following limitation: - The connector is limited by New Relic [Matric API limits and restricted attributes](https://docs.newrelic.com/docs/data-apis/ingest-apis/metric-api/metric-api-limits-restricted-attributes/). #### Opensearch Sink Connector The [OpenSearch Sink Connector for Confluent Cloud](cc-opensearch-sink.md#cc-opensearch-sink) has the following limitations: - The connector only allows you to create and manage up to 5 indexes. - The connector only supports HTTP POST requests. #### Oracle CDC Source Connector The [Oracle CDC Source Connector for Confluent Cloud](cc-oracle-cdc-source/cc-oracle-cdc-source.md#cc-oracle-cdc-source-configure-connector) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - If you change the connector configuration property `oracle.date.mapping` from `date` to `timestamp` the connector will not work, since this results in a breaking schema change. You must create a new connector if you want to change to the `timestamp` option. - The connector’s DDL parser does not support the following DDL statements: - ALTER TABLE statements to add or remove constraints. - ALTER TABLE statements dropping multiple columns in a single statement. - ALTER TABLE adding columns of TIMESTAMP type with DEFAULT. - ALTER TABLE with columns that contain user-defined types. - ALTER TABLE to rename tables or columns. - ALTER TABLE statements to remove all data from a specified partition of a partitioned table. - ALTER TABLE statements to modify LOB columns. - ALTER TABLE statements with ADD/DROP PARTITION. - ALTER TABLE statements to modify columns with NULL or NOT NULL that don’t contain the column type in the statement. Note that as a workaround, you can use the following statement: ```sql ALTER TABLE MODIFY NULL; ``` - GRANT statements to grant SELECT, UPDATE, INSERT, DELETE, and other privileges on tables. - DROP TABLE with CASCADE CONSTRAINTS. #### Oracle Database Sink Connector The [Oracle Database Sink (JDBC) Connector for Confluent Cloud](cc-oracle-db-sink.md#cc-oracle-db-sink) has the following limitations: * The Oracle Database version must be 11.2.0.4 or later. * Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). * The Oracle database and Kafka cluster should be in the same region. * When `input.data.format` is set to `STRING`, the entire Kafka record value is written into a single column specified by `string.output.value.column.name` (default `record_value`). The connector does not support splitting a `STRING` value across multiple columns. * When `input.data.format` is set to `STRING`, the value of `string.output.value.column.name` must not match any column listed in `pk.fields` for the target table. * When `input.data.format` is set to `STRING`, `pk.mode=record_value` is not supported. Only `pk.mode=none`, `pk.mode=kafka`, and `pk.mode=record_key` are valid with `STRING` values. * When `input.data.format` is set to `STRING` and `auto.create` is set to `true`, the column specified by `string.output.value.column.name` is created using the default string data type of the target database. * When `input.data.format` is set to `JSON`, schemaless JSON with (`value.converter.schemas.enable=false`) is not supported. Records must include an inline `{schema, payload}` envelope with (`value.converter.schemas.enable=true`), because the connector requires a value schema to map fields onto database columns. * See [Database considerations](cc-oracle-db-sink.md#cc-oracle-db-sink-db-troubleshooting) for additional information. #### Oracle Database Source (JDBC) Connector The [Oracle Database Source (JDBC) Connector for Confluent Cloud](cc-oracle-db-source.md#cc-oracle-db-source) has the following limitations: - The Oracle Database version must be 11.2.0.4 or later. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - A timestamp column must not be nullable for the *timestamp* or *timestamp+incrementing* mode. - Configuration properties that are not shown in the Cloud Console use the default values. See [JDBC Connector Source Connector Configuration Properties](../../../kafka-connect-jdbc/current/source-connector/source_config_options.html) for property definitions and default values. - When `output.data.format` is set to `STRING`, row values are emitted as a Kafka Connect `Struct` string representation (`Struct{COL1=value1,COL2=value2,...}`) without an accompanying schema. Downstream consumers must parse this representation manually. - When `output.data.format` is set to `JSON`, set `value.converter.schemas.enable` to `true` so that each emitted record includes an inline `{schema, payload}` envelope. The embedded schema preserves the type of every field and allows downstream systems to consume the records without having to infer the schema from the payload. - The connector doesn’t allow the use of special characters in table names (for example, `$`), which are not allowed in Kafka topic names. To work around this limitation, change the name of the target topic using the [ExtractTopic Single Message Transformation (SMT)](/platform/current/connect/transforms/extracttopic.html). Note that this SMT helps you extract the topic name from the key or value of the message; therefore, you must include the desired topic name in the payload. - The connector does not accept zero date values, for example, `0000-00-00` when using Date fields. You can convert them to NULL by using the driver property `tableName?zeroDateTimeBehavior=convertToNull` in the connection string. - The connector can skip records in `timestamp+incrementing` mode when multiple transactions write to the source database table, and some transactions commit later. If a transaction with a higher timestamp and incrementing ID commits first and is read by the connector, the connector will ignore any later transactions with older timestamps and incrementing IDs. As a workaround, set the `timestamp.delay.interval.ms` in the connector to match the transaction timeout on the database side. This ensures the connector considers records that are committed later, as long as they are committed within the `timestamp.delay.interval.ms` timeframe. #### NOTE - When you create views in Oracle Database, names are stored in uppercase by default unless enclosed in double quotes. Configuring the connector with lowercase view names can lead to `table not found` exception. To preserve the exact casing and avoid issues, define views using double quotes. For more information, see [Oracle Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/Database-Object-Names-and-Qualifiers.html#GUID-75337742-67FD-4EC0-985F-741C93D918DA/). #### PostgreSQL Sink Connector The [PostgreSQL Sink (JDBC) Connector for Confluent Cloud](cc-postgresql-sink.md#cc-postgresql-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The database and Kafka cluster should be in the same region. If you use a different region, be aware that you may incur additional data transfer charges. - For tombstone records, set `delete.enabled` to `true` - CockroachDB is not supported. To learn about other unsupported PostgreSQL databases, contact Confluent [Support](https://support.confluent.io/). - When `input.data.format` is set to `STRING`, the entire Kafka record value is written into a single column specified by `string.output.value.column.name` (default `record_value`). The connector does not support splitting a `STRING` value across multiple columns. - When `input.data.format` is set to `STRING`, the value of `string.output.value.column.name` must not match any column listed in `pk.fields` for the target table. - When `input.data.format` is set to `STRING`, `pk.mode=record_value` is not supported. Only `pk.mode=none`, `pk.mode=kafka`, and `pk.mode=record_key` are valid with `STRING` values. - When `input.data.format` is set to `STRING` and `auto.create` is set to `true`, the column specified by `string.output.value.column.name` is created using the default string data type of the target database. - When `input.data.format` is set to `JSON`, schemaless JSON with (`value.converter.schemas.enable=false`) is not supported. Records must include an inline `{schema, payload}` envelope with (`value.converter.schemas.enable=true`), because the connector requires a value schema to map fields onto database columns. #### Pagerduty Sink Connector There are no current limitations for the [PagerDuty Sink Connector [End of Life] for Confluent Cloud](cc-pagerduty-sink.md#cc-pagerduty-sink). #### PostgreSQL CDC Source (Debezium) [Legacy] Connector The [PostgreSQL CDC Source Connector (Debezium) [End of Life] for Confluent Cloud](cc-postgresql-cdc-source-debezium.md#cc-postgresql-cdc-source-debezium) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - For Azure, you must use a general purpose or memory-optimized PostgreSQL database. You cannot use a basic database. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - CockroachDB is not supported. To learn about other unsupported PostgreSQL databases, contact Confluent [Support](https://support.confluent.io/). - Clients from Azure Virtual Networks are not allowed to access the server by default. Make sure your Azure Virtual Network is correctly configured and that **Allow access to Azure Services** is enabled. - The following are the default partition and replication factor properties: - `topic.creation.default.partitions=1` - `topic.creation.default.replication.factor=3` - See the [After-state only output limitation](cc-postgresql-cdc-source-debezium.md#cc-postgresql-source-cdc-debezium-after-state-only) if you are planning to use the optional property `After-state only`. - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - The `pgoutput` logical decoding output plug-in does not capture values for generated columns, resulting in missing data for these columns in the connector’s output. - RDS Proxy does not support streaming replication mode. If you configure the connector to connect to an RDS Proxy, it cannot create the replication slot, leading to failure. However, if the replication slot exists before starting the connector with RDS Proxy, the connector will successfully complete the snapshotting phase but will fail during the streaming phase. #### PostgreSQL CDC Source V2 (Debezium) Connector The [PostgreSQL CDC Source Connector V2 (Debezium) for Confluent Cloud](cc-postgresql-cdc-source-v2-debezium/index.md#cc-postgresql-cdc-source-v2-debezium) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - For Azure, you must use a general purpose or memory-optimized PostgreSQL database. You cannot use a basic database. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - CockroachDB and NeonDB are not supported. To learn about other unsupported PostgreSQL databases, contact Confluent [Support](https://support.confluent.io/). - Clients from Azure Virtual Networks are not allowed to access the server by default. Make sure your Azure Virtual Network is correctly configured and that **Allow access to Azure Services** is enabled. - The following are the default partition and replication factor properties: - `topic.creation.default.partitions=1` - `topic.creation.default.replication.factor=3` - See the [After-state only output limitation](cc-postgresql-cdc-source-v2-debezium/cc-postgresql-cdc-source-v2-debezium.md#cc-postgresql-source-cdc-v2-debezium-after-state-only) if you are planning to use the optional property `after.state.only`. - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - The `pgoutput` logical decoding output plug-in does not capture values for generated columns, resulting in missing data for these columns in the connector’s output. - Incremental snapshots may fail when tables with generated columns are included in the capture list. The connector uses `SELECT *` queries that return generated columns, but these columns are excluded from Debezium’s internal schema representation. This mismatch causes an `IllegalArgumentException`. As a workaround, add the generated column to the `column.exclude.list` configuration property. For more information, see [Incremental snapshots with generated columns limitation](cc-postgresql-cdc-source-v2-debezium/cc-postgresql-cdc-source-v2-debezium.md#cc-postgresql-source-cdc-v2-debezium-generated-columns-limitation). - RDS Proxy does not support streaming replication mode. If you configure the connector to connect to an RDS Proxy, it cannot create the replication slot, leading to failure. However, if the replication slot exists before starting the connector with RDS Proxy, the connector will successfully complete the snapshotting phase but will fail during the streaming phase. - The connector supports only the millisecond-precision timestamp field `ts_ms` in the change event envelope, which indicates the time at which the connector processed the event. Higher-precision fields such as `ts_us` and `ts_ns` are not supported. - Multidimensional (2D or higher) PostgreSQL array columns aren’t supported. When the connector processes a multidimensional array column, it produces `null` values for each element in the outer array instead of the actual data. For example, a `bigint[][]` column with value `{{1001,1002},{1003,1004}}` is published as `{"array": [null, null]}`. As a workaround, exclude such columns using the `column.exclude.list` configuration property. - The connector supports reading only from single topic partition of the signaling topic to ensure signal ordering. Configure the topic with only one partition. Signaling does not work if messages are sent to any partition other than `partition-0`. #### PostgreSQL Source (JDBC) Connector The [PostgreSQL Source (JDBC) Connector for Confluent Cloud](cc-postgresql-source.md#cc-postgresql-source) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Clients from Azure Virtual Networks are not allowed to access the server by default. Make sure your Azure Virtual Network is correctly configured and enable “Allow access to Azure Services”. - CockroachDB is not supported. To learn about other unsupported PostgreSQL databases, contact Confluent [Support](https://support.confluent.io/). - A timestamp column must not be nullable for the `timestamp` or `timestamp+incrementing` mode. - If the connector is making numerous parallel insert operations in a large source table, insert transactions can commit out of order (this is typical). What this means is that a “greater” auto_increment ID (for example, 101) is committed earlier and a “smaller” ID (for example, 100) is committed later. The time difference here may only be a few milliseconds, but the commits are out of order nevertheless. Note that using *incrementing* mode to load data from such tables always results in some data loss. This happens because when the source connector worker reads (polls) the table, the connector gets the row with a greater offset value (with the smaller offset row remaining uncommitted). In the next iteration, although the uncommitted row is committed, the offset position has moved beyond that value, so the row is skipped. Using *timestamp+incrementing* mode is not a good choice either, because the tables may be very large (five to eight million rows added daily) and there is a high cost for any indexing approach, with the exception of PK indexing. - The connector does not support all data types. When it encounters an unknown data type, the data type is dropped from the source output. The following lists known unsupported data types. ### Known unsupported data types * \_abc * box * cardinal_number * character_data * cidr * circle * citext * hstore * inet * line * lseg * macaddr * macaddr8 * money * path * pg_lsn * pg_snapshot * point * polygon * refcursor * sql_identifier * tsquery * tsvector * txid_snapshot * uuid * yes_or_no - When `output.data.format` is set to `STRING`, row values are emitted as a Kafka Connect `Struct` string representation (`Struct{COL1=value1,COL2=value2,...}`) without an accompanying schema. Downstream consumers must parse this representation manually. - When `output.data.format` is set to `JSON`, set `value.converter.schemas.enable` to `true` so that each emitted record includes an inline `{schema, payload}` envelope. The embedded schema preserves the type of every field and allows downstream systems to consume the records without having to infer the schema from the payload. - The connector doesn’t allow the use of special characters in table names (for example, `$`), which are not allowed in Kafka topic names. To work around this limitation, change the name of the target topic using the [ExtractTopic Single Message Transformation (SMT)](/platform/current/connect/transforms/extracttopic.html). Note that this SMT helps you extract the topic name from the key or value of the message; therefore, you must include the desired topic name in the payload. - The connector does not accept zero date values, for example, `0000-00-00` when using Date fields. You can convert them to NULL by using the driver property `tableName?zeroDateTimeBehavior=convertToNull` in the connection string. - The connector can skip records in `timestamp+incrementing` mode when multiple transactions write to the source database table, and some transactions commit later. If a transaction with a higher timestamp and incrementing ID commits first and is read by the connector, the connector will ignore any later transactions with older timestamps and incrementing IDs. As a workaround, set the `timestamp.delay.interval.ms` in the connector to match the transaction timeout on the database side. This ensures the connector considers records that are committed later, as long as they are committed within the `timestamp.delay.interval.ms` timeframe. #### RabbitMQ Sink Connector The [RabbitMQ Sink Connector for Confluent Cloud](cc-rabbitmq-sink.md#cc-rabbitmq-sink) has the following limitation: - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Redis Sink Connector The [Redis Sink Connector [Deprecated] for Confluent Cloud](cc-redis-sink.md#cc-redis-sink) has the following limitations: - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Redis instance and Kafka cluster should be in the same region. - This connector does not support the ValueToKey (`org.apache.kafka.connect.transforms.ValueToKey`) Single Message Transform. #### Redis Kafka Sink Connector The [Redis Kafka Sink Connector for Confluent Cloud](cc-redis-kafka-db-sink.md#cc-redis-kafka-sink) has the following limitation: - The connector does not support private networking. #### Redis Kafka Source Connector The [Redis Kafka Source connector](cc-redis-kafka-source.md#cc-redis-kafka-source) has the following limitations: - The Keys Source connector can be configured with only one task. - The connector does not support private networking. #### Salesforce Bulk API Source Connector The [Salesforce Bulk API Source Connector for Confluent Cloud](cc-salesforce-bulk-api-source.md#cc-salesforce-bulk-api-source) has the following limitations: - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - Restarting: When the connector operates, it periodically records the last query time in the Connect offset topic. When the connector is restarted, it may fetch Salesforce objects with a `LastModifiedDate` that is later than last queried time. - API limits: The Salesforce Bulk API Source connector is limited by non-compound fields. For example, Bulk Query doesn’t support address, location fields. The connector discards address and geolocation fields. - The following Salesforce object (SObject) error message may be displayed when you are using the Salesforce Bulk API Source connector: ```text Entity 'Order' is not supported to use PKChunking. ``` For these SObjects, set the configuration property **Enable Batching** to false (CLI property `batch.enable=false`). - Unsupported SObjects: See [Supported and Unsupported SObjects](cc-salesforce-bulk-api-source.md#cc-salesforce-bulk-api-source-supported-objects) for a list of supported and unsupported SObjects. #### Salesforce Bulk API 2.0 Sink Connector The [Salesforce Bulk API 2.0 Sink Connector for Confluent Cloud](cc-salesforce-bulk-api-v2-sink.md#cc-salesforce-bulk-api-v2-sink) has the following limitations: - Salesforce imposes [API limits](https://developer.salesforce.com/docs/atlas.en-us.salesforce_app_limits_cheatsheet.meta/salesforce_app_limits_cheatsheet/salesforce_app_limits_platform_api.htm) over a 24-hour window. Exceeding Salesforce API limits will result in connector failure. - The connector is subject to daily limits on the number of records handled, the number of batches created (internal to Salesforce), and the total size of data. For detailed limitations, see [Bulk API Limits](https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/bulk_common_limits.htm). - There are Salesforce [data and file storage](https://help.salesforce.com/s/articleView?id=sf.overview_storage.htm&type=5) limitations based on the type of organization used. - The connector doesn’t currently support the [polymorphic fields](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_relationships_and_polymorph_keys.htm). - The connector currently supports only up to 5 Objects in a single instance. #### Salesforce Bulk API 2.0 Source Connector The [Salesforce Bulk API 2.0 Source Connector for Confluent Cloud](cc-salesforce-bulk-api-v2-source.md#cc-salesforce-bulk-api-v2-source) has the following limitations: - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - Restarting: When the connector operates, it periodically records the last query time in the Connect offset topic. When the connector is restarted, it may fetch Salesforce objects with a `LastModifiedDate` that is later than last queried time. - API limits: The Salesforce Bulk API Source connector is limited by non-compound fields. For example, Bulk Query doesn’t support address, location fields. The connector discards address and geolocation fields. - The connector currently supports only up to five SObjects in a single instance. #### Salesforce CDC Source Connector The [Salesforce CDC Source Connector for Confluent Cloud](cc-salesforce-source-cdc.md#cc-salesforce-source-cdc) has the following limitations: - The Salesforce user account configured for the connector must have permission to **View All Data**. For details, see [Required Permissions for Change Events Received by CometD Subscribers](https://developer.salesforce.com/docs/atlas.en-us.change_data_capture.meta/change_data_capture/cdc_security_perms.htm). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector does not support real time updates for platform event schema changes. To synchronize event schema changes with the connector, restart the connector from Confluent Cloud Console or use the [restart connector API](https://docs.confluent.io/cloud/current/ccloud/restart-connectv-1-connector/). - When you pause a connector, the connector continues to fetch records from the Salesforce endpoint. These records are not sent to the Kafka topic until the connector resumes. - The connector does not support parsing fields where Salesforce sends the data differences of fields in updated records. For details, see [Sending Data Differences for Fields of Updated Records](https://developer.salesforce.com/docs/atlas.en-us.change_data_capture.meta/change_data_capture/cdc_data_diff.htm). #### Salesforce Platform Event Sink Connector The [Salesforce Platform Event Sink Connector for Confluent Cloud](cc-salesforce-platform-event-sink.md#cc-salesforce-platform-event-sink) has the following limitations: - There are Salesforce [streaming allocations and limits](https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/limits.htm) that apply to this connector. For example, the number of API calls that can occur within a 24-hour period is capped for free developer org accounts. - There are [data and file storage limits](https://help.salesforce.com/articleView?id=overview_storage.htm&type=5) that are based on the type of organization you use. - The connector currently supports only up to 5 events in a single instance. #### Salesforce Platform Event Source Connector The [Salesforce Platform Event Source Connector for Confluent Cloud](cc-salesforce-platform-event-source.md#cc-salesforce-platform-event-source) has the following limitations: - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - The connector supports only up to 5 platform events. For standard limits on for platform events allocation, see [Salesforce Common Platform Event Allocations](https://developer.salesforce.com/docs/atlas.en-us.platform_events.meta/platform_events/platform_event_limits.htm#platform_events_limits_common) - Custom channels aren’t supported for standard platform events or legacy standard volume custom platform events. Thus multiple events are not supported. - The connector does not support real time updates for platform event schema changes. To synchronize event schema changes with the connector, restart the connector from Confluent Cloud Console or use the [restart connector API](https://docs.confluent.io/cloud/current/ccloud/restart-connectv-1-connector/). - When you pause a connector, the connector continues to fetch records from the Salesforce endpoint. These records are not sent to the Kafka topic until the connector resumes. - The connector does not support parsing fields where Salesforce sends the data differences of fields in updated records. For details, see [Sending Data Differences for Fields of Updated Records](https://developer.salesforce.com/docs/atlas.en-us.change_data_capture.meta/change_data_capture/cdc_data_diff.htm). #### Salesforce PushTopic Source Connector The [Salesforce PushTopic Source Connector for Confluent Cloud](cc-salesforce-pushtopic-source.md#cc-salesforce-pushtopic-source) has the following limitations: - Organizations can run multiple connectors with a limit of one task per connector (that is, `"tasks.max": "1"`). - Note the following limitations for at least once delivery: * When the connector operates, it periodically records the replay ID of the last record written to Kafka. When the connector is stopped and then restarted within 24 hours, the connector continues consuming the PushTopic where it stopped, with no missed events. However, if the connector stops for more than 24 hours, some events are discarded in Salesforce before the connector can read them. * If the connector stops unexpectedly due to a failure, it may not record the replay ID of the last record successfully written to Kafka. When the connector restarts, it resumes from the last recorded replay ID. This means that some events may be duplicated in Kafka. - When you pause a connector, the connector continues to fetch records from the Salesforce endpoint. These records are not sent to the Kafka topic until the connector resumes. #### Salesforce SObject Sink Connector The [Salesforce SObject Sink Connector for Confluent Cloud](cc-salesforce-SObjects-sink.md#cc-salesforce-sobjects-sink) has the following limitations: - There are Salesforce [streaming allocations and limits](https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/limits.htm) that apply to this connector. For example, the number of API calls that can occur within a 24-hour period is capped for free developer org accounts. - There are [data and file storage limits](https://help.salesforce.com/articleView?id=overview_storage.htm&type=5) that are based on the type of organization you use. - The connector doesn’t currently support the [polymorphic fields](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_relationships_and_polymorph_keys.htm). - The connector currently supports only up to 5 SObjects in a single instance. - The connector supports batching with the following limitations: - Batching requires API version 42.0 and later. - Upsert operation with batching requires API version 46.0 and later. - The connector does not support deletion using external ID in batches; hence, it falls back to individual processing. - The connector supports a maximum of 200 records per batch. #### Salesforce Source V2 connector The [Salesforce Source V2 Connector for Confluent Cloud](cc-salesforce-source-v2.md#cc-salesforce-source-v2) has the following limitations: - The connector does not support custom SOQL queries. - The connector does not support Big Objects with the `__b` suffix or External Objects with the `__x` suffix. - If `tasks.max` exceeds the number of configured SObjects, the connector caps the active task count at the SObject count. Unused task slots remain idle. The connector logs them at `INFO` at startup. - The connector does not populate compound fields such as Address and Location. Only their component fields carry data. - The connector does not support per-SObject customization. All configured SObjects share the same connector settings. #### ServiceNow Sink Connector The [ServiceNow Sink Connector for Confluent Cloud](cc-servicenow-sink.md#cc-servicenow-sink) has the following limitations: - The connector does not support reporter topics with CSFLE. - The connector does not transform nested JSON structures before sending data to ServiceNow. The ServiceNow Table API does not handle nested structures as expected, regardless of the target field type. For workarounds, see [Frequently asked questions](cc-servicenow-sink.md#cc-servicenow-sink-faq). #### ServiceNow Source Connector The [ServiceNow Source [Legacy] Connector [Deprecated] for Confluent Cloud](cc-servicenow-source.md#cc-servicenow-source) has the following limitation: - The connector does not support the following table types: * Sys Audit (sys_audit) * Audit Relationship Change (sys_audit_relation) #### ServiceNow Source V2 Connector The [ServiceNow Source V2 Connector for Confluent Cloud](cc-servicenow-source-v2.md#cc-servicenow-source-v2) has the following limitations: - The connector does not support schemaless formats. - The connector does not support evolving schema. Schema evolution should be done manually. - In contrast to the offset-based pagination used in the ServiceNow Source connector, the V2 connector uses timestamp-based pagination. The minimum polling interval (`table{i}.request.interval.ms`) is 2 seconds to prevent data loss due to any potential time drift between the ServiceNow instance and the connector instance. #### SFTP Sink Connector The [SFTP Sink Connector for Confluent Cloud](cc-sftp-sink.md#cc-sftp-sink) has the following limitations: - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. - `schema.compatibility` is set to `NONE`. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### SFTP Source Connector The [SFTP Source Connector for Confluent Cloud](cc-sftp-source.md#cc-sftp-source) has the following limitations: - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.HoistField$Value` - Currently, the SFTP source connector reads and moves a file only once while processing files from a specific SFTP directory. If a file with the same name is produced later in the same SFTP directory, the connector will not process it. - If the following error occurs, you must provide `WRITE` access to the SFTP server directory where the connector is accessing files. ```bash There were some errors with your configuration:\ninput.path: Could not write to the specified location configured in %s config. Check that that the user has write permissions for the specified location ``` - When the `schema.generation.enabled` configuration is enabled, the connector offers limited schema detection capabilities and does not support nested or array schemas. #### Snowflake Sink Connector The [Snowflake Sink Connector for Confluent Cloud](cc-snowflake-sink/cc-snowflake-sink.md#cc-snowflake-sink) has the following limitations: - The connector doesn’t support Snowflake’s [Client Redirect](https://docs.snowflake.com/en/user-guide/client-redirect#introduction-to-client-redirect) feature. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). - The connector does not remove Snowflake pipes when a connector is deleted. For instructions to manually clean up Snowflake pipes, see [Dropping Pipes](https://docs.snowflake.com/en/user-guide/kafka-connector-manage.html#dropping-pipes). - Note that Snowpipe Streaming for Kafka supports [insert-only operations](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-streaming-overview#insert-only-operations). For additional information, see the [Snowflake documentation](https://docs.snowflake.com/en/user-guide/data-load-snowpipe-streaming-overview). - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - Each task is limited to a number of topic partitions based on the `buffer.size.bytes` property value. For example, a `10` MB buffer size is limited to 50 topic partitions, a `20` MB buffer is limited to 25 topic partitions, `50` MB buffer is limited to 10 topic partitions, and a `100` MB buffer to 5 topic partitions. - In `SNOWPIPE_STREAMING` mode, each connector requires a unique channel name. For example, the combination `[channelA = ...]` must be unique across all created connectors. #### NOTE This restriction applies only when connectors are created in `SNOWPIPE_STREAMING` mode. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` - **Null-value record processing behavior:** When the connector is configured with `behavior.on.null.values=IGNORE`, it correctly drops records that contain null values without writing them to Snowflake. Because the committed Kafka offset only advances when a record is successfully written to Snowflake, the committed offset can freeze at the last written record. As the log-end offset continues to climb, monitoring metrics in the Confluent Cloud Console can report unbounded and growing lag (sometimes called “ghost lag”), even though the connector is healthy and actively processing records. #### Snowflake Source Connector The [Snowflake Source Connector for Confluent Cloud](cc-snowflake-source/cc-snowflake-source.md#cc-snowflake-source) has the following limitations: - The connector does not support Snowflake’s Structured and Geospatial data types. - The connector does not support Kafka record keys. - The connector does not support timestamp columns of types other than `TIMESTAMP_NTZ`. - Depending on the service environment, certain network access limitations may exist. Make sure the connector can reach your service. For details, see [Networking and DNS](overview.md#connect-internet-access-resources). #### Solace Sink Connector The [Solace Sink Connector for Confluent Cloud](cc-solace-sink.md#cc-solace-sink) has the following limitations: - The connector can create queues, but not durable topic endpoints. - A valid schema must be available in [Schema Registry](../get-started/schema-registry.md#cloud-sr-config) to use a Schema Registry-based format, like Avro. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### Solace Source Connector The [Solace Source Connector for Confluent Cloud](cc-solace-source.md#cc-solace-source) has the following limitations: - The maximum batch size is `2048` records per poll. The maximum poll duration is two minutes (`120,000 ms`). - When the connector reads from an exclusive queue, increasing the task count beyond `1` does not improve throughput. Additional tasks bind as standby consumers and become active only if the primary consumer disconnects. This limitation applies to consumer parallelism only; exclusive queues continue to provide guaranteed delivery. - The connector cannot connect to a Solace broker through an HTTP or SOCKS proxy. This applies to both HTTP CONNECT tunnels and SOCKS4 or SOCKS5 proxies. If all outbound TCP connections from Confluent Cloud must transit a corporate proxy, the connector cannot reach the Solace broker directly. - The connector supports egress Private Link or Private Service Connect connectivity only with self-managed Solace PubSub+ brokers running in customer-owned AWS, Azure, or Google Cloud infrastructure. The connector does not support private networking from Confluent Cloud to Solace cloud hosted brokers. - Solace enforces a per-message cap of 30 MB for guaranteed (persistent) delivery and 64 MiB for direct (non-persistent) delivery. On Solace Standard with the 100-connection scaling tier, the guaranteed cap drops to `10` MB (See [Solace System Scaling Parameters](https://docs.solace.com/Software-Broker/System-Scaling-Parameters.htm) and [Solace Adding Data Payloads](https://docs.solace.com/API/API-Developer-Guide/Adding-Data-Payloads.htm)). These caps can exceed Confluent Cloud Kafka’s `max.message.bytes` (8 MB on Basic, Standard, Enterprise, and Freight clusters, 20 MB on Dedicated clusters). If a Solace message exceeds the destination topic’s limit, the Kafka producer throws `RecordTooLargeException` and the task transitions to `FAILED`. #### Splunk Sink Connector The [Splunk Sink Connector for Confluent Cloud](cc-splunk-sink.md#cc-splunk-sink) has the following limitation: - If an invalid index is specified, the connector posts the event to Splunk successfully, but it appears to be discarded in Splunk. #### Zendesk Source Connector The [Zendesk Source Connector for Confluent Cloud](cc-zendesk-source.md#cc-zendesk-source) has the following limitations: - There is a limit of one task per connector instance. - For Schema Registry-based output formats, the connector attempts to deduce the schema based on the source API response returned. The connector registers a new schema for every NULL and NOT NULL value of an optional field in the API response. For this reason, the connector may register schema versions at a much higher rate than expected. ### Preview connector limitations See the following limitations for preview connectors. #### Google Cloud Dataproc Sink Connector The [Google Cloud Dataproc Sink Connector [End of Life] for Confluent Cloud](cc-gcp-dataproc-sink.md#cc-gcp-dataproc-sink) has the following limitations: - The Confluent Cloud cluster and the target Dataproc cluster must be in a VPC peering configuration. #### NOTE For a non-VPC peered environment, public inbound traffic access (`0.0.0.0/0`) must be allowed to the VPC where the Dataproc cluster is located. You must also make configuration changes to allow public access to the Dataproc cluster while retaining the private IP addresses for the Dataproc master and worker nodes (HDFS NameNode and DataNodes). For configuration details, see [Configuring a non-VPC peering environment](cc-gcp-dataproc-sink.md#cc-gcp-dataproc-public-ip-private-ip-config). For more information about public Internet access to resources, see [Networking and DNS](overview.md#connect-internet-access-resources). - The Dataproc image version must be 1.4 (or later). See [Cloud Dataproc Image version list](https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-versions). - One task can handle up to 100 partitions. - Input format JSON to output format AVRO does not work for the preview connector. - Partitioning (hourly or daily) is based on Kafka record time. - `flush.size` defaults to 1000. The value can be increased if needed. The value can be lowered (1 minimum) if you are running a [Dedicated Confluent Cloud cluster](../clusters/cluster-types.md#dedicated-cluster). The minimum value is 1000 for non-dedicated clusters. The following scenarios describe a couple of ways records may be flushed to storage: * You use the default setting of 1000 and your topic has six partitions. Files start to be created in storage after more than 1000 records exist in each partition. * You use the default setting of 1000 and the partitioner is set to Hourly. 500 records arrive at one partition from 2:00pm to 3:00pm. At 3:00pm, an additional 5 records arrive at the partition. You will see 500 records in storage at 3:00pm. #### NOTE The properties `rotate.schedule.interval.ms` and `rotate.interval.ms` can be used with `flush.size` to determine when files are created in storage. These parameters kick in and files are stored based on which condition is met first. For FieldPartitioner in fully managed connectors, an additional automatic rotation occurs when the number of open files reaches 50. This uploads the current batch of files to storage and creates a new batch. For example: You have one topic partition. You set `flush.size=1000` and `rotate.schedule.interval.ms=600000` (10 minutes). 500 records arrive at the topic partition from 12:01 to 12:10. 500 additional records arrive from 12:11 to 12:20. You will see two files in the storage bucket with 500 records in each file. This is because the 10 minute `rotate.schedule.interval.ms` condition tripped before the `flush.size=1000` condition was met. - `schema.compatibility` is set to `NONE`. - A valid schema must be available in Confluent Cloud Schema Registry to use a schema-based message format, like Avro. - For Confluent Cloud and Confluent Cloud Enterprise, organizations are limited to one task and one connector. Use of this connector is free for a limited time. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.TimestampRouter` - `io.confluent.connect.transforms.MessageTimestampRouter` - `io.confluent.connect.transforms.ExtractTopic$Header` - `io.confluent.connect.transforms.ExtractTopic$Key` - `io.confluent.connect.transforms.ExtractTopic$Value` - `io.confluent.connect.cloud.transforms.TopicRegexRouter` #### RabbitMQ Source Connector The [RabbitMQ Source Connector for Confluent Cloud](cc-rabbitmq-source.md#cc-rabbitmq-source) has the following limitations: - When paused, this connector continues to consume messages from RabbitMQ until the consumer times out. These messages remain in system memory while the connector is paused. There is no data loss when the connector resumes, since messages are acknowledged after they are flushed from memory and sent to Kafka. However, if you plan to keep this connector paused for an extended time, consider removing the connector, since message will continue to accumulate in system memory. - The connector does not currently support the following [Single Message Transformations (SMTs)](single-message-transforms.md#cc-single-message-transforms-limitations): - `org.apache.kafka.connect.transforms.ValueToKey` - `org.apache.kafka.connect.transforms.HoistField$Value`