Confluent Platform for Apache Flink Features and Support

Confluent Platform for Apache Flink® is compatible with open-source Flink. The following sections list all components that are supported by Confluent. Only configurations related to the listed components are supported. Other configurations or custom configurations are not supported.

For requirement and compatibility details, see Confluent Platform for Apache Flink compatibility.

Core components

The following core components are supported with Confluent Platform for Apache Flink:

  • Runtime

  • REST API

  • Web UI

  • Flink CLI

State backends

The following state backends are supported with Confluent Platform for Apache Flink:

  • RocksDB - recommended as the default state backend.

  • Memory - recommended for small state.

FileSystem implementations

The following FileSystem implementations are supported with Confluent Platform for Apache Flink:

  • AWS S3 - includes Presto and Hadoop variants

  • Azure blob storage

  • Google Cloud cloud storage

Data formats

The following data formats are supported with Confluent Platform for Apache Flink:

  • Avro

  • Avro (CSR)

  • CSV

  • ORC

  • Parquet

  • Protobuf

  • JSON

  • Debezium JSON

Flink APIs, libraries and metadata catalogs

The following table lists Flink APIs and their Confluent Platform for Apache Flink support.

Flink Component

Supported by Confluent

Notes

Flink SQL

Yes

SQL Shell is not currently supported.

Table API - Java

Yes

Python for this API not currently supported.

DataStream API - Java

Yes

Python for this API not currently supported.

DataSet API

No

Deprecated in open-source Flink and not supported in Confluent Platform for Apache Flink.

In addition:

  • Libraries: Complex Event Processing (CEP) is the only library supported with Confluent Platform for Apache Flink, for use with SQL. PyFlink, Flink ML, Stateful Functions and Queryable State are not supported.

  • Catalogs: GenericInMemoryCatalog and JdbcCatalog are supported with Confluent Platform for Apache Flink. Hive not currently supported.

Connectors

All Flink connectors are compatible with Confluent Platform for Apache Flink. However, support is limited to the connectors listed in the following table:

Connector

Supported by Confluent

Distribution channel

Notes

Kafka Source and Sink

Yes

Maven via packages.confluent.io

Java and SQL support only. Bundle with your user code JAR.

FileSystem Source and Sink

Yes

Open-source Flink

Java and SQL support only. Additional support charges apply. Not bundled with the Confluent Docker image.

JDBC

Yes

Open-source Flink

Java and SQL support only.

CDC Source

Yes

Open-source Flink

Databases: DB2, MySQL, Oracle, Postgres, SQLServer. Java and SQL support only.

All other connectors

No

Open-source Flink

No additional connectors are currently supported.

Deployment and monitoring

Note the following about deploying Flink jobs with Confluent Platform for Apache Flink:

  • Confluent Platform for Apache Flink supports Application Mode only.

  • Confluent Platform for Apache Flink supports high-availability deployment via Kubernetes. The default mode to deploy with Kubernetes is native. This is the only supported deployment solution for Confluent Platform for Apache Flink.

  • ZooKeeper is not supported.

  • The following metrics reporters are supported with Confluent Platform for Apache Flink:

    • Datadog

    • Prometheus

    • InfluxDB

    • JMX

    • Statsd

    • Graphite