Configure SQL Statements for Confluent Manager for Apache Flink

Confluent Manager for Apache Flink (CMF) provides several configuration properties that control the lifecycle of SQL statements.

You can set these properties in your Helm values file under the cmf.sql key, or pass them directly with --set during helm install or helm upgrade.

Statement cleanup

CMF periodically removes old statements that have reached a terminal phase (COMPLETED, FAILED, or DELETED). This prevents the metadata database from growing indefinitely.

When cleanup runs, it deletes statement records that have been in a terminal phase for longer than the configured retention period. Both the statement metadata and any associated Kubernetes resources are removed.

Property

Default

Description

cmf.sql.statement-cleanup-enabled

true

Enables periodic cleanup of old terminal statements.

cmf.sql.statement-retention-days

7

Number of days to retain terminal statements before they are eligible for cleanup.

cmf.sql.statement-cleanup-interval-hrs

6

How often, in hours, the cleanup job runs.

INSERT statement resource cleanup

When an INSERT INTO statement reaches the COMPLETED phase, CMF can automatically delete the associated Flink deployment resources (such as FlinkDeployment or FlinkSessionJob) from Kubernetes. This prevents resource leakage from completed bounded INSERT jobs.

The statement record itself is retained in CMF and can still be queried through the REST API or Confluent CLI. Only the underlying Kubernetes resources are removed.

Note

The Confluent Platform for Apache Flink Kubernetes Operator also provides a kubernetes.operator.jm-deployment.shutdown-ttl configuration parameter that controls how long JobManager deployments are retained after a job finishes. The CMF auto-cleanup acts immediately on completion, whereas the operator TTL is time-based. When both are active, whichever triggers first removes the resources.

Property

Default

Description

cmf.sql.insert-statement-auto-cleanup-enabled

true

When enabled, Kubernetes deployment resources are deleted immediately after INSERT INTO statements complete.

SELECT statement auto-termination

SELECT statements run as Flink jobs that buffer results in memory. To prevent idle SELECT statements from remaining on the cluster indefinitely when no client is consuming their results, CMF can automatically terminate them after a configurable timeout.

Property

Default

Description

cmf.sql.select-statement-auto-termination-enabled

true

Enables automatic termination of idle SELECT statements.

cmf.sql.select-statement-timeout-mins

10

Number of minutes before an idle SELECT statement is terminated.

cmf.sql.select-statement-auto-termination-check-frequency-mins

1

How often, in minutes, CMF checks for idle SELECT statements to terminate.

Example Helm configuration

The following example shows how to customize SQL statement behavior in your Helm values file:

# sql-values.yaml
cmf:
  sql:
    # Statement cleanup
    statement-cleanup-enabled: true
    statement-retention-days: 14
    statement-cleanup-interval-hrs: 12

    # INSERT resource cleanup
    insert-statement-auto-cleanup-enabled: true

    # SELECT auto-termination
    select-statement-auto-termination-enabled: true
    select-statement-timeout-mins: 15
    select-statement-auto-termination-check-frequency-mins: 2

You can also pass individual properties directly:

helm upgrade --install cmf confluentinc/confluent-manager-for-apache-flink \
  --set cmf.sql.statement-retention-days=14 \
  --set cmf.sql.select-statement-timeout-mins=15

Related content