# Configure Storage for Confluent Manager for Apache Flink Flink applications are deployed in Confluent Platform with Confluent Manager for Apache Flink® (CMF), a central management component that enables users to securely manage a fleet of Flink jobs across multiple environments. This topic describes how to configure storage for CMF. Starting with CMF 2.1.1, you can configure CMF to use PostgreSQL as the metadata backend instead of the default SQLite database. PostgreSQL is recommended for production deployments and provides better support for high availability scenarios. Starting with CMF 2.2.0, you can also configure CMF to use Microsoft SQL Server as the metadata backend. Starting with CMF 2.4.0, you can also configure CMF to use IBM Db2 as the metadata backend. ## Database options CMF supports two database types. After you have deployed with one database type, changing to another type requires data migration. In addition, changing the database type after initial deployment can result in data loss. Supported database types: - **local** (SQLite): The default option, stored in a persistent volume. Good for development, testing, and small deployments. Note that using SQLite on Network File Systems (NFS) is not recommended. For more information, see the [SQLite FAQ](https://sqlite.org/faq.html#q5). - **jdbc** (PostgreSQL, SQL Server, or Db2): External JDBC database. Recommended for production deployments and high availability. - PostgreSQL: Available starting with CMF 2.1.1 - Microsoft SQL Server: Available starting with CMF 2.2.0 - IBM Db2: Available starting with CMF 2.4.0 #### IMPORTANT SQLite is not supported in the Linux s390x architecture Docker images. ## Configure PostgreSQL To configure CMF to use PostgreSQL as the metadata backend, you need to: 1. Create a Kubernetes secret containing the PostgreSQL password. 2. Configure the database settings in your Helm values file. ### Prerequisites Before configuring PostgreSQL, ensure you have: - A PostgreSQL server accessible from your Kubernetes cluster - A database created in PostgreSQL - A PostgreSQL user with appropriate permissions - A Kubernetes secret containing the PostgreSQL password ### Example configuration The following configuration file shows how to provision Confluent Manager for Apache Flink with PostgreSQL as the metadata backend. You pass this configuration file to Helm when you install CMF. ```yaml # postgresql-values.yaml cmf: database: # Set type to "jdbc" to use PostgreSQL type: jdbc jdbc: # Database engine type engine: postgresql # Database server hostname or IP address url: postgresql.example.com # Database server port (default: 5432 for PostgreSQL) port: 5432 # Database name (must be created beforehand) database: cmf # Database username user: cmf # Database options, string to append to the JDBC URL for extra parameters # Example for PostgreSQL: options: "sslmode=require" # Refer to your database's documentation for supported parameters and the exact format options: "" # Database password configuration password: # Name of Kubernetes secret containing the password kubernetesSecretName: cmf-postgresql-password # Key within the secret that contains the password kubernetesSecretProperty: password ``` ### Create the password secret Before installing CMF, create a Kubernetes secret containing the PostgreSQL password: ```bash kubectl create secret generic cmf-postgresql-password \ --from-literal=password= \ --namespace= ``` ### Install with Helm When you make the `helm install` call, use the `-f` flag to pass the YAML file with the PostgreSQL configuration: ```bash helm upgrade --install cmf confluentinc/confluent-manager-for-apache-flink \ -f postgresql-values.yaml ``` ## Configure SQL Server Starting with CMF 2.2.0, you can configure CMF to use Microsoft SQL Server as the metadata backend. To configure CMF to use SQL Server as the metadata backend, you need to: 1. Create a Kubernetes secret containing the SQL Server password. 2. Configure the database settings in your Helm values file. ### Prerequisites Before configuring SQL Server, ensure you have: - A SQL Server instance accessible from your Kubernetes cluster - A database created in SQL Server - A SQL Server user with appropriate permissions - A Kubernetes secret containing the SQL Server password ### Example configuration The following configuration file shows how to provision Confluent Manager for Apache Flink with SQL Server as the metadata backend. You pass this configuration file to Helm when you install CMF. ```yaml # sqlserver-values.yaml cmf: database: # Set type to "jdbc" to use SQL Server type: jdbc jdbc: # Database engine type engine: sqlserver # Database server hostname or IP address url: sqlserver.example.com # Database server port. SQL Server listens on 1433 by default, and the # chart default is 5432, so set it explicitly. port: 1433 # Database name (must be created beforehand) database: cmf # Database username user: cmf # Database options, string to append to the JDBC URL for extra parameters # Example for SQL Server: options: "encrypt=true;trustServerCertificate=true;" # Refer to your database's documentation for supported parameters and the exact format options: "" # Database password configuration password: # Name of Kubernetes secret containing the password kubernetesSecretName: cmf-sqlserver-password # Key within the secret that contains the password kubernetesSecretProperty: password ``` ### Create the password secret Before installing CMF, create a Kubernetes secret containing the SQL Server password: ```bash kubectl create secret generic cmf-sqlserver-password \ --from-literal=password= \ --namespace= ``` ### Install with Helm When you make the `helm install` call, use the `-f` flag to pass the YAML file with the SQL Server configuration: ```bash helm upgrade --install cmf confluentinc/confluent-manager-for-apache-flink \ -f sqlserver-values.yaml ``` ## Configure Db2 Starting with CMF 2.4.0, you can configure CMF to use IBM Db2 as the metadata backend. #### IMPORTANT The IBM Db2 JDBC driver (`jcc`) is licensed under IBM’s International Program License Agreement (IPLA) and is not bundled with CMF. You must supply the driver yourself under IBM’s license terms. CMF loads the driver at startup from the loader-path directory (default `/opt/cmf/jdbc-drivers`, override with the `CMF_DRIVER_PATH` environment variable) and fails fast with an actionable error if the driver class is not present. To configure CMF to use Db2 as the metadata backend, you need to: 1. Create a Kubernetes secret containing the Db2 password. 2. Provision the IBM Db2 JDBC driver into the CMF pod at the loader-path directory. 3. Configure the database settings in your Helm values file. ### Prerequisites Before configuring Db2, ensure you have: - A Db2 instance accessible from your Kubernetes cluster - A database created in Db2 - A Db2 user with appropriate permissions on that database - A Kubernetes secret containing the Db2 password - Access to the IBM Db2 JDBC driver (`jcc`) JAR from a source you are licensed to use, such as your internal artifact repository ### Provision the Db2 JDBC driver Because the IBM Db2 JDBC driver is not bundled with CMF, you provision it into the CMF pod at startup. The recommended pattern uses an `initContainer` that downloads the driver JAR into a shared `emptyDir` volume mounted at the CMF loader path. The concrete YAML for this pattern is included in the `db2-values.yaml` example under **Example configuration** below. The example uses an example driver version and a placeholder fetch source; replace both with a driver version that matches your Db2 server and a source you are licensed to pull from, such as your internal artifact repository. #### IMPORTANT `mountedVolumes` and `initContainers` are top-level Helm values. If your existing values file already sets either key, append the Db2 driver entries to your existing lists rather than replacing them. ### Example configuration The following configuration file shows how to provision Confluent Manager for Apache Flink with Db2 as the metadata backend. It combines the database settings with the driver `initContainer` and mounted volume from the previous step. You pass this configuration file to Helm when you install CMF. ```yaml # db2-values.yaml cmf: database: # Set type to "jdbc" to use Db2 type: jdbc jdbc: # Database engine type engine: db2 # Database server hostname or IP address url: db2.example.com # Database server port (Db2 listens on 50000 by default; the chart # default is 5432, so set it explicitly) port: 50000 # Database name (must be created beforehand) database: cmf # Database username user: cmf # Database options, string to append to the JDBC URL for extra parameters. # For Db2, options are appended with a leading colon, and each option, # including the last one, must end with a semicolon, for example: # options: "sslConnection=true;" # Refer to your database's documentation for supported parameters and the exact format. options: "" # Database password configuration password: # Name of Kubernetes secret containing the password kubernetesSecretName: cmf-db2-password # Key within the secret that contains the password kubernetesSecretProperty: password # Provision the IBM Db2 JDBC driver into the CMF pod at the loader path. mountedVolumes: volumes: - name: db2-driver emptyDir: {} volumeMounts: - name: db2-driver mountPath: /opt/cmf/jdbc-drivers initContainers: - name: fetch-db2-driver image: command: ["sh", "-c"] args: - curl -fsSL -o /drivers/jcc.jar /com/ibm/db2/jcc/11.5.9.0/jcc-11.5.9.0.jar volumeMounts: - name: db2-driver mountPath: /drivers ``` ### Create the password secret Before installing CMF, create a Kubernetes secret containing the Db2 password: ```bash kubectl create secret generic cmf-db2-password \ --from-literal=password= \ --namespace= ``` ### Install with Helm When you make the `helm install` call, use the `-f` flag to pass the YAML file with the Db2 configuration: ```bash helm upgrade --install cmf confluentinc/confluent-manager-for-apache-flink \ -f db2-values.yaml ``` ## Configuration fields The following table describes the database configuration fields: | Field | Description | |---------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `database.type` | Type of database to use. Set to `jdbc` for PostgreSQL, SQL Server, or Db2. Default is `local` (SQLite). | | `database.jdbc.engine` | Database engine type. Supported values: `postgresql` (available starting with CMF 2.1.1), `sqlserver` (available starting with CMF 2.2.0), or `db2` (available starting with CMF 2.4.0). | | `database.jdbc.url` | Database server hostname or IP address. Required when using `jdbc` type. | | `database.jdbc.port` | Database server port. The Helm chart default is `5432` for every
engine. Set the port explicitly for SQL Server or Db2. If you set the
port to `0`, CMF uses the engine default: `5432` for PostgreSQL,
`1433` for SQL Server, or `50000` for Db2. | | `database.jdbc.database` | Name of the database. The database must be created beforehand. | | `database.jdbc.user` | Database username. Required when using `jdbc` type. | | `database.jdbc.options` | Optional string to append to the JDBC URL for extra connection
parameters. For example, for SQL Server:
`encrypt=true;trustServerCertificate=true;`. For PostgreSQL:
`sslmode=require`. For Db2: `sslConnection=true;`. For Db2, each
option must end with a semicolon. Refer to your database’s documentation
for supported parameters and the exact format. | | `database.jdbc.password.kubernetesSecretName` | Name of the Kubernetes secret containing the database password. Required when using `jdbc` type. | | `database.jdbc.password.kubernetesSecretProperty` | Key within the secret that contains the password. Optional. Default is
`password`. | ## Related content - [Install Confluent Manager for Apache Flink with Helm](helm.md#install-cmf-helm) - [Get Started with Confluent Platform for Apache Flink](../get-started/overview.md#cp-flink-get-started-overview)