Build Confluent Platform for Apache Flink Container Images from Published Artifacts

Confluent Platform for Apache Flink® runs on Kubernetes as a set of container images: the Flink image, the Confluent Platform for Apache Flink Kubernetes Operator image, and the Confluent Manager for Apache Flink (CMF) image. Confluent publishes these images on Docker Hub, and also publishes the underlying component artifacts as .tar.gz archives on packages.confluent.io/cp-flink/.

Most deployments use the published images directly, as described in Install Confluent Manager for Apache Flink with Helm. You can also build your own images from the published archives, when required. For example:

• Build on a base image that meets your organization’s compliance or hardening requirements.

• Add your own certificates, packages, or JARs at build time.

Important

Confluent does not provide support for issues that arise from custom base images. Support is limited to the Confluent Platform for Apache Flink components themselves. The images that Confluent publishes receive regular security and bug fixes, so keep your custom images up to date and test them thoroughly.

This topic shows how to build container images for the following components from their published archives:

• Flink (cp-flink)

• Confluent Platform for Apache Flink Kubernetes Operator (cp-flink-kubernetes-operator)

• CMF (cp-cmf)

Important

The component versions must be compatible with each other and with the CMF release you are deploying. The versions in the examples on this page are examples only. Replace them with the versions for your target release. For compatible versions, see Versions and Interoperability for Confluent Manager for Apache Flink.

The Dockerfiles on this page mirror the images that Confluent publishes:

• A multi-stage build with a Red Hat Universal Base Image (UBI) ubi-minimal builder stage.

• A ubi-micro runtime stage.

• An Azul Zulu Java Runtime Environment (JRE).

For brevity and clarity, these examples omit some details that the published images include, such as s390x support and signature verification of downloaded binaries. They also install the latest available JRE patch instead of pinning one, so your image can carry a newer JRE than the published image. The resulting images are functionally equivalent on the x86_64 and ARM64 platforms. You can substitute any base image that provides a compatible JRE for the component.

Step 1: Confirm prerequisites

1. Install a container build tool such as Docker.

   To verify that Docker is available, the following command should complete without error:

   docker version
   

2. Either mirror the required artifacts into your environment, or confirm that the machine where you build the images can access the following:

   • https://packages.confluent.io/cp-flink/ for the component archives.

   • The Red Hat UBI images on registry.access.redhat.com.

   • The Azul Zulu package repository on cdn.azul.com.

   • https://dl.fedoraproject.org for the EPEL repository, which provides jemalloc (Flink image only).

   • https://mirror.stream.centos.org for the Snappy package (Flink image only).

   • https://github.com for the gosu binary (Flink image only).

Step 2: Download and verify the archives

Confluent publishes each component under https://packages.confluent.io/cp-flink/<component>/archive/<version>/, together with a .sha512 checksum file.

The following table lists the archives and the JRE major version that each component requires. The versions shown are examples.

Component	Archive	JRE
Flink	cp-flink-2.2.0-cp2.tar.gz	11
Kubernetes Operator	cp-flink-kubernetes-operator-1.15.0-cp2.tar.gz	17
CMF	cmf-2.4.1.tar.gz	21

1. Set the version of each component you want to build. Replace the values with the versions for your target release.

   export FLINK_VERSION=2.2.0-cp2
   export OPERATOR_VERSION=1.15.0-cp2
   export CMF_VERSION=2.4.1
   

2. Download each archive and its checksum file.

   BASE=https://packages.confluent.io/cp-flink
   
   curl -fSLO ${BASE}/cp-flink/archive/${FLINK_VERSION}/cp-flink-${FLINK_VERSION}.tar.gz
   curl -fSLO ${BASE}/cp-flink/archive/${FLINK_VERSION}/cp-flink-${FLINK_VERSION}.tar.gz.sha512
   
   curl -fSLO ${BASE}/cp-flink-kubernetes-operator/archive/${OPERATOR_VERSION}/cp-flink-kubernetes-operator-${OPERATOR_VERSION}.tar.gz
   curl -fSLO ${BASE}/cp-flink-kubernetes-operator/archive/${OPERATOR_VERSION}/cp-flink-kubernetes-operator-${OPERATOR_VERSION}.tar.gz.sha512
   
   curl -fSLO ${BASE}/cp-cmf/archive/${CMF_VERSION}/cmf-${CMF_VERSION}.tar.gz
   curl -fSLO ${BASE}/cp-cmf/archive/${CMF_VERSION}/cmf-${CMF_VERSION}.tar.gz.sha512
   

3. Verify the checksums before you build. On Linux, use sha512sum. On macOS, use shasum -a 512.

   sha512sum -c cp-flink-${FLINK_VERSION}.tar.gz.sha512
   sha512sum -c cp-flink-kubernetes-operator-${OPERATOR_VERSION}.tar.gz.sha512
   sha512sum -c cmf-${CMF_VERSION}.tar.gz.sha512
   

   Each command should print OK. Do not build from an archive that fails verification.

Step 3: Build the Flink image

The cp-flink archive is Confluent’s Flink distribution and includes its own bin/docker-entrypoint.sh.

The published cp-flink images default to Java 11. Confluent also publishes a -java17 variant for every supported Flink version, and a -java21 variant for Flink 2.x.

1. Create a build directory and add the Flink archive to it.

   mkdir -p cp-flink-image
   cp cp-flink-${FLINK_VERSION}.tar.gz cp-flink-image/
   

2. Create cp-flink-image/Dockerfile with the following content. The JAVA_MAJOR build argument selects the JRE version. It defaults to 11 (the default published image); set it to 17 or 21 to match the -java17 or -java21 variants. The configuration step handles both config.yaml (Flink 1.19 and later) and flink-conf.yaml (Flink 1.18).

   ARG SRC_UBI_TAG=latest
   ARG INSTALL_ROOT=/mnt/rootfs
   FROM registry.access.redhat.com/ubi9/ubi-minimal:${SRC_UBI_TAG} AS builder
   ARG JAVA_MAJOR=11
   ARG INSTALL_ROOT
   ARG FLINK_VERSION=2.2.0-cpX
   ARG GOSU_VERSION=1.19
   ARG ARCH=amd64
   ENV FLINK_HOME=/opt/flink
   # config-parser-utils.sh needs JAVA_HOME, through bin/bash-java-utils.sh.
   ENV JAVA_HOME=/usr/lib/jvm/jre-${JAVA_MAJOR}
   USER root
   RUN microdnf install -y shadow-utils tar gzip wget yum findutils \
    && yum --nodocs install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm \
    && yum --nodocs install -y https://cdn.azul.com/zulu/bin/zulu-repo-1.0.0-1.noarch.rpm \
    && microdnf install -y zulu${JAVA_MAJOR}-ca-jre-headless \
    && ln -sf /usr/lib/jvm/zulu${JAVA_MAJOR} ${JAVA_HOME}
   ADD cp-flink-${FLINK_VERSION}.tar.gz /opt/
   RUN mv /opt/cp-flink-${FLINK_VERSION} ${FLINK_HOME}
   RUN groupadd --system --gid=9999 flink \
    && useradd --system --home-dir ${FLINK_HOME} --uid=9999 --gid=flink flink
   # Set the OpenShift permissions in the builder. Doing it in the runtime stage would
   # add a second copy of every file in a new layer.
   RUN chmod -R g+rwX ${FLINK_HOME}
   # Bind endpoints to all interfaces. Versions 1.19 and later keep this in
   # config.yaml, which config-parser-utils.sh rewrites; 1.18 uses flink-conf.yaml.
   RUN set -ex; \
       chown -R flink:flink ${FLINK_HOME}; \
       CONF_FILE="${FLINK_HOME}/conf/flink-conf.yaml"; \
       if [ ! -e "${CONF_FILE}" ]; then \
         /bin/bash "${FLINK_HOME}/bin/config-parser-utils.sh" \
             "${FLINK_HOME}/conf" "${FLINK_HOME}/bin" "${FLINK_HOME}/lib" \
             "-repKV" "rest.address,localhost,0.0.0.0" \
             "-repKV" "rest.bind-address,localhost,0.0.0.0" \
             "-repKV" "jobmanager.bind-host,localhost,0.0.0.0" \
             "-repKV" "taskmanager.bind-host,localhost,0.0.0.0" \
             "-rmKV" "taskmanager.host=localhost"; \
       else \
         sed -i -e 's/rest.address: localhost/rest.address: 0.0.0.0/g' \
                -e 's/rest.bind-address: localhost/rest.bind-address: 0.0.0.0/g' \
                -e 's/jobmanager.bind-host: localhost/jobmanager.bind-host: 0.0.0.0/g' \
                -e 's/taskmanager.bind-host: localhost/taskmanager.bind-host: 0.0.0.0/g' \
                -e '/taskmanager.host: localhost/d' "${CONF_FILE}"; \
       fi
   RUN mkdir -p ${INSTALL_ROOT} /etc/dnf /var/cache/microdnf; touch /etc/dnf/dnf.conf; \
       microdnf install -y --noplugins --config=/etc/dnf/dnf.conf --setopt=cachedir=/var/cache/microdnf \
         --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf \
         --installroot=${INSTALL_ROOT} --releasever=35 \
         hostname findutils util-linux gettext shadow-utils procps jemalloc zulu${JAVA_MAJOR}-ca-jre-headless \
    && microdnf clean all && rm -rf ${INSTALL_ROOT}/var/cache/* \
    && mkdir -p ${INSTALL_ROOT}/usr/lib/jvm \
    && ln -sf /usr/lib/jvm/zulu${JAVA_MAJOR} ${INSTALL_ROOT}${JAVA_HOME}
   # Snappy native library. It cannot be installed into ${INSTALL_ROOT}, so install the
   # RPM into the builder filesystem here and copy the .so into the runtime stage below.
   RUN case "${ARCH}" in \
           amd64) RPM_ARCH=x86_64 ;; \
           arm64) RPM_ARCH=aarch64 ;; \
           *) RPM_ARCH=${ARCH} ;; \
       esac; \
       set -ex; \
       wget -q https://mirror.stream.centos.org/9-stream/BaseOS/${RPM_ARCH}/os/Packages/snappy-1.1.8-8.el9.${RPM_ARCH}.rpm \
    && rpm -Uvh --noscripts snappy-*.rpm \
    && rm -f snappy-*.rpm
   RUN set -ex; mkdir -p ${INSTALL_ROOT}/usr/local/bin; \
       wget -nv -O ${INSTALL_ROOT}/usr/local/bin/gosu \
         "https://github.com/tianon/gosu/releases/download/${GOSU_VERSION}/gosu-${ARCH}"; \
       chmod +x ${INSTALL_ROOT}/usr/local/bin/gosu
   FROM registry.access.redhat.com/ubi9/ubi-micro:${SRC_UBI_TAG}
   ARG INSTALL_ROOT
   ARG JAVA_MAJOR=11
   USER root
   RUN mkdir /licenses
   COPY --from=builder /opt/flink/LICENSE /licenses/
   COPY --from=builder ${INSTALL_ROOT}/ /
   # jemalloc is included in ${INSTALL_ROOT} above; copy the Snappy library separately.
   COPY --from=builder /usr/lib64/libsnappy.so.1.1.8 /usr/lib64/libsnappy.so.1
   ENV FLINK_HOME=/opt/flink
   ENV JAVA_HOME=/usr/lib/jvm/jre-${JAVA_MAJOR}
   ENV PATH=$FLINK_HOME/bin:$JAVA_HOME/bin:$PATH
   # For OpenShift support, define the flink user numerically.
   RUN useradd --system --home-dir $FLINK_HOME --uid=9999 flink
   COPY --from=builder --chown=flink:0 /opt/flink/ $FLINK_HOME
   WORKDIR $FLINK_HOME
   # The entrypoint script ships in the archive.
   COPY --from=builder /opt/flink/bin/docker-entrypoint.sh /docker-entrypoint.sh
   RUN set -ex; chmod +x /docker-entrypoint.sh
   ENTRYPOINT ["/docker-entrypoint.sh"]
   EXPOSE 6123 8081
   USER 9999
   CMD ["help"]
   

3. Build the image. Set FLINK_VERSION to your release, JAVA_MAJOR to the Java version you want (11, 17, or 21), and ARCH to amd64 or arm64.

   docker build \
     --build-arg FLINK_VERSION=${FLINK_VERSION} \
     --build-arg JAVA_MAJOR=11 \
     --build-arg ARCH=amd64 \
     -t <your-registry>/cp-flink:${FLINK_VERSION} \
     cp-flink-image/
   

Step 4: Build the Kubernetes Operator image

The cp-flink-kubernetes-operator archive includes the operator JARs, the metrics reporter plugins, and its own bin/docker-entrypoint.sh.

1. Create a build directory and add the operator archive to it.

   mkdir -p cp-flink-kubernetes-operator-image
   cp cp-flink-kubernetes-operator-${OPERATOR_VERSION}.tar.gz cp-flink-kubernetes-operator-image/
   

2. Create cp-flink-kubernetes-operator-image/Dockerfile with the following content.

   ARG SRC_UBI_TAG=latest
   ARG INSTALL_ROOT=/mnt/rootfs
   
   FROM registry.access.redhat.com/ubi9/ubi-minimal:${SRC_UBI_TAG} AS builder
   ARG ZULU_JAVA17_JRE_VERSION=zulu17-ca-jre-headless
   ARG INSTALL_ROOT
   ARG OPERATOR_VERSION=1.15.0-cpX
   
   USER root
   RUN microdnf install -y yum findutils tar gzip \
    && yum --nodocs install -y https://cdn.azul.com/zulu/bin/zulu-repo-1.0.0-1.noarch.rpm \
    && microdnf install -y ${ZULU_JAVA17_JRE_VERSION}
   RUN mkdir -p ${INSTALL_ROOT} /etc/dnf /var/cache/microdnf; touch /etc/dnf/dnf.conf; \
       microdnf install -y --noplugins --config=/etc/dnf/dnf.conf --setopt=cachedir=/var/cache/microdnf \
         --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf \
         --installroot=${INSTALL_ROOT} --releasever=35 \
         shadow-utils ${ZULU_JAVA17_JRE_VERSION} \
    && microdnf clean all && rm -rf ${INSTALL_ROOT}/var/cache/*
   
   # Unpack the published archive.
   ADD cp-flink-kubernetes-operator-${OPERATOR_VERSION}.tar.gz /opt/
   
   # The runtime image.
   FROM registry.access.redhat.com/ubi9/ubi-micro:${SRC_UBI_TAG}
   ARG INSTALL_ROOT
   ARG OPERATOR_VERSION=1.15.0-cpX
   
   USER root
   COPY --from=builder ${INSTALL_ROOT}/ /
   RUN mkdir /licenses
   COPY --from=builder \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/doc/cp-flink-kubernetes-operator/LICENSE \
     /licenses/
   RUN ln -sf /usr/lib/jvm/java-17-zulu-openjdk-ca /usr/lib/jvm/java-17
   
   ENV FLINK_HOME=/opt/flink
   ENV JAVA_HOME=/usr/lib/jvm/java-17
   ENV FLINK_PLUGINS_DIR=$FLINK_HOME/plugins
   ENV OPERATOR_LIB=$FLINK_HOME/operator-lib
   ENV OPERATOR_JAR=flink-kubernetes-operator-${OPERATOR_VERSION}-shaded.jar
   ENV WEBHOOK_JAR=flink-kubernetes-webhook-${OPERATOR_VERSION}-shaded.jar
   ENV KUBERNETES_STANDALONE_JAR=flink-kubernetes-standalone-${OPERATOR_VERSION}.jar
   ENV PATH=$FLINK_HOME/bin:$JAVA_HOME/bin:$PATH
   
   RUN mkdir -p $OPERATOR_LIB $FLINK_PLUGINS_DIR
   RUN groupadd --system --gid=9999 flink \
    && useradd --system --home-dir $FLINK_HOME --uid=9999 --gid=flink flink
   
   WORKDIR /flink-kubernetes-operator
   
   # Lay out the operator jars, metrics plugins, license files, and entrypoint from
   # the archive.
   COPY --from=builder --chown=flink:flink \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/java/cp-flink-kubernetes-operator/*.jar \
     /flink-kubernetes-operator/
   COPY --from=builder --chown=flink:flink \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/java/cp-flink-kubernetes-operator/plugins/ \
     ${FLINK_PLUGINS_DIR}/
   COPY --from=builder --chown=flink:flink \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/doc/cp-flink-kubernetes-operator/NOTICE \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/doc/cp-flink-kubernetes-operator/LICENSE \
     /flink-kubernetes-operator/
   COPY --from=builder --chown=flink:flink \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/share/doc/cp-flink-kubernetes-operator/licenses/ \
     /flink-kubernetes-operator/licenses/
   COPY --from=builder \
     /opt/cp-flink-kubernetes-operator-${OPERATOR_VERSION}/bin/docker-entrypoint.sh \
     /docker-entrypoint.sh
   RUN chmod +x /docker-entrypoint.sh && chown -R flink:flink $FLINK_HOME /flink-kubernetes-operator
   
   USER flink
   ENTRYPOINT ["/docker-entrypoint.sh"]
   CMD ["help"]
   

3. Build the image.

   docker build \
     --build-arg OPERATOR_VERSION=${OPERATOR_VERSION} \
     -t <your-registry>/cp-flink-kubernetes-operator:${OPERATOR_VERSION} \
     cp-flink-kubernetes-operator-image/
   

Step 5: Build the CMF image

The cp-cmf archive includes the CMF application JAR, the SQL statement service JARs, the Flink filesystem plugins, and its own bin/docker-entrypoint.sh entrypoint. The entrypoint expects the CMF JAR at /app/cmf.jar, the statement service JARs in /app/sql, and any JDBC drivers that you supply in /opt/cmf/jdbc-drivers.

1. Create a build directory and add the CMF archive to it.

   mkdir -p cp-cmf-image
   cp cmf-${CMF_VERSION}.tar.gz cp-cmf-image/
   

2. Create cp-cmf-image/Dockerfile with the following content.

   ARG SRC_UBI_TAG=latest
   ARG INSTALL_ROOT=/mnt/rootfs
   
   FROM registry.access.redhat.com/ubi9/ubi-minimal:${SRC_UBI_TAG} AS builder
   ARG ZULU_JAVA21_JRE_VERSION=zulu21-ca-jre-headless
   ARG INSTALL_ROOT
   ARG CMF_VERSION=2.4.X
   
   USER root
   RUN microdnf install -y yum findutils tar gzip \
    && yum --nodocs install -y https://cdn.azul.com/zulu/bin/zulu-repo-1.0.0-1.noarch.rpm \
    && microdnf install -y ${ZULU_JAVA21_JRE_VERSION}
   RUN mkdir -p ${INSTALL_ROOT} /etc/dnf /var/cache/microdnf; touch /etc/dnf/dnf.conf; \
       microdnf install -y --noplugins --config=/etc/dnf/dnf.conf --setopt=cachedir=/var/cache/microdnf \
         --setopt=reposdir=/etc/yum.repos.d --setopt=varsdir=/etc/dnf \
         --installroot=${INSTALL_ROOT} --releasever=35 \
         shadow-utils ${ZULU_JAVA21_JRE_VERSION} \
    && microdnf clean all && rm -rf ${INSTALL_ROOT}/var/cache/* \
    && mkdir -p ${INSTALL_ROOT}/usr/lib/jvm \
    && ln -sf /usr/lib/jvm/java-21-zulu-openjdk-ca ${INSTALL_ROOT}/usr/lib/jvm/jre-21
   
   # Unpack the published archive and lay out the application under /app.
   ADD cmf-${CMF_VERSION}.tar.gz /opt/
   RUN set -eux; \
       SRC=/opt/cmf-${CMF_VERSION}; \
       mkdir -p /app/sql /opt/cmf/plugins /opt/cmf/user-libs /opt/cmf/jdbc-drivers; \
       cp "$SRC"/share/java/cmf/cmf-app-*.jar /app/cmf.jar; \
       cp "$SRC"/share/java/cmf-statement-service/*.jar /app/sql/; \
       cp -r "$SRC"/share/java/cmf-plugins/. /opt/cmf/plugins/; \
       cp "$SRC"/bin/docker-entrypoint.sh /app/docker-entrypoint.sh; \
       chmod +x /app/docker-entrypoint.sh; \
       mkdir -p /licenses; \
       cp "$SRC"/share/doc/cmf/LICENSE /licenses/
   
   # The runtime image.
   FROM registry.access.redhat.com/ubi9/ubi-micro:${SRC_UBI_TAG}
   ARG INSTALL_ROOT
   
   USER root
   COPY --from=builder ${INSTALL_ROOT}/ /
   
   ENV JAVA_HOME=/usr/lib/jvm/jre-21
   ENV PATH="$PATH:$JAVA_HOME/bin"
   
   COPY --from=builder /licenses /licenses
   
   # Dedicated application user (matches the published image UID and GID).
   ARG UID=1058
   ARG GID=1058
   RUN groupadd -g ${GID} appuser \
    && useradd -u ${UID} -g ${GID} -m -d /home/appuser -s /bin/bash appuser
   # Let the entrypoint self-register a passwd entry when the container runs as an
   # arbitrary UID (OpenShift pattern). Required for OS-user lookups, which artifact
   # I/O to s3://, gs://, and abfs:// depends on.
   RUN chmod g=u /etc/passwd
   
   COPY --from=builder --chown=appuser:appuser /app /app
   COPY --from=builder --chown=appuser:appuser /opt/cmf/plugins /opt/cmf/plugins
   COPY --from=builder --chown=appuser:appuser /opt/cmf/user-libs /opt/cmf/user-libs
   COPY --from=builder --chown=appuser:appuser /opt/cmf/jdbc-drivers /opt/cmf/jdbc-drivers
   
   WORKDIR /app
   USER appuser
   RUN mkdir local && chmod 777 local
   EXPOSE 8080
   ENTRYPOINT ["./docker-entrypoint.sh"]
   

3. Build the image.

   docker build \
     --build-arg CMF_VERSION=${CMF_VERSION} \
     -t <your-registry>/cp-cmf:${CMF_VERSION} \
     cp-cmf-image/
   

Step 6: Verify the images

Confirm that each image starts. Each image prints its version on startup.

• The Flink and Kubernetes Operator images accept a help argument that prints usage:

  docker run --rm <your-registry>/cp-flink:${FLINK_VERSION} help
  docker run --rm <your-registry>/cp-flink-kubernetes-operator:${OPERATOR_VERSION} help
  

• The CMF image starts the CMF application. Run it and confirm the startup banner appears in the logs, then stop the container:

  docker run --rm <your-registry>/cp-cmf:${CMF_VERSION}
  

Note

Outside a Kubernetes cluster, the Kubernetes Operator and CMF cannot reach a Kubernetes API server, so they exit after logging their startup information. This is expected. The purpose of this step is to confirm that the image is built correctly and the application starts.

Step 7: Use your images

Before you use these images with Helm, push them to a registry that your Kubernetes cluster can pull from. Then point each chart at your registry instead of the default Confluent images.

• For the Kubernetes Operator, set the image values when you install the flink-kubernetes-operator chart.

• For CMF, set the image.repository, image.name, and image.tag values, as described in Install Confluent Manager for Apache Flink with Helm.

CMF deploys your Flink image based on the image you reference in your Flink environment, application, or statement resources.

If your registry requires authentication, create an image pull secret and reference it through the imagePullSecretRef Helm value.

Related content

• Install Confluent Manager for Apache Flink with Helm

• Versions and Interoperability for Confluent Manager for Apache Flink

• Get Started with Applications in Confluent Manager for Apache Flink