Use Custom Docker Registry for Confluent Platform Using Confluent for Kubernetes

Confluent for Kubernetes (CFK) deploys publicly available Docker images hosted on Docker Hub from the confluentinc repositories.

If you want to have CFK deploy Confluent Platform images from a different registry, or from your own set of repositories on Docker Hub, take the following steps.

Step 1: Host the images in the custom registry

  1. Get a list of the images and their tags to install.

    The default CFK and Confluent Platform image URIs are:

    docker.io/confluentinc/confluent-init-container:<init-tag>
    docker.io/confluentinc/confluent-operator:<cfk-tag>
    docker.io/confluentinc/cp-enterprise-control-center:<tag>
    docker.io/confluentinc/cp-enterprise-replicator:<tag>
    docker.io/confluentinc/cp-kafka-rest:<tag>
    docker.io/confluentinc/cp-ksqldb-server:<tag>
    docker.io/confluentinc/cp-schema-registry:<tag>
    docker.io/confluentinc/cp-server:<tag>
    docker.io/confluentinc/cp-server-connect:<tag>
    docker.io/confluentinc/cp-zookeeper:<tag>
    
    • The <tag> for a Confluent Platform component image is the release version of the component. For example, the URI for the ZooKeeper version 7.6.1 image is docker.io/confluentinc/cp-zookeeper:7.6.1.
    • The <init-tag> for the init container is the version of CFK.
    • The default <cfk-tag> for the confluent-operator image is 0.921.20 for the current version. To select a different version tag, refer to Confluent for Kubernetes image tags.
  2. Pull the desired CFK and Confluent Platform images from the Confluent Docker repositories.

    By default, the docker pull command pulls the image matching the system’s architecture where the command is executed.

    • To pull the CFK images from the workstation with the same architecture of the CFK and Confluent Platform cluster you want to use:

      docker pull confluentinc/<image-name>:<image-tag>
      

      For example:

      docker pull confluentinc/confluent-operator:0.921.20
      
    • To pull the CFK images from a workstation whose architecture is different from the architecture of the Kubernetes cluster that you want to run CFK, specify the CFK architecture with the --platform flag:

      docker pull --platform <CFK cluster architecture> <image-name>:<image-tag>
      

      For example, if you run docker pull on an ARM-based workstation, but want to run CFK and Confluent Platform on a Kubernetes cluster running AMD64 worker nodes:

      docker pull --platform linux/amd64 confluentinc/confluent-operator:<tag>
      

      Alternatively, you can pull and push multi-architecture images (linux/amd64 and linux/arm64) of CFK and Confluent Platform components to your private repository.

  3. Push the CFK and Confluent Platform images to your target registry.

Step 2: Create a pull secret for the custom registry

If you need to authenticate to image registries with the username and password, for each of those registries, create a Kubernetes secret of the docker-registry type in the same namespace as the Confluent Platform component.

An image pull secret contains the credentials to the registry where the image will be pulled from.

<confluent-namespace> denotes the namespace you deploy this Confluent Platform component in.

kubectl create secret docker-registry <registry-secret> \
  --namespace <confluent-namespace>
  --docker-server=<Docker-registry-FQDN> \
  --docker-username=<Docker-username> \
  --docker-password=<Docker-password> \
  --docker-email=<Docker-email>

Step 3: Configure Confluent Platform to be installed from the custom registry

Configure the following settings in the custom resource (CR) for each Confluent Platform component:

spec:
  image:            # Changes in this property will roll the cluster.

    application:    # Required. Specify
                    # <Docker-registry FQDN>/<docker-repository-name>/<component-image-name>:<tag>

    init:           # Required. Specify
                    # <Docker-registry FQDN>/<docker-repository-name>/<init-container-image-name>:<tag>

    pullSecretRef:  # The image pull secret added in the previous step.

For example, to pull the Schema Registry image from a custom registry using the registry-secret created in the previous step:

spec:
  image:
    application: docker.io/myrepository/cp-schema-registry:7.6.1
    init: docker.io/myrepository/confluent-init-container:2.8.2
    pullSecretRef:
      - registry-secret