Use Custom Docker Registry for Confluent Platform Using Confluent for Kubernetes

Confluent for Kubernetes (CFK) deploys publicly available Docker images hosted on Docker Hub from the confluentinc repositories. A custom registry is any Docker image registry other than these default confluentinc repositories, such as a private registry or your own repositories on Docker Hub.

If you want CFK to deploy Confluent Platform images from a different registry, or from your own set of repositories on Docker Hub, take the following steps.

Step 1: Host the images in the custom registry

  1. Get a list of the images and their tags to install.

    The default CFK and Confluent Platform image URIs are:

    docker.io/confluentinc/confluent-init-container:<init-tag>
    docker.io/confluentinc/confluent-operator:<cfk-tag>
    docker.io/confluentinc/cp-enterprise-control-center-next-gen:<c3-tag>
    docker.io/confluentinc/cp-enterprise-control-center:<tag>
    docker.io/confluentinc/cp-enterprise-replicator:<tag>
    docker.io/confluentinc/cp-kafka-rest:<tag>
    docker.io/confluentinc/cp-ksqldb-server:<tag>
    docker.io/confluentinc/cp-schema-registry:<tag>
    docker.io/confluentinc/cp-server:<tag>
    docker.io/confluentinc/cp-server-connect:<tag>
    docker.io/confluentinc/cp-zookeeper:<tag>
    

    Placeholder

    Description

    <tag>

    The release version of the Confluent Platform component image. For example, the URI for the ZooKeeper version 8.3.0 image is docker.io/confluentinc/cp-zookeeper:8.3.0.

    <c3-tag>

    The release version of Control Center.

    <init-tag>

    The version of CFK used for the init container image.

    <cfk-tag>

    The default tag for the confluent-operator image, 0.1718.10 for the current version. For other version tags, see Confluent for Kubernetes image tags.

    Important

    Starting with Confluent Platform version 8.0, ZooKeeper is no longer part of Confluent Platform.

  2. Pull the desired CFK and Confluent Platform images from the Confluent Docker repositories.

    By default, the docker pull command pulls the image matching the system’s architecture where the command is executed.

    • To pull the CFK images from the workstation with the same architecture as the CFK and Confluent Platform cluster you want to use:

      docker pull confluentinc/<image-name>:<image-tag>
      

      For example:

      docker pull confluentinc/confluent-operator: 0.1718.10
      
    • To pull the CFK images from a workstation whose architecture is different from the architecture of the Kubernetes cluster on which you want to run CFK, specify the CFK architecture with the --platform flag:

      docker pull --platform <cfk_cluster_architecture> <image-name>:<image-tag>
      

      For example, if you run docker pull on an ARM-based workstation, but want to run CFK and Confluent Platform on a Kubernetes cluster running AMD64 worker nodes:

      docker pull --platform linux/amd64 confluentinc/confluent-operator:<tag>
      

      Alternatively, you can pull and push multi-architecture images (linux/amd64 and linux/arm64) of CFK and Confluent Platform components to your private repository.

  3. Push the CFK and Confluent Platform images to your target registry.

Step 2 (optional): Create a pull secret for the custom registry

An image pull secret contains the credentials to the registry where the image is pulled from.

Create a Kubernetes secret of the docker-registry type for each registry that requires authentication. Use the registry’s username and password, and create the secret in the same namespace as the Confluent Platform component.

<confluent-namespace> denotes the namespace you deploy this Confluent Platform component in.

kubectl create secret docker-registry <registry-secret> \
  --namespace <confluent-namespace> \
  --docker-server=<docker_registry_fqdn> \
  --docker-username=<docker_username> \
  --docker-password=<docker_password> \
  --docker-email=<docker_email>

Step 3: Install Confluent Platform from the custom registry

Configure the following settings in the custom resource (CR) for each Confluent Platform component:

spec:
  image:            # Changes in this property will roll the cluster.

    application:    # Required. Specify
                    # <docker_registry_fqdn>/<docker-repository-name>/<component-image-name>:<tag>

    init:           # Required. Specify
                    # <docker_registry_fqdn>/<docker-repository-name>/<init-container-image-name>:<tag>

    pullSecretRef:  # The image pull secret added in the previous step.

For example, to pull the Schema Registry image from a custom registry using the registry-secret created in the previous step:

spec:
  image:
    application: docker.io/myrepository/cp-schema-registry:8.3.0
    init: docker.io/myrepository/confluent-init-container:3.4.0
    pullSecretRef:
      - registry-secret

Next steps

After you install Confluent Platform from the custom registry, deploy your Confluent Platform cluster by following Deploy Confluent Platform Using Confluent for Kubernetes.