Configure Kafka Connect¶

Add connectors¶

Add a new connector to one of the following Connect images:

For Confluent Platform 6.2.x and above, use the cp-server-connect image.

The remainder of this document uses this image.
For Confluent Platform 6.1.x and below, use the cp-server-connect-operator image.

The image contains Connect and all of its dependencies. It does not contain any Connector jars.

To add new connectors to the Connect image, you need to build a new Docker image that has the new connectors installed.

Create a Dockerfile in <dockerfile-dir> to add one or more connectors to the cp-server-connect image.

You can either:

Pull connectors from Confluent Hub.
Use the connector JAR downloaded to the machine you are running the Docker build from.

To pull connectors from Confluent Hub:

Create a Dockerfile as follows:

FROM confluentinc/cp-server-connect:<Confluent Platform release>
USER root
RUN confluent-hub install --no-prompt <connector1>:<connector1-version> \
  && confluent-hub install --no-prompt <connector2>:<connector2-version> \
  && ...
USER 1001

An example Dockerfile to create a Docker image with the data-gen connector from Confluent Hub:

FROM confluentinc/cp-server-connect:6.2.0
USER root
RUN  confluent-hub install --no-prompt confluentinc/kafka-connect-datagen:0.3.3
USER 1001

To use the connector JAR downloaded to the machine you are running the Docker build from:

Create a Dockerfile as follows:

FROM confluentinc/cp-server-connect:<Confluent Platform release>
ADD <local-connector1-path> /usr/share/java/<connector1> \
  && <local-connector2-path> /usr/share/java/<connector2> \
  && ...
USER 1001

An example Dockerfile to use the data-gen connector existing on your local machine in the <connector-dir> directory:

FROM confluentinc/cp-server-connect:6.2.0
ADD​ my-connector-dir/confluentinc-kafka-connect-datagen /usr/share/java/confluentinc-kafka-connect-datagen
USER 1001

Build and push the image with the following commands;

docker build <dockerfile-dir> -t <someregistry>/<somerepository>:<sometag>

docker push <someregistry>/<somerepository>:<sometag>

Get the Docker image details from the output of the above process and specify the repository and tag in the Connect custom resource (CR):
```
connect:
  spec:
    image:
      application: <someregistry>/<somerepository>:<sometag>
```