# Manage Kafka Connect Using Confluent Control Center for Confluent Platform Confluent Control Center provides the convenience of managing connectors for multiple Kafka Connect clusters. Use Control Center to do any of the following: - [Add a connector](#c3-add-connector) by completing UI fields. - Add a connector by [uploading a connector configuration file](#c3-upload-connector-config). - [Download connector configuration files](#c3-download-connector-config) to reuse in another connector or cluster, or to use as a template. - [Edit a connector configuration](#c3-edit-connector) and relaunch it. - [Pause a running connector](#c3-pause-resume-connector); resume a paused connector. - [Delete a connector](#c3-delete-connector). - View the status of connectors in Connect clusters. Control Center uses the Connect API to get information on running connectors. For more information about Connect, see [Kafka Connect](/platform/current/connect/index.html#kafka-connect). For more information about connectors, see [Kafka Connectors](/platform/current/connect/kafka_connectors.html#connectors-self-managed-cp). ## Configure Control Center for Connect clusters When you start a local Confluent Platform install using the [confluent local services start](https://docs.confluent.io/confluent-cli/current/command-reference/local/services/confluent_local_services_start.html) command from the Confluent CLI, a default Connect cluster named **connect-default** is created for you. To configure Control Center to communicate with your own Connect clusters in a production environment, set the `confluent.controlcenter.connect..cluster=:` option in the Control Center configuration file (`control-center.properties`). This option is required for connecting Control Center to your Connect clusters. For the full configuration property definition, see the Connect Confluent Control Center [configuration properties](installation/configuration.md#c3-connect-cluster-setting) in the [Control Center Configuration Reference for Confluent Platform](installation/configuration.md#controlcenter-configuration). ### Multiple Kafka Connect clusters You can configure Confluent Control Center to manage multiple Kafka Connect clusters. For example, if you have a development Connect cluster and a production Connect cluster, you could add two `confluent.controlcenter.connect..cluster=:` references to the Control Center configuration properties file (`control-center.properties`). An example of this is shown below: ```text ############################# Control Center Settings ############################# ... # A comma separated list of Connect host names confluent.controlcenter.connect.development.cluster=http://connect-dev:8083 confluent.controlcenter.connect.production.cluster=https://connect-prod:8443 ... ``` For the full configuration property definition, see the Connect Confluent Control Center [configuration properties](installation/configuration.md#c3-connect-cluster-setting) in the [Control Center Configuration Reference for Confluent Platform](installation/configuration.md#controlcenter-configuration). ### All Connect Clusters page The **All Kafka Connect Clusters** page provides an overview of all Connect clusters. To access the **All Kafka Connect Clusters** page: - Click the **Connect** menu for a cluster. - Click the **Connect** panel on the cluster **Overview** page. Use the **All Kafka Connect Clusters** page to: - Search for a Connect cluster by its cluster name and ID. - Sort a column by clicking in the column title cell. - Navigate to the [Connectors page](#c3-connectors-page). - View a breakdown for each Connect cluster by connector status. The page shows the total number of running, degraded, failed, and paused connectors. ![image](images/c3-all-connect-clusters-page.png) ### Connectors page The **Connectors** page provides an overview of the connectors configured for a Connect cluster. To access the **Connectors** page, click the **Cluster name** link in the [All Connect Clusters page](#c3-all-connect-clusters-page) for a Connect cluster. Use the **Connectors** page to: - View a summary of connectors for the Connect cluster. - Search for a connector by its name and ID. - Filter the view of connectors by Source or Sink type. - [Add a connector (non-RBAC environment)](#c3-add-connector) by either completing the UI fields manually or [uploading a connector configuration file](#c3-upload-connector-config). - Sort a column by clicking in the column title cell. - [Add a connector (non-RBAC environment)](#c3-add-connector) to the Connect cluster. - Navigate to view connector details, [edit a connector](#c3-edit-connector), or [delete a connector](#c3-delete-connector). ![image](images/c3-connectors-page.png) The **Status**, **Name**, **Category**, **Type**, **Topics** (for sink connectors only), and **Number of Tasks** are shown for each configured connector. When reducing the number of tasks in a connector’s configuration (for example, from 32 to 30 or from 36 to 12) the **Number of Tasks** field will display similar to the following: ![image](images/c3-connector-number-of-tasks.png) ## Add a connector (non-RBAC environment) Follow these steps to configure a source or sink connector by completing the applicable UI fields. You can also add a connector by [uploading a connector configuration file](#c3-upload-connector-config). For details about connector settings common and unique to source and sink connectors, see [Configuring Connectors](/kafka-connectors/self-managed/configuring.html). This procedure is applicable to a non-RBAC workflow. For an role-based access control (RBAC) workflow, see [Add a connector (RBAC environment)](#c3-add-connector-rbac-workflow)). There are two steps (tabs) to complete in this workflow: - Set up the connection. - Test and verify. **To add a connector in a non-RBAC environment** 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. ![image](images/c3-all-connect-clusters-page.png) 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. ![image](images/c3-no-connectors.png) 3. Click **Add connector**. The Browse page for selecting connectors opens. The connectors that initially appear in this page are [bundled](/kafka-connectors/self-managed/supported.html) with Confluent Platform. To narrow the available selections, select either **Sources** or **Sinks** from the **Filter by type** menu. ![image](images/c3-connect-select-connector.png) 4. Click the tile for the connector you want to configure. The **Add Connector** page opens to the **01 Setup Connection** tab. Use the shortcut panel to the right to navigate the list of configurations. ![image](images/c3-add-connector.png) 5. Complete the fields as appropriate for the connector. Required fields are indicated with an asterisk. ### Generalized GCS Source connector configuration When configuring the [Generalized Google Cloud Storage Source connector](https://docs.confluent.io/kafka-connectors/gcs-source/current/generalized/overview.html), you won’t be able to add the `topic.regex.list` configuration parameter if the mode for the connector is set to `RESTORE_BACKUP`, which is the default mode. If you set the mode to `GENERIC`, you will see `topic.regex.list` listed as an option under **Kafka Topic Regex** in the **Topic** section. For more details about each of these parameters, see the [Generalized GCS Source Connector Configuration Properties](https://docs.confluent.io/kafka-connectors/gcs-source/current/configuration_options.html#generalized-connector-parameters) page. 6. (Optional) If there are additional properties you need to add, click **Add a property**. The **Additional Properties** dialog opens for you to enter the property name. After entering the property name, enter the value for the property. ![image](images/c3-connector-add-property.png)![image](images/c3-connector-add-prop-modal.png)![image](images/c3-connector-addl-props.png) To delete a property, click the trash icon. You can undo the operation. 7. Click **Continue**. The **02 Test and verify** page opens. (If the 02 Security page opens, see [RBAC workflow](#c3-add-connector-rbac-workflow).) ![image](images/c3-connect-download-config-link.png) 8. (Optional) Click **Download connector config file**. See [Download a connector configuration file](#c3-download-connector-config) for details. 9. Review the information and click **Launch**. The information displayed is sent to the [Connect REST API](/platform/current/connect/references/restapi.html#connect-userguide-rest). - If the configuration was successful, the connector appears in the connectors table within the [Connectors page](#c3-connectors-page). Green bars indicate the connector is running. - If the configuration was unsuccessful, the **Status** column indicates Failed. Red bars indicate the connector is not running. In the **Name** column, click the link for the connector and edit the configuration fields. Repeat the process as necessary. ## Add a connector (RBAC environment) Follow this procedure to add a connector in Control Center when RBAC is enabled. There are typically three steps (tabs) to adding a connector when RBAC is enabled: 1. Set up the connection 2. Configure security (RBAC) 3. Test and verify To use this procedure, your connector configuration must use the standard SASL/OAUTHBEARER mechanism, and your worker configuration must not contain principal credentials. Otherwise, skip this procedure (02 Security page). The SASL/OAUTHBEARER mechanism supports two types of user stores: - LDAP based - OAuth client credentials based To use the SASL/OAUTHBEARER mechanism, exclude the `producer.sasl.jaas.config` and `consumer.sasl.jaas.config` properties from your worker file. Instead, specify those properties on a per-connector basis with the `producer.override.sasl.jaas.config` and `consumer.override.sasl.jaas.config` properties. To specify these configurations in a connector, use the **Additional Properties** fields in the **Add Connector > 01 Setup Connection** tab. For instructions on adding a property, see the [additional properties step](#c3-add-connector-prop-step). If you specify these additional properties, do not enter the Connector Username and Password fields in the Add Connector page 02 Security tab. Simply click **Continue** to skip the 02 Security tab and go directly to the 03 Test and Verify tab. #### IMPORTANT If your [worker configuration](/platform/current/connect/rbac/connect-rbac-worker.html#connect-rbac-workers) does not use the SASL/OAUTHBEARER LDAP-based mechanism, do not enter the Connector Username and Password fields in the 02 Security tab; the connector configuration will not work if you do. If you configure a connector to use a different authentication mechanism, it should work as long as that mechanism is enabled on the Kafka cluster. **To add a connector in a standard RBAC environment** Complete the steps for adding a connector as described in the non-RBAC workflow, [Add a connector (non-RBAC environment)](#c3-add-connector). This procedure describes the details for the **02 Security** step in the RBAC connector workflow. 1. [Log in](security/c3-rbac-login.md#c3-rbac-login) to Control Center. 2. Follow steps 1 through 7 in [Add a connector (non-RBAC environment)](#c3-add-connector). 3. (Optional/Conditional) Take one of the following actions, depending on the user store your connector uses: - For connectors using OAuth client credentials based configs, skip this step. - For connectors using LDAP-based credentials, in the step 02 Security page, enter your credentials: - Connector Username (`principal.service.name`) - Connector Password (`principal.service.password`) ![image](images/c3-rbac-connector-step3-collapsed.png) 4. (Optional) If you do not have the information, click the **How do I get these credentials?** link. The page expands. ![image](images/c3-rbac-connector-template-upper.png)![image](images/c3-rbac-connector-template-lower.png) 1. Review the information presented on the page, including the documentation links. 2. Click **Copy to Clipboard**. Send the information to your RBAC administrator so that they can grant you the necessary privileges. 3. When you receive the information from your RBAC administrator, complete your credentials. 5. Click **Continue**. The 03 Test and verify page opens. 6. Review the information and click **Launch**. ## Download a connector configuration file Download a connector configuration file while you are [adding a connector](#c3-add-connector). After downloading the configuration file, you can upload the configuration file into another Connect cluster or use it as a template for another connector in the same or another Connect cluster. Supported formats for download: - `.json` file - `.properties` file 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. 3. Click **Add connector**. Complete the fields as described in [adding a connector](#c3-add-connector). Continue to the Test and verify page. ![image](images/c3-connect-download-config-link.png) 4. Click **Download connector config file**. The Download config file dialog opens. ![image](images/c3-connector-download-config.png) 5. Click the file type to download: **Download .JSON** or **Download .properties**. The file is downloaded into your downloads directory. When you are done downloading files, click **x** to close the download dialog. Example JSON connector config file: ```JSON { "name": "datagen-users", "connector.class": "io.confluent.kafka.connect.datagen.DatagenConnector", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "kafka.topic": "users", "max.interval": "990", "iterations": "1000000000", "quickstart": "users" } ``` Example properties connector config file: ```bash name = datagen-users connector.class = io.confluent.kafka.connect.datagen.DatagenConnector key.converter = org.apache.kafka.connect.storage.StringConverter kafka.topic = users max.interval = 990 iterations = 1000000000 quickstart = users ``` 6. Click **Launch** to launch the connector. ## Upload a connector configuration file to add a connector Upload a connector configuration file to add a connector. You can upload a connector config file from either the Browse connectors page when adding a connector, or directly from the [Connectors page](#c3-connectors-page). Supported formats for upload: - `.json` file - `.properties` file 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. ![image](images/c3-connect-upload-config-connectors.png) 3. Click **+ Upload connector config file**. 4. In the **Downloads** directory (or navigate to the directory where your config resides), select the configuration file you want to upload and click **Open**. - The Connector UI fields are populated with the entries in the configuration file. - If an error is displayed, make sure the file has valid JSON if applicable. Ensure the JSON or properties file has valid properties for the type of connector. ## Edit a connector [Pause](#c3-pause-resume-connector) the connector before making configuration changes. 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. 3. Click the **Name** column for the connector. 4. Click the **Settings** tab. To view a summary, click the **Overview** tab. 5. Make the changes you want to the connector fields. To get more information about a field, hover over the info (i) icon. **Connector class** and **Name** cannot be changed. 6. Click **Continue**. 7. (Optional) Click **Download connector config file**. See [Download a connector configuration file](#c3-download-connector-config) for details. 8. Review the information and click **Launch**. ## Pause and resume a running connector Pause a running connector if you need to troubleshoot or make configuration changes. You can pause or resume a connector from either the **Overview** or the **Settings** pages. The **Overview** and **Connectors** pages display `Paused` in the **Connectors** summary panel and the **Status** column for a paused connector. See the following example from the **Connectors** page: ![image](images/c3-paused-status-connectors-page.png) 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. 3. Select a running connector, the select **Pause**. To allow the connector to run again, click **Resume** . ## Restart a connector Restart a connector if you need to reboot the connector and all its tasks. You can restart the connector by clicking **Restart** on the **Overview** page. ![image](images/c3-connect-restart.png) 1. Select a cluster from the navigation bar and click the **Connect** menu. The [All Connect Clusters page](#c3-all-connect-clusters-page) opens. 2. In the **Cluster name** column, click the **connect-default** link (or the link for your Connect cluster). The [Connectors page](#c3-connectors-page) opens. 3. Select a connector to restart and click **Restart** to restart the connector. 4. Verify the tasks have restarted and transition to the **Runnning** state. ### Restart failed components In the event that some or all of a connector’s tasks have failed, and/or the connector has failed, Control Center will display a message in the **Overview** page. If there is a failure, Control Center will display a message notifying one of the following: - Tasks have failed. To restart the failed tasks, click **Restart Failed Tasks** to reboot only the failed tasks as shown in the following example: ![image](images/c3-connect-failed-tasks.png) - The connector has failed. To restart only the failed connector, click **Restart Failed Connector**. - The connector and tasks have failed. To restart the failed connector and tasks, click **Restart Failed Connector And Tasks**. Be sure to verify the failed components have restarted and transitioned to the **Running** state. ## Delete a connector If you no longer need a connector, you can delete it. If the connector is running, deleting it automatically stops the connector. Optionally, [pause](#c3-pause-resume-connector) the connector before deleting it. **To delete a connector** 1. From the **Home** page, click the **Connectors** tile in the **Your resource overview** panel. The Connectors dialog opens. ![Home page with the Connectors tile highlighted in the Your resource overview panel](images/c3-delete-connector-home-connectors.png) 2. In the Connectors dialog, click the connector you want to delete. The connector **Overview** page opens. ![Connectors dialog with a connector selected for deletion](images/c3-delete-connector-modal.png) 3. On the connector page, click the **Settings** tab. ![Connector Settings tab highlighted on the connector overview page](images/c3-delete-connector-settings.png) 4. On the **Settings** tab, click **Delete connector**. You are prompted to confirm the deletion. 5. Click **Confirm**.