.. _zendesk_source_connector: Zendesk Source Connector for |cp| ================================= Zendesk Support is a system for tracking, prioritizing, and solving customer support tickets. The |kconnect-long| Zendesk Source connector copies data into |ak-tm| from various Zendesk support tables such as ``tickets``, ``ticket_audits``, ``ticket_fields``, ``groups``, ``organizations``, ``satisfaction_ratings``, and others, using the `Zendesk Support API `_. Please find the list of supported Zendesk tables in the :ref:`supported tables section `. .. _zendesk_source_connector_feature: Features -------- The Zendesk Source Connector offers the following features: * **Quick Turnaround**: The Zendesk connector ensures that data between your Zendesk Tables and corresponding |ak| topics are synced quickly, without unnecessary lag. The poll frequency on each table has been specifically configured based on the size of the table, so that larger and more dynamic tables, like ``Tickets``, are polled more frequently than the static tables like ``Organizations``. * **At Least Once Delivery**: The connector guarantees no loss of messages from Zendesk to |ak|. Messages may be reprocessed because of task failure or API limits, which may cause duplication. * **Schema Detection and Evolution**: The connector supports automatic schema detection and backward compatible schema evolution for all supported tables. * **Real-time and Historical Lookup**: The connector supports fetching all the past historical records for all tables. It can also be configured to pull in data from only a specified time in the past (see :ref:`configuration property ` ```zendesk.since``). * **Automatic Retries**: In case of a connection error between the API server and |kconnect-long|, the connector may receive a *not OK* response from the API server or no response at all. In such cases, the connector can be made robust using the automatic retry mechanism with linear backoff using :ref:`configuration properties ` ``max.retries`` and ``retry.backoff.ms``. * **Intelligent backoffs**: If there are too many requests because of `support API rate limits `_, the connector intelligently spaces out the HTTP fetch operations to ensure a smooth balance between recency, API limits, and back pressure. * **Resource Balance and throughput**: Different resources with Zendesk could have different rates of creation and updation. Such resources can be balanced among the workers, with reduced hot-spotting, by keeping the resources in configuration ``zendesk.tables`` sorted by the order of their expected cardinality. Also, the ``task.max``, ``max.in.flight.requests``, and ``max.batch.size`` configuration properties can be used to improve overall throughput. .. _zendesk_source_connector_resources: Supported Tables ------------------- The following tables from Zendesk are supported in this version of |kconnect-long| Zendesk Source connector: ``custom_roles``, ``groups``, ``group_memberships``, ``organizations``, ``organization_subscriptions``, ``organization_memberships``, ``satisfaction_ratings``, ``tickets``, ``ticket_audits``, ``ticket_fields``, ``ticket_metrics`` and ``users``. .. _zendesk_source_connector_prereqs: Prerequisites ------------- The following are required to run the |kconnect-long| Zendesk Source Connector: * |ak| Broker: |cp| 3.3.0 or above, or |ak| 0.11.0 or above * |kconnect-long|: |cp| 4.1.0 or above, or |ak| 1.1.0 or above * Java 1.8 * Zendesk API: `Support APIs `__ should be enabled for the Zendesk account. Also either ``oauth2`` or ``password`` mechanism should be enabled in the Zendesk account. For information, look at `Using the API dashboard: Enabling password or token access `__. * Zendesk account type: Certain tables, such as ``custom_roles``, can only be accessed if Zendesk Account is an ``Enterprise`` account. Refer `Custom Agent Roles `__. * Zendesk settings: Some settings may need to be enabled to ensure export is possible. Example, ``satisfaction_ratings`` can only be exported if it is enabled. Refer to `Support API: Satisfaction Ratings `__. .. _zendesk_source_connector_install: Install the Zendesk Source Connector ------------------------------------ .. include:: ../includes/connector-install.rst .. include:: ../includes/connector-install-hub.rst .. codewithvars:: bash confluent-hub install confluentinc/kafka-connect-zendesk:latest .. include:: ../includes/connector-install-version.rst .. codewithvars:: bash confluent-hub install confluentinc/kafka-connect-zendesk:1.0.1 ------------------------------ Install the connector manually ------------------------------ `Download and extract the ZIP file `__ for your connector and then follow the manual connector installation :ref:`instructions `. Configuration Properties ------------------------ For a complete list of configuration properties for this connector, see :ref:`zendesk_source_connector_config`. .. include:: ../includes/connect-to-cloud-note.rst .. _zendesk_connector_license: License ------- .. include:: ../includes/enterprise-license.rst See :ref:`zendesk-source-connector-license-config` for license properties and :ref:`zendesk_source_license-topic-configuration` for information about the license topic. .. _zendesk_source_connector_quickstart: Quick Start ----------- Prerequisite `Zendesk Developer Account `__ In this quick start guide, the Zendesk Connector is used to consume records from a Zendesk resource called ``tickets`` and send the records to a |ak| topic named ``ZD_tickets``. #. Install the connector through the `Confluent Hub Client `_. .. codewithvars:: bash # run from your confluent platform installation directory confluent-hub install confluentinc/kafka-connect-zendesk:latest #. Start the |cp|. .. include:: ../../includes/cli-new.rst .. codewithvars:: bash |confluent_start| #. Check the status of all services. .. codewithvars:: bash |confluent_status| #. Configure your connector by first creating a JSON file named ``zendesk.json`` with the following properties. .. codewithvars:: bash // substitute <> with your config { "name": "ZendeskConnector", "config": { "connector.class": "io.confluent.connect.zendesk.ZendeskSourceConnector", "key.converter": "org.apache.kafka.connect.storage.StringConverter", "value.converter": "org.apache.kafka.connect.json.JsonConverter", "value.converter.schemas.enable": "false", "confluent.topic.bootstrap.servers": "127.0.0.1:9092", "confluent.topic.replication.factor": 1, "confluent.license": "", // leave it empty for evaluation license "tasks.max": 1, "poll.interval.ms": 1000, "topic.name.pattern": "ZD_${entityName}", "zendesk.auth.type": "basic", "zendesk.url": "https://.zendesk.com", "zendesk.user": "", "zendesk.password": "", "zendesk.tables": "tickets", "zendesk.since": "2019-08-01" } } #. Start the Zendesk Source connector by loading the connector's configuration with the following command: .. include:: ../../includes/confluent-local-consume-limit.rst .. codewithvars:: bash |confluent_load| zendesk|dash| -d zendesk.json #. Confirm that the connector is in a ``RUNNING`` state. .. codewithvars:: bash |confluent_status| ZendeskConnector #. Create one ticket record using Zendesk API as follows. .. codewithvars:: bash curl https://{subdomain}.zendesk.com/api/v2/tickets.json \ -d '{"ticket": {"subject": "My printer is on fire!", "comment": { "body": "The smoke is very colorful." }}}' \ -H "Content-Type: application/json" -v -u {email_address}:{password} -X POST #. Confirm the messages were delivered to the ``ZD_tickets`` topic in |ak|. Note, it may take a minute before the record populates the topic. .. codewithvars:: bash |confluent_consume| ZD_tickets|dash| --from-beginning .. _zendesk_source_connector_license: Additional Documentation ------------------------ .. toctree:: :maxdepth: 1 connector_config changelog