<a id="connect-http-source-connector"></a>

# HTTP Source Connector for Confluent Platform

The Kafka Connect HTTP Source connector makes HTTP requests to an API,
processes JSON responses, and produce those responses to Kafka. You can use it to
periodically poll a JSON-based HTTP API and produce the resulting data to Kafka.

#### IMPORTANT


Effective July 6, 2025, only self-managed connector versions that meet or exceed the minimum version listed on the
[Supported Connector Versions](https://docs.confluent.io/platform/7.8/connect/supported-connector-version.html#supported-connector-versions-till-cp-7-8)
page receive support from Confluent. Older, unsupported connector versions have been removed from Confluent Marketplace and
are no longer available for download.

## Features

The HTTP Source connector for Confluent Platform includes the following features:

### At least once delivery

The connector guarantees that records are delivered
at least once to the Kafka topic.

### Offset modes

The connector supports the following modes:

- `SIMPLE_INCREMENTING`: For the first request, the connector computes the
  offset based on the initial offset set in the `http.initial.offset`
  property. For subsequent requests, the connector increments the offset by the
  number of records in the previous response. For instance, the connector
  increments the offset by one for an object type response that indicates a
  single record. For an array type response, the connector increments the offset
  by the length of the array.
- `CHAINING`: For the first request, the connector computes the offset based
  on the initial offset set in the `http.initial.offset` property. Upon
  receiving a successful response to the first request, the connector parses the
  response and generates one or more records (depending on whether the JSON
  response is an object or an array) with the offset value set to the value from
  the response at the path configured in the `http.offset.json.pointer`
  configuration property.
- `CURSOR_PAGINATION`: In this mode, the property
  `http.next.page.json.pointer` is used to configure the offset value. The
  offset for the last record in each page is set to the next page value. For
  additional information, see [Offset Mode Use Case
  Examples](/cloud/current/connectors/cc-http-source.html#cc-http-source-example-use-cases).

### Multiple tasks

The connector supports running one or more tasks. More tasks
may improve performance when multiple entities are configured.

### Output data formats

The connector supports Avro, JSON Schema, Protobuf, and
JSON (schemaless) output record value formats. Schema Registry must be enabled
to use a Schema Registry-based format (for example, Avro, JSON Schema, or
Protobuf).

### Client-side encryption

This connector supports Client-Side Field Level Encryption (CSFLE) and Client-Side Payload Encryption (CSPE). For more information, see [Manage Client-Side Encryption](https://docs.confluent.io/platform/current/connect/manage-csfle.html).

## Limitations

The HTTP Source connector for Confluent Platform includes the following limitations:

- The connector does not support APIs that rely on timestamp range-based
  queries.
- The connector cannot parse responses in any format other than JSON.

## Install the HTTP Source connector

You can install this connector by using the [confluent
connect plugin
install](https://docs.confluent.io/confluent-cli/current/command-reference/connect/plugin/confluent_connect_plugin_install.html)
command, or by manually downloading the ZIP file.

### Prerequisites

- You must install the connector on every machine where Connect will run.
- An installation of the [Confluent CLI](https://docs.confluent.io/confluent-cli/current/installing.html).
- An installation of the latest (`latest`) connector version.

### Install the connector using the Confluent CLI

To install the `latest` connector version, navigate to your Confluent Platform installation
directory and run the following command:

```bash
confluent connect plugin install confluentinc/kafka-connect-http-source:latest
```

You can install a specific version by replacing `latest` with a version number
as shown in the following example:

```bash
confluent connect plugin install confluentinc/kafka-connect-http-source:1.0.0
```

### Install the connector manually

[Download and extract the ZIP file](https://www.confluent.io/hub/confluentinc/kafka-connect-http-source) for your
connector and then follow the manual connector installation
[instructions](/kafka-connectors/self-managed/userguide.html#connect-installing-plugins).

## License

You can use this connector for a 30-day trial period without a license key.

After 30 days, you must purchase a connector subscription which includes [Confluent enterprise license](/platform/current/installation/license.html#enterprise-subscription-license) keys to subscribers, along with [enterprise-level support](https://www.confluent.io/subscription/) for Confluent Platform and your connectors. If you are a subscriber, you can contact [Confluent Support](https://support.confluent.io/) for more information.

For license properties, see [Configuration Reference for HTTP Source Connector for Confluent Platform](configuration.md#http-source-connector-configuration).

## Configuration properties

For a complete list of configuration properties for this connector, see
[Configuration Reference for HTTP Source Connector for Confluent Platform](configuration.md#http-source-connector-configuration).

<a id="http-source-connector-quickstart"></a>

## Quick start

This quick start guide will help you set up the Confluent Platform HTTP Source Connector to
consume records from an HTTP endpoint and send them to Kafka. This guide assumes
you are using a multi-tenant environment. For local testing, refer to
[Running Connect in standalone
mode](/kafka-connectors/self-managed/userguide.html#configuring-and-running-workers).

1. Install the connector using the following Confluent CLI command. Note that
   you must run this command from your Confluent Platform installation directory:
   ```shell
   confluent-hub install confluentinc/kafka-connect-http-source:latest
   ```
2. Start Confluent Platform.
   ```bash
   confluent local start
   ```
3. Check the status of all services.
   ```bash
   confluent local services status
   ```
4. Create a `http-source.json` file with the following contents:
   ```json
     {
       "name": "HttpSourceConnector",
       "config": {
         "connector.class": "io.confluent.connect.http.HttpSourceConnector",
         "topic.name.pattern": "topic-http",
         "url": "https://<endpoint>.com/api/data",
         "tasks.max": "1",
         "http.offset.mode": "SIMPLE_INCREMENTING",
         "key.converter": "org.apache.kafka.connect.json.JsonConverter",
         "value.converter": "org.apache.kafka.connect.json.JsonConverter",
         "confluent.topic.bootstrap.servers": "<server hostname>:9092",
         "confluent.license": "", // leave it empty for evaluation license
         "confluent.topic.replication.factor": "1"
     }
   }
   ```
5. Load the Http Source connector by posting configuration to Connect REST
   server. Note that you must include a double dash (`--`) between the topic
   name and your flag. For more information, see [What does “–” (double dash /
   double hyphen) mean?](https://unix.stackexchange.com/questions/11376/what-does-double-dash-mean-also-known-as-bare-double-dash).
6. Confirm the connector is in `RUNNING` state by entering the following command:
   ```shell
   confluent local services connect connector status HttpSourceConnector
   ```
7. Verify the connector is working by simulating sending data to the HTTP
   endpoint. For example, use `curl` to POST data.
   ```none
   curl -X POST \
     https://<endpoint>.com/api/data \
     -H 'Accept: application/json' \
     -H 'Authorization: Basic <token>' \
     -H 'Content-Type: application/json' \
     -H 'cache-control: no-cache' \
     -d '{"data": "This is a test"}'
   ```
8. Confirm the messages were delivered to the `topic-http` topic in Kafka.
   ```shell
   confluent local services kafka consume topic-http --from-beginning
   ```

## Related content

- [HTTP Source Connector for Confluent Cloud](/cloud/current/connectors/cc-http-source.html)
- [Message Delivery Guarantees](https://docs.confluent.io/kafka/design/delivery-semantics.html)