Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Configuration Properties

To use this connector, specify the name of the connector class in the connector.class configuration property.

connector.class=com.splunk.kafka.connect.SplunkSinkConnector

Connector-specific configuration properties are described below.

splunk.hec.token

Splunk Http Event Collector (HEC) token.

  • Type: password
  • Importance: high
splunk.hec.uri

Splunk HEC URIs. Either a list of FQDNs or IPs of all Splunk indexers, separated with a ,, or a load balancer. The connector load balances to indexers using round robin. Splunk Connector round robins to this list of indexers: https://hec1.splunk.com:8088,https://hec2.splunk.com:8088,https://hec3.splunk.com:8088

  • Type: string
  • Importance: high
splunk.hec.ssl.trust.store.password

Password for the trust store.

  • Type: password
  • Default: [hidden]
  • Importance: high
splunk.hec.ssl.trust.store.path

Path on the local disk to the certificate trust store.

  • Type: string
  • Default: “”
  • Importance: high
splunk.hec.total.channels

Total HEC Channels used to post events to Splunk. When enabling HEC ACK, setting to the same or 2X number of indexers is generally good.

  • Type: int
  • Default: 2
  • Importance: high
splunk.header.custom

This setting enables looking for Record headers with these values and adding them to each event if present. Multiple headers are separated by comma. For example: custom_header_1,custom_header_2,custom_header_3.

  • Type: string
  • Default: “”
  • Importance: medium
splunk.header.host

Header to use for Splunk Header Host.

  • Type: string
  • Default: splunk.header.host
  • Importance: medium
splunk.header.index

Header to use for Splunk Header Index.

  • Type: string
  • Default: splunk.header.index
  • Importance: medium
splunk.header.source

Header to use for Splunk Header Source.

  • Type: string
  • Default: splunk.header.source
  • Importance: medium
splunk.header.sourcetype

Header to use for Splunk Header Sourcetype.

  • Type: string
  • Default: splunk.header.sourcetype
  • Importance: medium
splunk.header.support

This setting enables Kafka Record headers to be used for meta data override.

  • Type: boolean
  • Default: false
  • Importance: medium
splunk.hec.ack.enabled

When set to true, the connector polls event ACKs for POST events before check-pointing the Kafka offsets. This setting enables guaranteed delivery and prevents data loss but may result in lower overall throughput.

  • Type: boolean
  • Default: false
  • Importance: medium
splunk.hec.ack.poll.interval

Controls the event ACKs polling interval. This setting is only applicable when splunk.hec.ack.enabled is set to true. By default, this setting is 10 seconds.

  • Type: int
  • Default: 10
  • Importance: medium
splunk.hec.ack.poll.threads

Controls how many threads should be spawned to poll event ACKs. This setting is used for performance tuning and is only applicable when splunk.hec.ack.enabled is set to true. By default, this is set to 2.

  • Type: int
  • Default: 2
  • Importance: medium
splunk.hec.backoff.threshhold.seconds

The amount of time the connector waits before attempting to resend failed events to Splunk.

  • Type: int
  • Default: 60
  • Importance: medium
splunk.hec.event.timeout

This setting determines how long the connector will wait for an event to be acknowledged before timing out and attempting to resend the event. This setting is applicable when splunk.hec.ack.enabled is set to true. By default, this is set to 300 seconds.

  • Type: int
  • Default: 300
  • Importance: medium
splunk.hec.http.keepalive

This setting enables or disables HTTP connection keep-alive. By default, this is set to true.

  • Type: boolean
  • Default: true
  • Importance: medium
splunk.hec.max.batch.size

The maximum batch size when posting events to Splunk. The size is the actual number of Kafka records, not the byte size. By default, this is set to 500.

  • Type: int
  • Default: 500
  • Importance: medium
splunk.hec.max.http.connection.per.channel

The maximum number of HTTP connections pooled for one HEC Channel when posting events to Splunk.

  • Type: int
  • Default: 2
  • Importance: medium
splunk.hec.max.outstanding.events

The maximum amount of unacknowledged events kept in memory by the connector. When the threshold is exceeded, a back pressure event is triggered to slow the collection of events. By default, this threshold is set to 1000000 events.

  • Type: int
  • Default: 1000000
  • Importance: medium
splunk.hec.max.retries

The maximum number of retries for a failed batch before the task is killed. When set to -1 (the default) the connector retries indefinitely.

  • Type: int
  • Default: -1
  • Importance: medium
splunk.hec.raw

Enable this setting to ingest data using the /raw HEC endpoint instead of the /event HEC endpoint. By default, this setting is false and the /event HEC endpoint is used.

  • Type: boolean
  • Default: false
  • Importance: medium
splunk.hec.raw.line.breaker

This setting is used to specify a custom line breaker to help Splunk separate events correctly. For example, you can specify ##### as a special line breaker and Splunk will split events on those characters. This is only applicable when splunk.hec.raw is set to true.

  • Type: string
  • Default: “”
  • Importance: medium
splunk.hec.ssl.validate.certs

Enables or disables HTTPS certification validation. By default, this is set to true.

  • Type: boolean
  • Default: true
  • Importance: medium
splunk.hec.use.record.timestamp

When set to true, the timestamp is retrieved from the Kafka record and passed to Splunk as a HEC meta-data override. This indexes events in Splunk with the record timestamp. By default, this is set to true.

  • Type: boolean
  • Default: true
  • Importance: medium
splunk.indexes

Splunk index names for Kafka topic data separated by a comma for multiple topics to indexers. Example: “prod-index1,prod-index2,prod-index3”

  • Type: string
  • Default: “”
  • Importance: medium
splunk.sources

Splunk event source metadata for Kafka topic data. The same configuration rules as indexes apply. If unconfigured, the default source binds to the HEC token.

  • Type: string
  • Default: “”
  • Importance: medium
splunk.sourcetypes

Splunk event source type metadata for Kafka topic data. The same configuration rules as indexes apply here. If unconfigured, the default source binds to the HEC token. Only configure this when using the JSON Event endpoint (splunk.hec.raw=false).

  • Type: string
  • Default: “”
  • Importance: medium
splunk.hec.json.event.enrichment

This setting is used to enrich raw data with extra metadata fields. It contains a list of key value pairs separated by ,. The configured enrichment metadata will be indexed along with raw event data by Splunk. This is only applicable to the /event HEC endpoint (splunk.hec.raw=false). Data enrichment for the /event HEC endpoint is only available in Splunk Enterprise 6.5 and above. By default, this setting is empty.

  • Type: string
  • Default: “”
  • Importance: low
splunk.hec.json.event.formatted

This setting ensures events are pre-formatted into the proper HEC JSON format, have meta-data and event data so that they are indexed correctly by Splunk.

  • Type: boolean
  • Default: false
  • Importance: low
splunk.hec.socket.timeout

The maximum duration in seconds to read/write data to network before an internal TCP Socket timeout occurs. By default, this is set to 60 seconds.

  • Type: int
  • Default: 60
  • Importance: low
splunk.hec.threads

Controls how many threads are spawned to do data injection via HEC in a single connector task. By default, this is set to 1.

  • Type: int
  • Default: 1
  • Importance: low
splunk.hec.track.data

When set to true, data loss and data injection latency metadata will be indexed along with raw data. This setting only works in conjunction with /event HEC endpoint (splunk.hec.raw=false).

  • Type: boolean
  • Default: false
  • Importance: low