Configuration Reference for Splunk Sink Connector for Confluent Platform

To use this connector, specify the name of the connector class in the connector.class configuration property.

connector.class=com.splunk.kafka.connect.SplunkSinkConnector

Connector-specific configuration properties are described below.

Note

These are properties for the self-managed connector. If you are using Confluent Cloud, see Splunk Sink Connector for Confluent Cloud.

splunk.hec.token

Splunk Http Event Collector (HEC) token.

• Type: password

• Importance: high

splunk.hec.uri

Splunk HEC URIs. Either a list of FQDNs or IPs of all Splunk indexers, separated with a ,, or a load balancer. The connector load balances to indexers using round robin. Splunk Connector round robins to this list of indexers: https://hec1.splunk.com:8088,https://hec2.splunk.com:8088,https://hec3.splunk.com:8088

• Type: string

• Importance: high

splunk.hec.ssl.trust.store.password

Password for the trust store.

• Type: password

• Default: [hidden]

• Importance: high

splunk.hec.ssl.trust.store.path

Path on the local disk to the certificate trust store.

• Type: string

• Default: “”

• Importance: high

splunk.hec.total.channels

Total HEC Channels used to post events to Splunk. When enabling HEC ACK, setting to the same or 2X number of indexers is generally good.

• Type: int

• Default: 2

• Importance: high

splunk.header.custom

This setting enables looking for Record headers with these values and adding them to each event if present. Multiple headers are separated by comma. For example: custom_header_1,custom_header_2,custom_header_3.

• Type: string

• Default: “”

• Importance: medium

splunk.header.host

Header to use for Splunk Header Host.

• Type: string

• Default: splunk.header.host

• Importance: medium

splunk.header.index

Header to use for Splunk Header Index.

• Type: string

• Default: splunk.header.index

• Importance: medium

splunk.header.source

Header to use for Splunk Header Source.

• Type: string

• Default: splunk.header.source

• Importance: medium

splunk.header.sourcetype

Header to use for Splunk Header Sourcetype.

• Type: string

• Default: splunk.header.sourcetype

• Importance: medium

splunk.header.support

This setting enables Kafka Record headers to be used for meta data override.

• Type: boolean

• Default: false

• Importance: medium

splunk.hec.ack.enabled

When set to true, the connector polls event ACKs for POST events before check-pointing the Kafka offsets. This setting enables guaranteed delivery and prevents data loss but may result in lower overall throughput.

• Type: boolean

• Default: false

• Importance: medium

splunk.hec.ack.poll.interval

Controls the event ACKs polling interval. This setting is only applicable when splunk.hec.ack.enabled is set to true. By default, this setting is 10 seconds.

• Type: int

• Default: 10

• Importance: medium

splunk.hec.ack.poll.threads

Controls how many threads should be spawned to poll event ACKs. This setting is used for performance tuning and is only applicable when splunk.hec.ack.enabled is set to true. By default, this is set to 2.

• Type: int

• Default: 2

• Importance: medium

splunk.hec.backoff.threshhold.seconds

The amount of time the connector waits before attempting to resend failed events to Splunk.

• Type: int

• Default: 60

• Importance: medium

splunk.hec.event.timeout

This setting determines how long the connector will wait for an event to be acknowledged before timing out and attempting to resend the event. This setting is applicable when splunk.hec.ack.enabled is set to true. By default, this is set to 300 seconds.

• Type: int

• Default: 300

• Importance: medium

splunk.hec.http.keepalive

This setting enables or disables HTTP connection keep-alive. By default, this is set to true.

• Type: boolean

• Default: true

• Importance: medium

splunk.hec.max.batch.size

The maximum batch size when posting events to Splunk. The size is the actual number of Kafka records, not the byte size. By default, this is set to 500.

• Type: int

• Default: 500

• Importance: medium

splunk.hec.max.http.connection.per.channel

The maximum number of HTTP connections pooled for one HEC Channel when posting events to Splunk.

• Type: int

• Default: 2

• Importance: medium

splunk.hec.max.outstanding.events

The maximum amount of unacknowledged events kept in memory by the connector. When the threshold is exceeded, a back pressure event is triggered to slow the collection of events. By default, this threshold is set to 1000000 events.

• Type: int

• Default: 1000000

• Importance: medium

splunk.hec.max.retries

The maximum number of retries for a failed batch before the task is killed. When set to -1 (the default) the connector retries indefinitely.

• Type: int

• Default: -1

• Importance: medium

splunk.hec.raw

Enable this setting to ingest data using the /raw HEC endpoint instead of the /event HEC endpoint. By default, this setting is false and the /event HEC endpoint is used.

• Type: boolean

• Default: false

• Importance: medium

splunk.hec.raw.line.breaker

This setting is used to specify a custom line breaker to help Splunk separate events correctly. For example, you can specify ##### as a special line breaker and Splunk will split events on those characters. This is only applicable when splunk.hec.raw is set to true.

• Type: string

• Default: “”

• Importance: medium

splunk.hec.ssl.validate.certs

Enables or disables HTTPS certification validation. By default, this is set to true.

• Type: boolean

• Default: true

• Importance: medium

splunk.hec.use.record.timestamp

When set to true, the timestamp is retrieved from the Kafka record and passed to Splunk as a HEC meta-data override. This indexes events in Splunk with the record timestamp. By default, this is set to true.

• Type: boolean

• Default: true

• Importance: medium

splunk.indexes

Splunk index names for Kafka topic data separated by a comma for multiple topics to indexers. Example: “prod-index1,prod-index2,prod-index3”

• Type: string

• Default: “”

• Importance: medium

splunk.sources

Splunk event source metadata for Kafka topic data. The same configuration rules as indexes apply. If unconfigured, the default source binds to the HEC token.

• Type: string

• Default: “”

• Importance: medium

splunk.sourcetypes

Splunk event source type metadata for Kafka topic data. The same configuration rules as indexes apply here. If unconfigured, the default source binds to the HEC token. Only configure this when using the JSON Event endpoint (splunk.hec.raw=false).

• Type: string

• Default: “”

• Importance: medium

splunk.hec.json.event.enrichment

This setting is used to enrich raw data with extra metadata fields. It contains a list of key value pairs separated by ,. The configured enrichment metadata will be indexed along with raw event data by Splunk. This is only applicable to the /event HEC endpoint (splunk.hec.raw=false). Data enrichment for the /event HEC endpoint is only available in Splunk Enterprise 6.5 and above. By default, this setting is empty.

• Type: string

• Default: “”

• Importance: low

splunk.hec.json.event.formatted

This setting ensures events are preformatted into the proper HEC JSON format and have metadata and event data so that they are indexed correctly by Splunk. Set this property to true for events that are already in HEC format.

• Type: boolean

• Default: false

• Importance: low

splunk.hec.socket.timeout

The maximum duration in seconds to read/write data to network before an internal TCP Socket timeout occurs. By default, this is set to 60 seconds.

• Type: int

• Default: 60

• Importance: low

splunk.hec.threads

Controls how many threads are spawned to perform data injection through HEC in a single connector task.

• Type: int

• Default: 1

• Importance: low

splunk.hec.track.data

When set to true, data loss and data injection latency metadata will be indexed along with raw data. This setting only works in conjunction with /event HEC endpoint (splunk.hec.raw=false).

• Type: boolean

• Default: false

• Importance: low