Important

You are viewing documentation for an older version of Confluent Platform. For the latest, click here.

Kafka Connect SNMP Trap Source Connector for Confluent Platform¶

The SNMP Trap Source Connector is used to receive data (SNMP traps) from devices through SNMP and convert the trap messages into Kafka records.

Simple Network Management Protocol (SNMP) is an application-layer protocol used to manage and monitor network devices and their functions. SNMP provides a common language for network devices to relay management information within single and multi-vendor environments in a local area network (LAN) or wide area network (WAN). The most recent iteration of SNMP, version 3, includes security enhancements that authenticate and encrypt SNMP messages as well as protect packets during transit.

The connector should be configured to listen to PDUs (a SNMP protocol data unit) from an SNMP device.

The connector interprets only those PDUs that are of type trap.

This connector supports both SNMP V2 and V3 protocols and it can be configured using snmp.v3.enabled parameter. If the connector is configured to listen an SNMP V3, then the following SNMPv3 options need to be provided.

Authentication Protocol: Specifies the SNMPv3 authentication protocol or type and its value can be any of [md5, sha, hmac128sha224, hmac192sha256, hmac256sha384, hmac384sha512].
Authentication Password: Specifies the SNMPv3 authentication passphrase or password.
Privacy/Encryption Protocol: Specifies the SNMPv3 privacy/encryption protocol and its value can be any of [des, 3des, aes, aes128, aes192, aes256].
Privacy Password: Specifies the SNMPv3 encryption password.
Security Name: Specifies the SNMPv3 security name or user name.

Prerequisites¶

The following are required to run the Kafka Connect SNMP Trap Source Connector:

Kafka Broker: Confluent Platform 3.3.0 or above, or Kafka 0.11.0 or above
Connect: Confluent Platform 4.0.0 or above, or Kafka 1.0.0 or above
Java: 1.8

Limitations¶

The connector supports only one task because the connector will open a listener port based on the configurations of snmp.listen.protocol, snmp.listen.address, and snmp.listen.port. For multiple tasks, multiple ports need to be opened. Currently this is not supported.
The authentication protocol with sha2 is not currently supported.

Install the SNMP Trap Source Connector¶

You can install this connector by using the Confluent Hub client (recommended) or you can manually download the ZIP file.

Install the connector using Confluent Hub¶

Prerequisite: Confluent Hub Client must be installed. This is installed by default with Confluent Enterprise.

Navigate to your Confluent Platform installation directory and run the following command to install the latest (latest) connector version. The connector must be installed on every machine where Connect will run.

confluent-hub install confluentinc/kafka-connect-snmp-trap:latest

You can install a specific version by replacing latest with a version number. For example:

confluent-hub install confluentinc/kafka-connect-snmp-trap:1.0.0-preview

Install Connector Manually¶

Download and extract the ZIP file for your connector and then follow the manual connector installation instructions.

License¶

You can use this connector for a 30-day trial period without a license key.

After 30 days, this connector is available under a Confluent enterprise license. Confluent issues enterprise license keys to subscribers, along with providing enterprise-level support for Confluent Platform and your connectors. If you are a subscriber, please contact Confluent Support at support@confluent.io for more information.

See Confluent Platform license for license properties and License topic configuration for information about the license topic.

Quick Start¶

The following uses the SnmpTrapSourceConnector to receive data (SNMP traps) from devices through SNMP and send them to the Kafka topic.

Install the connector through the Confluent Hub Client.
```
# run from your Confluent Platform installation directory
confluent-hub install confluentinc/kafka-connect-snmp-trap:latest
```
Tip

By default, the plugin is installed into share/confluent-hub-components and the directory is added to the plugin path. If this is the first connector you have installed, you may need to restart the Connect server for the plugin path change to take effect.

Property-based example¶

Create a snmp-trap-source-quickstart.properties file with the following contents or use the snmp-trap-source-quickstart.properties. This configuration is used typically along with standalone workers.:
```
name=SnmpTrapSourceConnector
tasks.max=1
connector.class=io.confluent.connect.snmp.SnmpTrapSourceConnector
snmp.v3.enabled=true
kafka.topic=snmp-kafka-topic
snmp.batch.size=50
snmp.listen.address=<ip-address to listen trap from>
snmp.listen.port=<port to listen trap from>
auth.password=<Auth-Password>
privacy.password=<privacy-password>
security.name=<security-name>
confluent.topic.bootstrap.servers=localhost:9092
confluent.topic.replication.factor=1
confluent.license=
```
Tip

The following defines the Confluent license stored in Kafka, so we need the Kafka bootstrap addresses. The replication.factor may not be larger than the number of Kafka brokers in the destination cluster, so here we set this to ‘1’ for demonstration purposes. Always use at least ‘3’ in production configurations.
Load the SNMP Trap Source Connector.

Caution

You must include a double dash (--) between the connector name and your flag. For more information, see this post.

Tip

The command syntax for the Confluent CLI development commands changed in 5.3.0. These commands have been moved to confluent local. For example, the syntax for confluent start is now confluent local start. For more information, see confluent local.
```
confluent local load snmp-trap-source -- -d snmp-trap-source-quickstart.properties
```
Important

Don’t use the Confluent CLI in production environments.
Confirm that the connector is in a RUNNING state.
```
confluent local status snmp-trap-source
```
The SNMP device should be running and generating PDUs. The connector will listen and push PDUs of type trap to a Kafka topic.

Confirm that the messages are being sent to Kafka.

kafka-avro-console-consumer --bootstrap-server localhost:9092 --property schema.registry.url=http://localhost:8081 --topic snmp-kafka-topic --from-beginning

A sample SNMP PDU of type trap might look like this for sysDescr Oid. Refer - https://www.alvestrand.no/objectid/1.3.6.1.2.1.1.1.html:

TRAP[
      {
        contextEngineID=80:00:00:59:03:78:d2:94:b8:9f:95,
        contextName=
      },
      requestID=2058388122,
      errorStatus=0,
      errorIndex=0,
      VBS[
           1.3.6.1.2.1.1.1.0 = 24-Port Gigabit Smart Switch with PoE and 4 SFP uplinks
         ]
    ]

Data in Kafka topic:

{
  "peerAddress":"127.0.0.1/55159",
  "securityName":"admin",
  "variables":[
    {
      "oid":"1.3.6.1.2.1.1.1.0",
      "type":"octetString",
      "counter32":null,
      "counter64":null,
      "gauge32":null,
      "integer":null,
      "ipaddress":null,
      "null":null,
      "objectIdentifier":null,
      "octetString":null,
      "opaque":null,
      "timeticks":null,
      "metadata":{
        "string":"24-Port Gigabit Smart Switch with PoE and 4 SFP uplinks"
      }
   }]
}

REST-based example¶

Use this setting with distributed workers. Write the following JSON to snmp-trap-source-config.json, configure all of the required values, and use the following command to post the configuration to one of the distributed connect workers. For more information, see the Kafka Connect REST API.

 {
 "name": "SnmpTrapSourceConnector",
 "config": {
     "name":"SnmpTrapSourceConnector",
     "connector.class":"io.confluent.connect.snmp.SnmpTrapSourceConnector",
     "tasks.max":"1",
     "kafka.topic":"snmp-kafka-topic",
     "snmp.v3.enabled":"true",
     "snmp.batch.size":"50",
     "snmp.listen.address":"<ip-address to listen trap from>",
     "snmp.listen.port":"<port to listen trap from>",
     "auth.password":"<Auth-Password>",
     "privacy.password":"<privacy-password>",
     "security.name":"<security-name>",
     "confluent.topic.bootstrap.servers":"localhost:9092",
     "confluent.topic.replication.factor":"1"
 }
}

Use curl to post the configuration to one of the Kafka Connect Workers. Change http://localhost:8083/ the endpoint of one of your Kafka Connect worker(s).

curl -sS -X POST -H 'Content-Type: application/json' --data @snmp-trap-source-config.json http://localhost:8083/connectors

Use the following command to update the configuration of existing connector.

curl -s -X PUT -H 'Content-Type: application/json' --data @snmp-trap-source-config.json http://localhost:8083/connectors/snmpTrapSourceConnector/config

Check that the connector started successfully. Review the Connect worker’s log by entering the following:

confluent local log connect

The SNMP device should be running and generating PDUs. The connector will listen and push PDUs of type trap to Kafka topic.

Record Schema¶

The source connector creates records in the following format:

Key Schema¶

The Key is a struct with the following fields:

Field Name	Schema Type	Optional?	Description
peerAddress	string	mandatory	Remote address of the host sending the trap.

Value Schema¶

The Value is a struct with the following fields:

Field Name	Schema Type	Optional?	Description
peerAddress	string	mandatory	Remote address of the host sending the trap.
securityName	string	mandatory	Community name the event was sent to.
variables	array of struct	mandatory	Variables for this trap.

Variable Schema¶

The Variable is a struct with the following fields:

Field Name	Schema Type	Optional?	Description
oid	string	mandatory	OID
type	string	mandatory	Syntax type for variable binding.
counter32	int32	Optional	Counter32 value. Ranges from 0 to 4294967295.
counter64	int64	Optional	Counter64 value. Ranges from 0 to 18446744073709551615.
gauge32	int32	Optional	Gauge32 value. Ranges from 0 to 4294967295.
integer	int32	Optional	Integer value.
ipaddress	string	Optional	IpAddress value.
null	string	Optional	null value.
objectIdentifier	string	Optional	OID value.
octetString	string	Optional	Octet string value.
opaque	string	Optional	opaque value.
timeticks	int32	Optional	timeticks value. Ranges from 0 to 4294967295.
metadata	string	Optional	metadata field that contains value object of variables.