Hostname Resolver Transformation

It is often useful to rekey the Apache Kafka® Source Records host field’s value as an IP address into a resolved hostname for that IP address. The Hostname Resolver Transformation makes this easy. The transformation will perform a reverse DNS lookup to replace IP addresses with hostnames. For example, 127.0.0.1 will be replaced with localhost on most machines.

For example, use the transformation by including the following lines in your connector configuration:

transforms=hostname
transforms.hostname.type=io.confluent.connect.syslog.HostnameResolverTransformation

Configuration options

field.name

The name of the field that contains the ip address of the remote host.

  • Type: string
  • Default: remoteAddress
  • Importance: medium
dns.cache.stats.secs

The amount of time in seconds to dump the cache statistics for the DNS cache.

  • Type: int
  • Default: 300
  • Valid Values: [1,…]
  • Importance: low
dns.cache.ttl.secs

The amount of time to cache a DNS entry for in seconds.

  • Type: int
  • Default: 300
  • Valid Values: [1,…]
  • Importance: low
dns.query.timeout.secs

The amount of time in seconds before the query is timed out.

  • Type: int
  • Default: 15
  • Valid Values: [1,…]
  • Importance: low

FieldValueIsIP predicate

When running the Confluent-provided *.HostnameResolverTransformation, the transform is run on every single Syslog message that flows through the connector. Even though DNS caching is available, this still results in unwanted reverse DNS and cache lookups on Syslog messages that are already parsed and contain a hostname. At scale, this degrades the throughput of the Syslog connector and requires additional Connect workers to be running to support data volumes while using the transformation.

To workaround this issue, configure a custom transform predicate, FieldValueIsIP, which is intended to be used with *.HostnameResolverTransformation. Use the FieldValueIsIP predicate to only run the hostname transformation when the host is an unresolved IP address. This reduces the amount of reverse DNS lookups by skipping data that the hostname has already found inside the Syslog message. For more details about the FieldValueIsIP predicate and an example of how to use it, see the Predicate - FieldValueIsIP topic on GitHub.