New component: DNS lookup processor

[kaisecheng]

The purpose and use-cases of the new component

The DNS lookup Processor is for resolving hostnames to IP addresses and vice versa.
It performs A record/CNAME record lookups to resolve hostnames to IP addresses, and PTR record lookups for reverse DNS resolution (IP to hostname). The resolved value replaces the original value.

The processor utilizes two LRU (Least Recently Used) caches with TTL (Time-to-Live): one for storing successful resolutions (hit cache) and another for storing failed lookups.

The processor can resolve hostnames/IPs using the system's default resolver, a custom hostfile, or a custom DNS server.

Example configuration for the component

procoessors:
  dnslookup:
    # A list of attributes to be resolved from hostname to IP
    resolve_attributes: []
    # A list of attributes to be reverse-looked up from IP to hostname
    reverse_attributes`: []
    # Maximum number of failed resolutions to cache. Default value is `1000`. Set to `0` to disable caching
    failed_cache_size: 1000 
    # Time-to-live (in seconds) for failed resolution cache entries. Default value is `5` seconds
    failed_cache_ttl: 5
    # Maximum number of successful resolutions to cache. Default value is `1000`. Set to `0` to disable caching 
    hit_cache_size: 1000 
    # Time-to-live (in seconds) for successful resolution cache entries. Default value is `60` seconds
    hit_cache_ttl: 60 
    # Maximum number of retry attempts for DNS lookups. Default value is `2`
    max_retries: 2 
    # Timeout (in seconds) for DNS lookups. Default value is `0.5`
    timeout: 0.5
    # Path to a custom hostfile
    hostfile: "/path/to/hosts"
    # Address of a custom DNS server
    nameserver: 8.8.8.8
    # Enable system's default resolver. Default value is `false`
    enable_system_resolver: false 
    # Defines how the processor handles errors. Valid values are:
    # - `ignore`: Processor logs errors and continues to the next processor (default).
    # - `silent`: Processor ignores errors without logging and continues to the next processor.
    # - `propagate`: Processor returns errors, resulting in the payload being dropped from the collector.
    error_mode`: ignore

Resolution Order

The resolution order depends on the specified configuration and the enable_system_resolver setting:
With enable_system_resolver set to true:

1. When both hostfile and nameserver are specified:
   Custom hostfile -> Custom DNS server -> System's default resolver

2. When only nameserver is specified:
   Custom DNS server -> System's default resolver

3. When only hostfile is specified:
   Custom hostfile -> System's default resolver

4. When neither hostfile nor nameserver are specified:
   System's default resolver

With enable_system_resolver set to false, this affects all scenarios above by removing System's default resolver from the resolution order.
The processor will fail to start and throw an error If neither hostfile nor nameserver are specified and enable_system_resolver is set to false.

Considerations

• Performance impact of DNS resolutions: If each DNS lookup takes 2 milliseconds, the maximum throughput a single processor can achieve is 500 events per second.

Telemetry data types supported

Logs, metrics, traces.

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
• If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

No response

Sponsor (optional)

No response

Additional context

No response

[strawgate]

Could we have this support multiple nameservers right away, or is the intention to add that in a later revision? Any thoughts on TCP transport?

[kaisecheng]

Could we have this support multiple nameservers

Yes, it makes sense to support multiple.

Any thoughts on TCP transport?

I think it is a nice to have feature and could be in a later version