You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(Please see the discussion for details and decided to make it with processor: #34062)
When monitoring/analyzing network flow, for example threat hunting security use cases, it's often required to make "joins" on network source and destination info where community ID simplifies and gives a better user experience when analyzing the data (aggregate by community ID to collect statistics, etc.)
Community ID is a single unique ID based on the network flow info. It is an additional flow identifier and doesn't replace existing flow identification mechanisms already supported by the monitors. See the specification.
Elastic (it is not a vendor specific) best practise examples
Initial implementation will be minimum configurable with KISS logic.
Mapping fields:
[IN] source ip: source.address
[IN]source port: source.port
[IN]destination ip: destination.address
[IN]destination port: destination.port
[IN]transport protocol: network.transport
[OUT] target: defaults to network.community_id - discussion/investigation required
(todo-1) double check naming convention
(todo-2) add sub-field to network field by default
(todo-3) best practise when adding fields
Validation:
Parse IP against IPv4|IPv6, error if invalid IP
If network.type is available, error if IP mismatches
Error if port is not in range of 1 ~ 65535
Error if transport protocol is not TCP|UDP|SCTP
Future enhancement
configurable source and destination attributes which useful when using after DNS resolver processor
IANA number: need a field specification
ICMP type: to support ICMP, need a field specification
ICMP code: to support ICMP, need a field specification
Example configuration for the component
processors:
communityid:
# super simple stupid config, by default uses `source.address`, `source.port`, `destination.address`, `destination.port` and `network.transport` to calculate the flow
# a target field to be assigned with calculated community ID, defaults to `network.community_id`
target: network.community_id
# a seed value to prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme. Defaults to 0 (zero).
seed: 0
# Defines how the processor handles errors. Accepts `log` or `silent`
# - `log`: Processor logs errors and continues to the next processor (default).
# - `silent`: Processor ignores errors without logging and continues to the next processor.
error_mode: log
Telemetry data types supported
traces, metrics and logs
Is this a vendor-specific component?
This is a vendor-specific component
If this is a vendor-specific component, I am a member of the OpenTelemetry organization.
If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.
The description above tells how a community ID is calculated, but it doesn't explain how the payload is transformed. Please add that information to the description.
The purpose and use-cases of the new component
(Please see the discussion for details and decided to make it with processor: #34062)
When monitoring/analyzing network flow, for example threat hunting security use cases, it's often required to make "joins" on network source and destination info where community ID simplifies and gives a better user experience when analyzing the data (aggregate by community ID to collect statistics, etc.)
Community ID is a single unique ID based on the network flow info. It is an additional flow identifier and doesn't replace existing flow identification mechanisms already supported by the monitors. See the specification.
Elastic (it is not a vendor specific) best practise examples
community_id
: https://www.elastic.co/guide/en/beats/filebeat/current/community-id.htmlInitial implementation
Initial implementation will be minimum configurable with KISS logic.
Mapping fields:
source.address
source.port
destination.address
destination.port
network.transport
network.community_id
- discussion/investigation requiredValidation:
network.type
is available, error if IP mismatchesFuture enhancement
Example configuration for the component
Telemetry data types supported
traces, metrics and logs
Is this a vendor-specific component?
Not a vendor specific.
Code Owner(s)
evan-bradley
Sponsor (optional)
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: