Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Missing netflow field definitions for vendor fields #23771

Closed
andrewkroh opened this issue Jan 30, 2021 · 2 comments · Fixed by #23773
Closed

[Filebeat] Missing netflow field definitions for vendor fields #23771

andrewkroh opened this issue Jan 30, 2021 · 2 comments · Fixed by #23773

Comments

@andrewkroh
Copy link
Member

andrewkroh commented Jan 30, 2021

The vendor fields contained in https://github.com/elastic/beats/tree/master/x-pack/filebeat/input/netflow/decoder/fields are missing from the generated fields.yml. The go generate command declared here is what creates the fields.yml file for the netflow input. Perhaps the other CSV files can be appended to the list of arguments to the command?

//go:generate go run fields_gen.go -output _meta/fields.yml --column-name=2 --column-type=3 --header _meta/fields.header.yml decoder/fields/ipfix-information-elements.csv

This is where the field names are embedded into the source:

//go:generate go run gen.go -output zfields_ipfix.go -export IpfixFields --column-id=1 --column-name=2 --column-type=3 ipfix-information-elements.csv
//go:generate go run gen.go -output zfields_cert.go -export CertFields --column-pen=1 --column-id=2 --column-name=3 --column-type=4 cert_pen6871.csv
//go:generate go run gen.go -output zfields_cisco.go -export CiscoFields --column-pen=2 --column-id=3 --column-name=1 --column-type=4 cisco.csv
//go:generate go run gen.go -output zfields_assorted.go -export AssortedFields --column-pen=1 --column-id=2 --column-name=3 --column-type=4 assorted.csv

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@andrewkroh
Copy link
Member Author

andrewkroh commented Jan 30, 2021

This is needed because the tests for the netflow package in elastic/integrations are validating that all fields are declared. And some of the tests involve vendor specific fields taken from a Cisco IPFIX pcap.

Relates: elastic/integrations#377

andrewkroh added a commit to andrewkroh/beats that referenced this issue Feb 12, 2021
andrewkroh added a commit that referenced this issue Feb 16, 2021
#23773)

* Add field definitions for known Netflow/IPFIX vendor fields

* Add default_field: false to netflow

* Refactor generator to merge all fields to remove duplicates

* Remove indent flag from usage

Closes #23771
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants