Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make network_direction, registered_domain and convert processors compatible with ES older than 7.13.0 #26676

Merged

Conversation

adriansr
Copy link
Contributor

@adriansr adriansr commented Jul 1, 2021

What does this PR do?

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

  • Replaces usages of convert processor using type: ip with an equivalent grok expression. Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
  • Removes the network_direction processor.
  • Removes the registered_domain processor.

Why is it important?

To ensure modules work with versions older than 7.13.0.

Checklist

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 1, 2021
@adriansr adriansr added backport-v7.13.0 Automated backport with mergify backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify bug Filebeat Filebeat review Team:Security-External Integrations labels Jul 1, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 1, 2021
@adriansr adriansr changed the title Module compat 7x convert network dir Make network_direction and convert processors compatible with ES older than 7.13.0 Jul 1, 2021
@adriansr adriansr requested a review from andrewkroh July 1, 2021 16:35
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewkroh
Copy link
Member

Would you mind adding a dropProcessor case for registered_domain which was introduced in 7.13.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jul 1, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #26676 updated

  • Start Time: 2021-07-01T20:16:42.684+0000

  • Duration: 97 min 53 sec

  • Commit: 1a1ffa1

Test stats 🧪

Test Results
Failed 0
Passed 14209
Skipped 2312
Total 16521

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 14209
Skipped 2312
Total 16521

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you!

@adriansr adriansr changed the title Make network_direction and convert processors compatible with ES older than 7.13.0 Make network_direction, registered_domain and convert processors compatible with ES older than 7.13.0 Jul 2, 2021
@adriansr adriansr merged commit 65d2193 into elastic:master Jul 2, 2021
mergify bot pushed a commit that referenced this pull request Jul 2, 2021
…atible with ES older than 7.13.0 (#26676)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)
mergify bot pushed a commit that referenced this pull request Jul 2, 2021
…atible with ES older than 7.13.0 (#26676)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)
mergify bot pushed a commit that referenced this pull request Jul 2, 2021
…atible with ES older than 7.13.0 (#26676)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)
adriansr added a commit that referenced this pull request Jul 2, 2021
…atible with ES older than 7.13.0 (#26676) (#26693)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
adriansr added a commit that referenced this pull request Jul 2, 2021
…atible with ES older than 7.13.0 (#26676) (#26691)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
adriansr added a commit that referenced this pull request Jul 5, 2021
… convert processors compatible with ES older than 7.13.0 (#26692)

Adds three new Filebeat fileset compatibility tweaks to support Elasticsearch versions before 7.13.0:

- Replaces usages of convert processor using type: ip with an equivalent grok expression.
  Convert to ip type is used to make a conditional field copy if the source field is a valid IP address.
- Removes the network_direction processor.
- Removes the registered_domain processor.

(cherry picked from commit 65d2193)

* Fix changelog

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
v1v added a commit to v1v/beats that referenced this pull request Jul 5, 2021
…stage-failed-within-same-build

* upstream/master: (36 commits)
  Revert "[CI] fight the flakiness with some retry option in the CI only for the Pull Requests (elastic#26617)" (elastic#26704)
  Packaging: linux/armv7 is not supported (elastic#26706)
  Cyberarkpas: Link to official docs on how to setup TLS (elastic#26614)
  Make network_direction, registered_domain and convert processors compatible with ES older than 7.13.0 (elastic#26676)
  Disable armv7 packaging (elastic#26679)
  [Heartbeat] use --params flag for synthetics (elastic#26674)
  Update dependent package to avoid downloading a suspicious file (elastic#26406)
  [mergify] set title and allow bp in any direction (elastic#26648)
  Fix memory leak in SQL helper when database is not available (elastic#26607)
  [CI] fight the flakiness with some retry option in the CI only for the Pull Requests (elastic#26617)
  [mergify] automate PRs that change the backport rules (elastic#26641)
  [Metricbeat] Add Airflow module in xpack (elastic#26220)
  chore: add-backport-next (elastic#26620)
  [metricbeat] Add state_job metricset (elastic#26479)
  CI: jenkins labels are less time consuming now (elastic#26613)
  [MetricBeat] [AWS] Fix aws metric tags with resourcegroupstaggingapi paginator (elastic#26385) (elastic#26443)
  Move openmetrics module to oss (elastic#26561)
  Skip flaky test TestFilestreamMetadataUpdatedOnRename (elastic#26609)
  [filebeat][fortinet] Use default add_locale for fortinet.firewall (elastic#26524)
  Enroll proxy settings (elastic#26514)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.13.0 Automated backport with mergify backport-v7.14.0 Automated backport with mergify backport-v7.15.0 Automated backport with mergify bug Filebeat Filebeat review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants