Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick #20908 to 7.9: [Filebeat] backwards compatibility for set processor #21084

Merged
merged 2 commits into from
Sep 15, 2020
Merged

Cherry-pick #20908 to 7.9: [Filebeat] backwards compatibility for set processor #21084

merged 2 commits into from
Sep 15, 2020

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Sep 14, 2020 •
edited by zube bot
Loading

Cherry-pick of PR #20908 to 7.9 branch. Original message:

What does this PR do?

When loading a pipeline this change checks the elasticsearch version
and if the version is less than 7.9.0 it will replace the
"ignore_empty_value" option with an equivalent if statement on the set
processor.

Why is it important?

This allows filebeat > 7.9.0 to be used with older versions of
elasticsearch. Without it the pipelines fail to load because the
option isn't supported.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

run filebeat modules enable zeek && filebeat -e setup

Logs

Error

2020-09-01T12:42:48.781-0500	ERROR	fileset/setup.go:81	Error loading pipeline: 1 error: Error loading pipeline for fileset zeek/x509: couldn't load pipeline: couldn't load json. Error: 400 Bad Request: {"error":{"root_cause [{"type":"parse_exception","reason":"processor [set] doesn't support one or more provided configuration parameters [ignore_empty_value]",
...

With fix:

2020-09-01T14:04:56.470-0500	DEBUG	[modules] fileset/pipelines.go:277     	in pipeline filebeat-8.0.0-zeek-x509-pipeline replacing unsupported 'ignore_empty_value' with if ctx?.zeek?.x509?.certificate?.subject?.state != null in set processor

@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 14, 2020
@leehinman @andrewkroh
[Filebeat] backwards compatibility for set processor (elastic#20908)
3cea2d0 
- "ignore_empty_value" option for the set processor only works on
Elasticsearch >= 7.9.0.  This change removes that option and replaces
it with an if statement if pipeline is loaded on an earlier version of
elasticsearch.

(cherry picked from commit 0032c0c)
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 14, 2020 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21084 updated]

  • Start Time: 2020-09-15T13:34:48.252+0000

  • Duration: 61 min 31 sec

Test stats 🧪

Test Results
Failed 0
Passed 5563
Skipped 824
Total 6387

leehinman
leehinman approved these changes Sep 15, 2020
View reviewed changes
Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewkroh thanks for catching this backport

@leehinman leehinman merged commit 5a06f7e into elastic:7.9 Sep 15, 2020
@zube zube bot removed the [zube]: Done label Dec 15, 2020
@andrewkroh andrewkroh deleted the backport_20908_7.9 branch January 14, 2022 14:08
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
@andrewkroh @leehinman
[Filebeat] backwards compatibility for set processor (elastic#20908) (e…
d15f604 
…lastic#21084)

- "ignore_empty_value" option for the set processor only works on
Elasticsearch >= 7.9.0.  This change removes that option and replaces
it with an if statement if pipeline is loaded on an earlier version of
elasticsearch.

(cherry picked from commit b6162c4)

Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leehinman leehinman leehinman approved these changes

Assignees
No one assigned
Labels
backport
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@andrewkroh @elasticmachine @leehinman