Fix rfc5464 date parsing in the syslog input #26419

faec · 2021-06-22T19:57:04Z

What does this PR do?

Fixes a reported issue where the Syslog input misparses days of the month that begin with a 0.

Added a test exhibiting the bug, updated the ragel spec, reran go generate, and confirmed that the test now passes.

Checklist

My code follows the style guidelines of this project
~~I have commented my code, particularly in hard-to-understand areas~~
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

elasticmachine · 2021-06-22T19:57:05Z

Pinging @elastic/agent (Team:Agent)

mergify · 2021-06-22T19:59:54Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b syslog-date upstream/syslog-date
git merge upstream/master
git push upstream syslog-date

elasticmachine · 2021-06-22T20:13:48Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #26419 updated
Start Time: 2021-06-23T15:47:45.000+0000
Duration: 120 min 42 sec
Commit: 3528650

Test stats 🧪

Test	Results
Failed	0
Passed	14125
Skipped	2311
Total	16436

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	14125
Skipped	2311
Total	16436

jsoriano · 2021-06-23T07:48:18Z

filebeat/input/syslog/format_check.go

 	st7:
-		if (p)++; (p) == (pe) {
+		if ( p)++; ( p) == ( pe) {


Interesting, why is ragel adding these spaces? 🤔

Looks like linters don't like them.

This is interesting, I am not sure why Ragel would add it, I think

jsoriano · 2021-06-23T15:57:28Z

make update doesn't work but make check does?

@faec what version of Go do you have locally? Check these issues #26419 #26186.

faec · 2021-06-23T16:05:03Z

@faec what version of Go do you have locally? Check these issues #26419 #26186.

I checked the go version, it's 1.16.5 as it should be. The problem is in the build somewhere: make update didn't run gofmt on format_check.go so the linter failed on the whitespace. But make check did run it, and left the corrected file in place, so I was able to commit that.

ph · 2021-06-23T18:00:31Z

filebeat/input/syslog/rfc5424_test.go

+	test := []testRule{
+		{
+			title: "Test two-digit mdays",
+			log:   []byte(`<165>1 2003-08-07T05:14:15.000003-07:00 192.0.2.1 myproc 8710 - - %% It's time to make the do-nuts.`),


This test made me smile :)

(cherry picked from commit 0ae157e)

(cherry picked from commit 0ae157e) Co-authored-by: Fae Charlton <fae.charlton@elastic.co>

* master: (32 commits) [Metricbeat] Change Account ID to Project ID in `gcp.billing` module (elastic#26412) update libbeat fields.ecs.yml file and ecsVersion to 1.10.0 (elastic#26121) [Filebeat] Update AWS ELB ingest pipeline (elastic#26441) [FIlebeat] add strict_date_optional_time_nanos date format to PanOS module (elastic#26158) Fix the irregular and typo on prometheus module. (elastic#25726) [Filebeat] Parse additonal debug data fields for Okta module (elastic#25818) fix: update MSSQL Server linux image's Docker registry (elastic#26440) Update indexing.go godocs (elastic#26408) Do not close filestream harvester if an unexpected error is returned when close.on_state_change.* is enabled (elastic#26411) Add support for copytruncate method when rotating input logs with an external tool in `filestream` input (elastic#23457) Allow fields with ip_range datatype (elastic#26444) Add Anomali ThreatStream support to threatintel module (elastic#26350) fix: use the right param type (elastic#26469) [Automation] Update elastic stack version to 8.0.0-7640093f for testing (elastic#26460) Set SM Filebeat modules as GA (elastic#26226) Fix rfc5464 date parsing in the syslog input (elastic#26419) Add linked account information into billing metricset (elastic#26285) [Filebeat] Update HA Proxy log grok patterns (elastic#25835) disable metricbeat logstash test_node_stats (elastic#26436) chore: pass BEAT_VERSION when running E2E tests (elastic#26291) ...

manarth · 2021-08-01T09:33:28Z

The fix is reported in the release notes for 7.13.3.

The changes don't appear to be in the code tagged as 7.13.3 or 7.13.4.

Perhaps an issue with the build system, where the release notes are including references to code which isn't included in the build?

faec added bug Team:Elastic-Agent Label for the Agent team backport-v7.14.0 Automated backport with mergify labels Jun 22, 2021

faec self-assigned this Jun 22, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 22, 2021

faec added 3 commits June 22, 2021 16:01

add failing test

8dcf2d1

Correct handling of MDAY in rfc5464 date parsing

8dff69a

update changelog

2a026b3

faec force-pushed the syslog-date branch from 77d7956 to 2a026b3 Compare June 22, 2021 20:02

faec requested a review from jsoriano June 22, 2021 21:38

add license boilerplate

08c2128

jsoriano approved these changes Jun 23, 2021

View reviewed changes

make update doesn't work but make check does?

3528650

ph approved these changes Jun 23, 2021

View reviewed changes

ph reviewed Jun 23, 2021

View reviewed changes

faec merged commit 0ae157e into elastic:master Jun 23, 2021

faec deleted the syslog-date branch June 23, 2021 18:13

mergify bot pushed a commit that referenced this pull request Jun 23, 2021

Fix rfc5464 date parsing in the syslog input (#26419)

78bd4e8

(cherry picked from commit 0ae157e)

mergify bot mentioned this pull request Jun 23, 2021

[7.x](backport #26419) Fix rfc5464 date parsing in the syslog input #26453

Merged

faec mentioned this pull request Jun 23, 2021

[build] Some files required by make check aren't updated by make update #26454

Closed

faec added a commit that referenced this pull request Jun 24, 2021

Fix rfc5464 date parsing in the syslog input (#26419) (#26453)

8fe3d21

(cherry picked from commit 0ae157e) Co-authored-by: Fae Charlton <fae.charlton@elastic.co>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix rfc5464 date parsing in the syslog input #26419

Fix rfc5464 date parsing in the syslog input #26419

faec commented Jun 22, 2021 •

edited

Loading

elasticmachine commented Jun 22, 2021

mergify bot commented Jun 22, 2021

elasticmachine commented Jun 22, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

jsoriano Jun 23, 2021

jsoriano Jun 23, 2021 •

edited

Loading

ph Jun 23, 2021

jsoriano commented Jun 23, 2021

faec commented Jun 23, 2021

ph Jun 23, 2021

manarth commented Aug 1, 2021

Fix rfc5464 date parsing in the syslog input #26419

Fix rfc5464 date parsing in the syslog input #26419

Conversation

faec commented Jun 22, 2021 • edited Loading

What does this PR do?

Checklist

elasticmachine commented Jun 22, 2021

mergify bot commented Jun 22, 2021

elasticmachine commented Jun 22, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

jsoriano Jun 23, 2021

Choose a reason for hiding this comment

jsoriano Jun 23, 2021 • edited Loading

Choose a reason for hiding this comment

ph Jun 23, 2021

Choose a reason for hiding this comment

jsoriano commented Jun 23, 2021

faec commented Jun 23, 2021

ph Jun 23, 2021

Choose a reason for hiding this comment

manarth commented Aug 1, 2021

faec commented Jun 22, 2021 •

edited

Loading

elasticmachine commented Jun 22, 2021 •

edited by jenkins-beats-ci bot

Loading

jsoriano Jun 23, 2021 •

edited

Loading