Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check when response content type is nil #28457

Merged
merged 3 commits into from
Oct 15, 2021
Merged

Check when response content type is nil #28457

merged 3 commits into from
Oct 15, 2021

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Oct 14, 2021 •
edited
Loading

What does this PR do?

Filebeat failed to process any SQS message with aws-s3 input and here is the error in Filebeat log:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x10f339af4]

goroutine 42 [running]:
github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*s3ObjectProcessor).download(0xc0007e22c0)
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/s3_objects.go:181 +0xf4
github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*s3ObjectProcessor).ProcessS3Object(0xc0007e22c0)
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/s3_objects.go:136 +0x165
github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*sqsS3EventProcessor).processS3Events(0xc0005f4190, {0x110a71950, 0xc000522100}, 0xc0003359e0, {0xc000537400, 0x203000})
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/sqs_s3_event.go:245 +0x49f
github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*sqsS3EventProcessor).ProcessSQS(0xc0005f4190, {0x110a71950, 0xc000522100}, 0xc0007dc440)
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/sqs_s3_event.go:111 +0x246
github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*sqsReader).Receive.func1({{}, 0xc000687da0, 0xc000b2c0f0, 0xc000b2c120, 0x0, 0x0, 0xc000b2c150, 0xc000b2c180}, {0xc052496a37d38cb8, 0x287b04758, ...})
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/sqs.go:90 +0x1b8
created by github.com/elastic/beats/v7/x-pack/filebeat/input/awss3.(*sqsReader).Receive
        /Users/kaiyansheng/go/src/github.com/elastic/beats/x-pack/filebeat/input/awss3/sqs.go:82 +0x595

This PR is to check if GetObject API call response content type exists.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 14, 2021
@mergify
Copy link
Contributor

mergify bot commented Oct 14, 2021

This pull request does not have a backport label. Could you fix it @kaiyan-sheng? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Oct 14, 2021
@kaiyan-sheng kaiyan-sheng added backport-v7.15.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify Team:Integrations Label for the Integrations team labels Oct 14, 2021
@kaiyan-sheng kaiyan-sheng self-assigned this Oct 14, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 14, 2021
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Oct 14, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 14, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-10-15T16:35:18.611+0000

  • Duration: 104 min 14 sec

  • Commit: 6cf48ff

Test stats 🧪

Test Results
Failed 0
Passed 7717
Skipped 1201
Total 8918

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

andrewkroh
andrewkroh approved these changes Oct 15, 2021
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aspacca
Copy link
Contributor

aspacca commented Oct 15, 2021

LGTM, can you add a test case on s3_objects_test.go? @kaiyan-sheng
see the one I've already added for resp = nil at https://github.com/elastic/beats/blob/master/x-pack/filebeat/input/awss3/s3_objects_test.go#L152-L171

aspacca
aspacca reviewed Oct 15, 2021
View reviewed changes
x-pack/filebeat/input/awss3/s3_objects_test.go
},
}

if contentType != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@kaiyan-sheng kaiyan-sheng merged commit ccc2206 into elastic:master Oct 15, 2021
@kaiyan-sheng kaiyan-sheng deleted the fix_aws_s3 branch October 15, 2021 18:49
mergify bot pushed a commit that referenced this pull request Oct 15, 2021
@kaiyan-sheng
Check when response content type is nil (#28457)
ba190ec 
(cherry picked from commit ccc2206)
@mergify mergify bot mentioned this pull request Oct 15, 2021
Merged
mergify bot pushed a commit that referenced this pull request Oct 15, 2021
@kaiyan-sheng
Check when response content type is nil (#28457)
b4d3527 
(cherry picked from commit ccc2206)
@mergify mergify bot mentioned this pull request Oct 15, 2021
Merged
kaiyan-sheng added a commit that referenced this pull request Oct 15, 2021
@mergify @kaiyan-sheng
Check when response content type is nil (#28457) (#28477)
38b3d0d 
(cherry picked from commit ccc2206)

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
kaiyan-sheng added a commit that referenced this pull request Oct 15, 2021
@mergify @kaiyan-sheng
Check when response content type is nil (#28457) (#28478)
bbc3b7a 
(cherry picked from commit ccc2206)

Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>
Icedroid pushed a commit to Icedroid/beats that referenced this pull request Nov 1, 2021
@kaiyan-sheng
Check when response content type is nil (elastic#28457)
c299de3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aspacca aspacca aspacca approved these changes

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

@kaiyan-sheng kaiyan-sheng

Labels
backport-v7.15.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify Team:Integrations Label for the Integrations team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@kaiyan-sheng @elasticmachine @aspacca @andrewkroh