Skip to content

Commit

Permalink
aws-s3 - Disable event normalization processing
Browse files Browse the repository at this point in the history
Disable event normalization for the aws-s3 input to reduce allocations when processing events.
The input only produces basic types in its events. Either it puts a string into the `message` field
or it decodes json into a map[string]interface with encoding/json. Both of those should be fine
for the downstream processors and outputs.

Relates elastic#33657
  • Loading branch information
andrewkroh committed Nov 14, 2022
1 parent 00f57f3 commit 4ccb09d
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]
- Improve httpjson documentation for split processor. {pull}33473[33473]
- Added separation of transform context object inside httpjson. Introduced new clause `.parent_last_response.*` {pull}33499[33499]
- Cloud Foundry input uses server-side filtering when retrieving logs. {pull}33456[33456]
- Disable "event normalization" processing for the aws-s3 input to reduce allocations. {pull}33673[33673]

*Auditbeat*

Expand Down
8 changes: 8 additions & 0 deletions x-pack/filebeat/input/awss3/input.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ func (in *s3Input) Run(inputContext v2.Context, pipeline beat.Pipeline) error {
client, err := pipeline.ConnectWith(beat.ClientConfig{
CloseRef: inputContext.Cancelation,
ACKHandler: awscommon.NewEventACKHandler(),
Processing: beat.ProcessingConfig{
// This input only produces events with basic types so normalization
// is not required.
EventNormalization: boolPtr(false),
},
})
if err != nil {
return fmt.Errorf("failed to create pipeline client: %w", err)
Expand Down Expand Up @@ -368,3 +373,6 @@ func getProviderFromDomain(endpoint string, ProviderOverride string) string {
}
return "unknown"
}

// boolPtr returns a pointer to b.
func boolPtr(b bool) *bool { return &b }

0 comments on commit 4ccb09d

Please sign in to comment.