[Filebeat] Update fileset input merging to replaces paths and append processors.

[blakerouse]

What does this PR do?

Updates vendored package of go-ucfg to 0.8.3, needed to use the new field merging adjustments. Updates fileset input merging to replaces paths and append processors when merging configurations.

Why is it important?

Allows a container to define modules that contain processors at the same time as adding extra processors during autodiscovery.

docker run -l 'co.elastic.logs/processors.add_fields={"fields":{"foo":"bar"}}' -l co.elastic.logs/module=elasticsearch busybox echo hello

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• [ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Run filebeat with configuration:

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

output.console:
  pretty: true

Start container:

docker run -l 'co.elastic.logs/processors.add_fields={"fields":{"foo":"bar"}}' -l co.elastic.logs/module=elasticsearch busybox echo hello

Related issues

• Closes Error when combining processors autodiscover hint with modules using processors #16190

Use cases

Screenshots

Logs

2020-02-20T06:39:46.264-0500    DEBUG   [autodiscover]  autodiscover/autodiscover.go:190        Generated config: map[audit:map[enabled:true input:map[paths:[/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log] processors:[map[add_fields:map[fields:map[foo:bar]]]] stream:all type:container]] deprecation:map[enabled:true input:map[paths:[/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log] processors:[map[add_fields:map[fields:map[foo:bar]]]] stream:all type:container]] gc:map[enabled:true input:map[paths:[/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log] processors:[map[add_fields:map[fields:map[foo:bar]]]] stream:all type:container]] module:elasticsearch server:map[enabled:true input:map[paths:[/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log] processors:[map[add_fields:map[fields:map[foo:bar]]]] stream:all type:container]] slowlog:map[enabled:true input:map[paths:[/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log] processors:[map[add_fields:map[fields:map[foo:bar]]]] stream:all type:container]]]
2020-02-20T06:39:46.264-0500    DEBUG   [autodiscover]  autodiscover/autodiscover.go:252        Got a meta field in the event
2020-02-20T06:39:46.265-0500    DEBUG   [autodiscover]  cfgfile/list.go:62      Starting reload procedure, current runners: 0
2020-02-20T06:39:46.265-0500    DEBUG   [autodiscover]  cfgfile/list.go:80      Start list: 1, Stop list: 0
2020-02-20T06:39:46.271-0500    DEBUG   [processors]    processors/processor.go:101     Generated new processors: add_locale=[format=offset], add_fields={"fields":{"foo":"bar"}}
2020-02-20T06:39:46.271-0500    DEBUG   [input] log/config.go:204       recursive glob enabled
2020-02-20T06:39:46.271-0500    DEBUG   [input] log/input.go:164        exclude_files: [(?-s:.)gz(?-m:$)]. Number of stats: 0
2020-02-20T06:39:46.271-0500    DEBUG   [input] log/input.go:185        input with previous states loaded: 0
2020-02-20T06:39:46.271-0500    INFO    log/input.go:152        Configured paths: [/var/lib/docker/containers/fe58462fbaec77346b02d431375b39508c386bc31ef930320a4e82643ca02e93/*-json.log]

[exekias]

This is looking great!

Can you add a CHANGELOG entry?

[blakerouse]

@exekias Added

[nerddelphi]

@blakerouse and @exekias

Will that fix my issue? #16464 (comment) ?

[blakerouse]

@nerddelphi yes it will fix the issue that you described in your last comment for the bug.