Introduce add_labels and add_tags processors

[urso]

Introduce add_fields and add_tags processors. The fields/tags settings will be marked deprecated starting with 6.7.

Replace custom processors in pipeline/processors.go with new definitions.

Cleanup pipeline/processors.go by removing dead code.

Fix recent regressions from beat -> agent renaming potentially triggering a panic due to concurrent map updates.

[kaiyan-sheng]

This is a great example for me. Thanks! Just curious, is this test := test necessary here?

[urso]

It depends. I tend to put it here so to bind the name locally in case I enable t.Parallel(). functions/go-routines in a for-block accessing a loop variable will see the value being changed otherwise.

[ph]

handy tricks, I should use it more often.

[ruflin]

Great to see these 2 new processor. Left some feedback related to ECS.

If we remove the feature in 7.0, would be perhaps provide some migration code to modify the config to use the processors instead? Not sure how comment the usage of these 2 is, but I assume not rare.

[urso]

Note: I will add documentation later. There is some more cleanup/renaming of other processors I want to do as well first.

[ph]

handy tricks, I should use it more often.

[ruflin]

Will the following config work?

processors:
  add_labels:
    labels:
      foo.bar: 17

I'm mainly concerned about the . in the key. If not, is that something we want to support? Would be nice to have some system tests to confirm this works or not.

[urso]

Will the following config work?

What do you mean by work. It works the same current fields settings work. You configuration will produce a document saying:

{
  ...
  "labels": {
    "foo": {
      "bar": 17
    }
  },
  ...
}

The way we read config files one can only choose between dedot or flat dottet output always. The information whether a dot was used in the config or not is lossed thanks to the config file parser.