Breaking change for 8.0, namespace_annotations replaced by namespace.annotations

[MichaelKatsoulis]

What does this PR do?

As part of #16483 and after merging of #27917 which

• Updates kubernetes.namespace from keyword to group field.
• Moves kubernetes.namespace which was the name of the namespace to kubernetes.namespace.name field.
• Renames namespace_labels and namespace_annotations metadata to namespace.labels and namespace.annotations

Agent's k8s provider needs to be handled accordingly.
In this PR

• The mappings created by pod and service watcher are updated to add the namespace annotations correctly as namespace.annotations instead of namespace_annotations.

Why is it important?

The fields namespace_annotations will not be published anymore in 8.0 release. The namespace annotations will be part of namespace field which is updated from string to group.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[ChrsMark]

If k8sMapping["namespace"] return nil it will panic. Perhaps you can check the value first or you can just add the namespace.annotations key at top level at kubemetaMap.(common.MapStr).Clone(). Sth like this:

ckMeta := kubemetaMap.(common.MapStr).Clone()
if len(namespaceAnnotations) != 0 {
    ckMeta.Put("kubernetes.annotations", namespaceAnnotations)
}
k8sMapping := map[string]interface{}(ckMeta)

[MichaelKatsoulis]

TBH I thought there is no way that this can be nil for a pod or a service

[MichaelKatsoulis]

I updated it

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-10-11T15:18:43.230+0000

• Duration: 90 min 7 sec

• Commit: c4312e9

Test stats 🧪

Test	Results
Failed	0
Passed	7109
Skipped	16
Total	7125

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

[MichaelKatsoulis]

Waiting for #28247 and #27917 to be merged before merging this one.