[Packetbeat] Update DNS protocol to use ECS fields

[andrewkroh]

This updates the DNS protocol to have more closely follow ECS.
The DNS tunneling dashboard has been updated to work with the new
field names.

In order to better interoperate with other data sources the trailing dot
has been removed from domain names. For example, previously Packetbeat
would produce dns.question.name:elastic.co. and now it will simply produce
dns.question.name:elastic.co. It's a breaking change but it will be make it
easier to pivot with other data sources.

Part of #7968

Here's a summary of what fields changed.

Changed

• bytes_in -> source.bytes
• bytes_out -> destination.bytes
• notes -> error.message
• responsetime -> event.duration (unit are now nanoseconds)
• transport -> network.transport

Added

• event.end
• event.start
• network.bytes
• network.community_id
• network.protocol = dns
• network.type

Unchanged Packetbeat Fields

• method - dns opcode
• query = class {{ dns.question.class }}, type {{ dns.question.type }}, {{ dns.question.name }}
• request - text representation of the entire request
• response - text representation of the entire response
• resource - dns.question.name
• status
• type = dns (we might remove this since we have event.dataset)

[ruflin]

Could you add a changelog entry and update ecs-migration.yml. I assume also some updates to fields.yml for dns are needed?

[ruflin]

My first though was to fix it to Beta2 but at this stage the go code was not in yet :-(

[andrewkroh]

Luckily the code generated from this commit is exactly the same as what would be in beta2 if I backported (I started a backport and realized they’d be the same).

[andrewkroh]

The are no changes needed to fields.yml in this PR. When I finish updating the last protocol in Packetbeat then there should be several updates to removing unused fields. And then I can add aliases to ecs-migration.yml. None of the DNS specific fields were changed.

I’ll update the changelog.

[andrewkroh]

CHANGELOG.next is now updated.

[ruflin]

As far as I can see for example bytes_out was renamed to destination.bytes? That would mean bytes_out in the fields.yml should become an alias to destination.bytes? Or you can't change it yet because it's also used in other places / protocols?

For the ecs-migration.yml: I would prefer to have a list for each protocol so if we map bytes_out to destination.bytes multiple time, it's ok if it shows up there multiple times.

What we often do in these ECS migration PR's is also having in the PR description the list of fields which we renamed.

[andrewkroh]

@ruflin I added a few fields to ecs-migration.yml (all with alias: false because the fields are still being used by other protocols). And I updated the PR description.

Can you please take another look.

[ruflin]

LGTM

We normally don't rename the fields in the dashboard as I was hoping to do this with a script all at once. But it should not cause any issue if we have some dashboards already renamed. Good to go.

Thanks for the detailed PR description.