Split ecs.version into parts

[rw-access]

Scenario

ecs.version is good information for compatibility, but because it's a string, it's actually non-trivial to write a query that validates semver. If we add major, minor and patch then we can match this much better.

Problem

For instance, let's say I want to search my data for semantic matches for 1.3.0, because that's the first version that has a field I'm looking for. I could write that in KQL a few different ways:

• ecs.version:(1.3.0 or 1.4.0 or 1.5.0 or 1.6.0 or 1.7.0 or ...)
• ecs.version:1.* and ecs.version >= 1.3.0

But both have problems. The first requires you to know all of the versions. It also doesn't check for patch versions, but those could be done by replacing .0 with .*.

The second approach does a string comparison and not a semantic comparison. That would mean that we'd miss 1.10.0 when comparing >= 1.3.0, since >= is lexicographical.

Proposal

We should still keep ecs.version. It's super cheap anyway and contains the full string. But we can also add ecs.major, ecs.minor and ecs.patch. If they are stored as numbers then we can make semantic validation trivial: ecs.major: 1 and ecs.minor >= 3. Even if we're comparing to a hypothetically patched 1.3.2, we could do ecs.major:1 and (ecs.minor:3 and ecs.patch:2 or ecs.minor > 3).

[ruflin]

This reminds me of elastic/elasticsearch#48878 Would be nice to have support for it directly in ES.

[webmat]

Yes, we're not going to add discrete major/minor/patch fields exactly because semver is coming.

We haven't had time to investigate how to integrate them broadly in the schema yet. So I'm not sure if they can be a drop-in replacement for the current keyword version fields, or if we'll have to integrate them more gradually or what.

But anyone is welcome to step forward here and help on this :-)

[rw-access]

Ah, apparently the relevance of ECS was raised on that issue earlier by webmat. elastic/elasticsearch#48878 (comment)

Is there anything worthwhile doing for ECS in the interim? I don't know the roadmap for the ES issue. I can continue using the workarounds for the time being, but if this is something of priority to others, I can tackle the major/minor/patch for now, and we can change the data type of ecs.version for 2.0?

[dainperkins]

One thing that would be nice to add is a vendor version - for vendors (or users) tracking their particular implementation versions.

[ebeahan]

Yes, we're not going to add discrete major/minor/patch fields exactly because semver is coming.

Yes - given that work in ES for the version data type is under way (#887, elastic/elasticsearch#59773), let's hold off on implementing the discrete fields.