Out of range query against field of long type gets clipped

[mhko]

We are running ES v5.6.3. When issuing a search query against a field of long type, we noticed that values are clipped to LONG_MAX/MIN when the value is out of range for the long type.

To reproduce

"docs": {
    "properties": {
        "Id": {
            "type": "keyword"
        },
        "long": {
            "type": "long"
        },
   }
}

2. Index a document with Long.MAX_VALUE (9223372036854775807).

3. Issue a search against the field "long" with a out-of-range values

_search/?q=long:9223372036854775808
_search/?q=long:9223372036854775809

returns the document with Long.MAX_VALUE (9223372036854775807) indexed above.

In ES1, such query throws a NumberFormatException. We believe ES6 would throw as well, looking at the code.

We think dd37cd14a4 is the responsible commit, line 731 of NumberFieldMapper.java.

+                try { 
+                    return Long.parseLong(stringValue); 
+                } catch (NumberFormatException e) { 
+                    return (long) Double.parseDouble(stringValue);   <-------------- 731
+                }

[nik9000]

@mhko I'm with you, this seems strange. It feels like it shouldn't return the document at all.

[scottsom]

As the author of the offending commit, I feel obligated to chime in.

There are unfortunate problems that happen to the precision when coercing types of such large numbers with truncation. I noted the lost precision of large doubles but overlooked the impact and breaking change on the handling of whole numbers beyond Long.MAX_VALUE, so that's my mistake.

While you could probably add some logic here with BigInteger / BigDecimal to detect when precision will be lost and throw an exception, just getting rid of truncation is probably the better option as is planned in #25861

[scottsom]

Thinking about this some more, if #25861 does still want to accept "1.0", which Long.parseLong will reject then it would still have to fallback to something like Double.parseDouble, so not as simple as relying on the fix for that issue.

[jimczi]

@mhko sorry for the delay answering this.
This is already fixed for the next release (6.1) by:
#26137
which uses BigDecimal as you @scottsom suggested.