Elasticsearch "Painless" returns wrong and rounded values

Question

Running Elasticsearch 7.3.2. I have two float fields "timeEnd" and timeStart. I simply want to return difference.

In Kibana index-patterns I created a scripted variable "rate":

if (doc['timeEnd'].length==0) return null;
if (doc['timeStart'].length==0) return null;
return doc['timeEnd'].value-doc['timeStart'].value;

This is what it returns:

[
 {
  "timeEnd": null,
  "timeStart": 1571760869.518571,
  "rate": [
   null
  ]
 },
 {
  "timeEnd": 1571760898.821922,
  "timeStart": 1571760736.872881,
  "rate": [
   128
  ]
 },
 {
  "timeEnd": 1571760893.483377,
  "timeStart": 1571760879.161604,
  "rate_mbps": [
   0
  ]
 },...

Most returned values are 0 or 128.

I have faced similar problem. The solution is to use 'double' instead of 'float'. Ideally, float should have been able to handle calculation (add/sub) on these values but for some reason it shows problem. Hence, just change the data-type to 'double' — Atur, Oct 25 '19 at 10:59
@Atur thanks I will try that. Are you aware of any bug report? This is a rather serious issue. — user3037720, Oct 25 '19 at 16:02
I am not aware of any bug, but I can search for it. This looks like a common case, so there should already be a bug logged. — Atur, Oct 25 '19 at 16:12

score 1 · Accepted Answer · answered Oct 25 '19 at 04:29

1

timeEnd and timeStart are of float type , which is a single-precision 32-bit floating point number. You need to change them to double which is a double-precision 64-bit floating point number

answered Oct 25 '19 at 04:29

jaspreet chahal

8,817
2
11
29

thanks for the quick response. Does it mean I have to change template (dynamically generated) and reindex all the data? – user3037720 Oct 25 '19 at 16:01
1

Yes..reindex will be beneficial/required. Also I would suggest you to use dynamic strict or false, instead of true. Self generated templates (mapping) sometimes cause problems later on – Atur Oct 25 '19 at 16:14
1

This is not bug , it the size of the data type. Float can only store nine values before decimal point and double is a larger type – jaspreet chahal Oct 27 '19 at 11:10

score 0 · Answer 2 · answered Mar 05 '21 at 17:39

As @jaspreet-chahal correctly mentioned, this is not a bug.

The float numeric field type in elasticsearch follows the IEEE 754 standard(wikipedia) for a single-precision floating point floating point format, restricted to finite values (i.e. not Infinite nor NaN see @elastic/elasticsearch#27653).

For example:

The equivalent float of 1571760898.821922 is 1571760896, hence the error in your scripted field.

https://www.h-schmidt.net/FloatConverter/IEEE754.html

Please mark jaspreet's answer as the accepted solution. Thanks

Elasticsearch "Painless" returns wrong and rounded values

2 Answers2