How to calcuate starttime & endtime using painless script

Question

I am working with ELK and i have created index pattern with build_date. Now to calculate the Avg of build duration, i need to find the start-time and end-time in minutes using painless script.

My logstash output data given below

"build_end_time" => "2021-01-13 01:29:49",
"build_duration" => "6409651",
"build_start_time" => "2021-01-12 23:43:00",
"build_date" => "2021-01-12",
"@timestamp" => 2021-02-02T11:40:50.747Z,

Scripted field settings given below.

Name: Duration_time
Language: painless
Type: number
Format: Duration
Input format: minutes 
Output format: Human readable
Popularity: 0

Script: def doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay

it throws - Script is invalid.

{
 "root_cause": [
  {
   "type": "script_exception",
   "reason": "compile error",
   "script_stack": [
    "def doc['build_end_time'].date.m ...",
    "       ^---- HERE"
   ],
   "script": "def doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay",
   "lang": "painless",
   "position": {
    "offset": 7,
    "start": 0,
    "end": 32
   }
  }
 ],
 "type": "search_phase_execution_exception",
 "reason": "all shards failed",
 "phase": "query",
 "grouped": true,
 "failed_shards": [
  {
   "shard": 0,
   "index": "build-logs",
   "node": "JSvuaBbCQr6uI5qKvavj7Q",
   "reason": {
    "type": "script_exception",
    "reason": "compile error",
    "script_stack": [
     "def doc['build_end_time'].date.m ...",
     "       ^---- HERE"
    ],
    "script": "def doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay",
    "lang": "painless",
    "position": {
     "offset": 7,
     "start": 0,
     "end": 32
    },
    "caused_by": {
     "type": "illegal_argument_exception",
     "reason": "invalid sequence of tokens near ['['].",
     "caused_by": {
      "type": "no_viable_alt_exception",
      "reason": null
     }
    }
   }
  }
 ]
}

Can someone help me to work this?

score 0 · Answer 1 · answered Feb 02 '21 at 13:02

0

This looks like a syntactic error.

Instead of

def doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay

use

return doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay

The return is actually not required -- you can leave it out entirely.

The def keyword defines something. So you could in theory say:

def result = doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay

but you'd need to return something -- so:

def result = '...'; return result

answered Feb 02 '21 at 13:02

Joe - GMapsBook.com

15,787
4
23
68

You mean something like this `def result = (doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay); return` ? – user4948798 Feb 02 '21 at 13:23
`return msDiff` at the end. You want to return the difference, don't you? – Joe - GMapsBook.com Feb 02 '21 at 13:24
Does `return doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay` work? – Joe - GMapsBook.com Feb 02 '21 at 13:33
No, it shows - `"org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:823)", "org.elasticsearch.index.query.QueryShardContext.lambda$lookup$1(QueryShardContext.java:328)",` – user4948798 Feb 02 '21 at 13:43
even `def result = doc['build_end_time'].date.millisOfDay - doc['build_start_time'].date.millisOfDay` shows same error. – user4948798 Feb 02 '21 at 13:52
Yea the dates are actually strings so you'll need to [convert them to dates](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html) first. – Joe - GMapsBook.com Feb 02 '21 at 14:05
Ok. So dates need to be converted in my logstash.conf file? – user4948798 Feb 02 '21 at 14:35
Yup, definitely convert them first. – Joe - GMapsBook.com Feb 02 '21 at 14:43
Ok. Even i have 'build_duration' with milliseconds. I think even in that I can achieve duration in mintines? – user4948798 Feb 02 '21 at 14:49
sure -- just divide by 1000 * 60? – Joe - GMapsBook.com Feb 02 '21 at 15:43
Ok using painless script only? – user4948798 Feb 02 '21 at 15:47
Yes, painless should be enough. – Joe - GMapsBook.com Feb 02 '21 at 15:50
Good, let me know how it went. – Joe - GMapsBook.com Feb 02 '21 at 16:02
To Calculate the Average for `build_duration` and convert `milliseconds to hh:mm:ss` create filed as follows `Language: painless Type: number Format: Duration Input Format: Minutes Output Format: Human Readable` script: `String milliSinceEpochString = doc['build_duration.keyword'].value; long milliSinceEpoch = Long.parseLong(milliSinceEpochString); Instant instant = Instant.ofEpochMilli(milliSinceEpoch); ZoneId z = ZoneId.of("Z"); LocalDateTime l = LocalDateTime.ofInstant(instant, z); return l;` – user4948798 Feb 04 '21 at 05:56
preview result shows like `[ { "_id": "37361323", "build_duration": "1062735", "build_test": [ "1970-01-01T00:17:42.735Z" ] },` – user4948798 Feb 04 '21 at 05:57

How to calcuate starttime & endtime using painless script

1 Answers1