Present average processing time in Kibana

Question

I have my application logs from logstash in the below format.

{
"Timestamp": "2015-09-09T10:54:57.4562725+00:00",
"Message": "Started processing",
"MessageId": "b80fb2aa-4b7b-4f49-9e60-865c6afa688e",
"ClientName": "TestClient"

}

{
"Timestamp": "2015-09-09T10:55:57.4562725+00:00",
"Message": "Done processing",
"MessageId": "b80fb2aa-4b7b-4f49-9e60-865c6afa688e",
"ClientName": "TestClient"

}

{
"Timestamp": "2015-09-09T10:55:57.4562727+00:00",
"Message": "Time Elapsed: 561 ms",
"MessageId": "b80fb2aa-4b7b-4f49-9e60-865c6afa688e",
"ClientName": "TestClient"

}

What I am trying to achieve is to get the average processing time (all logging done with same MessageId are part of one processing cycle).

The last log in every transaction has the processing time ( as shown above: "Message": "Time Elapsed: 561 ms"), how can i get the average? Any ideas?

have you tried to match a filter against the different messages and add a tag or type for the 3 different messages? then you can filter by time elapsed — Dude, Sep 22 '15 at 14:29

score 0 · Answer 1 · answered Sep 25 '15 at 22:05

You'll need to extract the milliseconds in the message with "Time Elapsed" using the grok filter, then pass it to the metrics filter. You'll get a %{processingTime.mean} variable you can use in filters or output. Something like:

filter {
  grok {
    match => { "Message" => "Time Elapsed: %{INT:processingTime} ms" }
    add_tag => [ "has_processingTime" ]
  }
  if "has_processingTime" in [tags] {
    metrics {
      timer => [ "processingTime", "%processingTime" ]
      add_tag => "metric"
    }
  }
}

Additionally, you'll get other interesting metrics like min, max, percentiles, and rates. See https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html.

Present average processing time in Kibana

1 Answers1