1

I'm trying to handle an nginx access log in logstash.

In order to get millisecond accuracy for my timestamps, I'm using the $msec variable. This means that timestamps will be of the form 1430832725.814, where the integer part is a UNIX timestamp, and the fraction part is in milliseconds. Unfortunately, I can't find a logstash date format which can parse UNIX timestamps.

Is there another way to get nginx to log with millisecond accuracy? Or a way to get the logstash date filter to parse UNIX timestamps?

leedm777
  • 305
  • 5
  • 9
  • Have you read docs? «There are a few special exceptions.... “UNIX_MS” - will parse unix time in milliseconds since epoch» – Alexey Ten May 05 '15 at 13:53
  • I missed that. But I don't have milliseconds since the epoch. I have seconds, with millisecond resolution. – leedm777 May 05 '15 at 17:49

2 Answers2

0

The UNIX format literal can handle fractional seconds, so it will be able to parse an $msec field.

leedm777
  • 305
  • 5
  • 9
0

firstly, using grok filter to map the $msec field to a output field

grok {
 match => { "message" =>\[%{GREEDYDATA:unix_timestamp}\] 
}

then using another Date filter to convert it to @timestamp field

  date {
     match => ["unix_timestamp", "UNIX"]
  }

If you want to map to fields other than @timestamp, refer to Logstash reference

Popeye
  • 101