0

I am trying to parse log files from IIS to the ELK stack (Logstash:2.3, Elastic:2.3 and Kibana:4.5, CentOS 7 vm).

I have attempted to parse a date field from the log message as the event timestamp using the date filter below in my logstash configuration:

date {
    match => ["date_timestamp", "yyyy-MM-dd HH:mm:ss"]
    timezone => "Europe/London"
    locale => "en"
    target => "@timestamp"
}

The first few characters of the entire log message that was parsed to Elastic Search is:

"message": "2016-03-01 03:30:49  .........

The date field above was parsed to Elastic Search as:

"date_timestamp": "16-03-01 03:30:49",

However, the event timestamp that was parsed to Elastic Search using the date filter above is:

"@timestamp": "0016-03-01T03:32:04.000Z",

I will like the @timestamp to be exactly 2016-03-01T03:30:49 as I can't immediately figure out why there is a difference between the hours and minutes.

I have looked at similar problems and documentations such as this one on SO and this one on logstash documentation and logstash documentation.

Any pointer in the right direction will be appreciated.

Regards

SO

Community
  • 1
  • 1
SOJ
  • 595
  • 5
  • 11

1 Answers1

1

in your date_timestamp you have only 2 characters for year: "16-03-01 03:30:49", so the date pattern in your date filter is incorrect, should be: 

date {
    match => ["date_timestamp", "yy-MM-dd HH:mm:ss"]
    timezone => "Europe/London"
    locale => "en"
    target => "@timestamp"
}
Кирилл Полищук
  • 845
  • 6
  • 4
  • thank you for your response - however, the original message that was parsed to elastic search is this: "message": "2016-03-01 03:30:49 ......... - the "16-03-01 03:30:49" should have been 2016-03-01 03:30:49 - isn't it? – SOJ Apr 14 '16 at 12:50
  • your suggestion works - thanks. I did not previously figure that that the format is that of the parsed field and not format of the original message. In the absence of any other answer, I will mark this as accepted because it works for me. Cheers - SO. – SOJ Apr 14 '16 at 14:20