2

Summary: I have my log with the date format as shown :

2013/05/09-05:19:16.772

Now I want to use logstash and send these logs to elastic search. But the problem is that I want that the timestamp value should be that of the logs and not the current time.

Therefore, I have written the following. This fails saying this:

Invalid format: "2013/05/09-05:19:16.876" is malformed at "/05/09-05:19:16.876", :backtrace=>["org.joda.time.format.DateTimeFormatter.parseDateTime(DateTimeFormatter.java:866)"

My conf file is:

input {
  stdin {
      type => "stdin-type"
  }
}

filter {
  grok {
    type => "stdin-type"
    patterns_dir=>["./patterns"]
    pattern => "%{PARSE_ERROR}"
    add_tag=>"%{type1},%{type2},%{slave},ERR_SYSTEM"
  }

  date {
    type => "stdin-type"
    match=>["ts","yyyy/mm/dd-HH:mm:ss.SSS"]
    locale=>"en"
  }

  mutate {
    type=>"stdin-type"
    replace => ["@message", "%{message}" ]
    replace => ["@timestamp", "%{ts}" ]
  }
}

output {
  stdout { debug => true debug_format => "json"}
  elasticsearch { }
}

I am really stuck here. Need some expert help.

Thanks.

Yohann
  • 6,047
  • 3
  • 26
  • 33
user2359303
  • 261
  • 2
  • 7
  • 15

3 Answers3

4

You have specifed lowercase m instead of M. m is for minutes and M is for months.

Try:

yyyy/MM/dd-HH:mm:ss.SSS
Wilfred Hughes
  • 29,846
  • 15
  • 139
  • 192
Chakra Yadavalli
  • 399
  • 1
  • 7
1

What is not readily apparent from the error until you enable verbose mode (-vv) from the command line is that the 'date' filter is using the JodaTime library. In my case, I was using a regex from a previous step to parse the date and time. This obviously won't work because it's not what JodaTime expects. For the allowable date time formats, you should look at the DateTimeFormat documentation.

Richard Nienaber
  • 10,324
  • 6
  • 55
  • 66
0

Try

YYYY/MM/dd-HH:mm:ss.SSS

instead

Nikolay Bryskin
  • 21
  • 1
  • 2