0

I am using logstashto read the postgres data using jdbc input plugin and push to elastic

The data is coming properly and things seems to be working fine just for a small problem i.e.

My logs table has a field requesttimestamp with datatype of timestamp. Since there is a historical data and also to ensure that the timelines are based on the data and not the time of run, I am trying to set the value of @timestamp with requettimestamp.

the filter configuration is as follows:

{ match => ["requesttimestamp", "yyyy-MM-dd HH:mm:ss.SSS"] }

but while running it is tagging it as _dateparsefailure and using the system time as @timestamp

I also tried using the following format : ISO8601 but nothing seems to be working.

I am sure it is something simple but not able to put the finger to it.

Vivek
  • 156
  • 3
  • 18
  • What is the value in the field `requesttimestamp`? What's the value when logstash prints it (with output stdout)? – baudsp Nov 02 '17 at 15:05
  • its as follows `"requesttimestamp":"2017-11-01T18:29:22.499Z"` i had even tried giving the following format `yyyy-MM-dd'T'HH:mm:ss.SSS` but i still get `_dateparsefailure` – Vivek Nov 02 '17 at 16:37
  • I have tested the example (`"2017-11-01T18:29:22.499Z"`) with ISO8601 and it worked on my platform. Perhaps when you're receiving it, the field `requesttimestamp` is already a timestamp, so that's why the match fails. Other than that, I have no idea. – baudsp Nov 02 '17 at 17:33
  • Looks like that was the problem. since logstash is directly reading from the database the the type is directly being derived as `timestamp` and hence seems to be failing during the filter step. I removed the filter itself and is working fine now ! Thanks for the direction – Vivek Nov 10 '17 at 06:10

0 Answers0