0

I am using LogStash 7.3.2 to fetch incremental data from SQL Server using this query:

select * from mytable where lastupdatetimestamp > :sql_last_value

I also have specified last_run_metadata_path in logstash config file.

It works fine but sometimes it is throwing an exception:-

Exception when executing JDBC query {:exception=>#

transition (daylight savings time 'gap'): 1942-09-01T00:00:00.000 (Asia/Kolkata)>}

Why am I getting this exception and due to this exception it does not save last timestamp value and again it fetches duplicate records from SQL Server.

Any help regarding this would be highly appreciated.

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Raj
  • 429
  • 2
  • 6
  • 26

1 Answers1

0

As stated here

In Logstash if a datetime string has no TZ part it is assumed to be a UTC time.

If you know that the datetime is a local time string then you need to tell the date filter which timezone the date was generated in. You may use one of the Joda timezones,

e.g. America/New_York, America/Detroit or EST5EDT - these are merely labels that point to Timezone Java code that know what changes in clocks occurred in all timezones for quite some time into the past.

See this page for info on how the updates are followed. http://www.oracle.com/technetwork/java/javase/tzdata-versions-138805.html 100

Once the datetime string is converted to a Timestamp object (by the date filter) it is considered UTC time.

Vignesh Kumar A
  • 27,863
  • 13
  • 63
  • 115