0

I've already tested the sql_last_value with an integer and datetime type of fields in my jdbc plugin within the logstash input and it's working perfectly.

I'm just curios, is there any possibility of using a varchar/string type of field, in order to update the sql_last_value time to time in case if there aren't any datetime or int field in my table.

Any help could be appreciated.

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
Kulasangar
  • 9,046
  • 5
  • 51
  • 82
  • Can you please show you config file. I am facing similar kind of problem. My logstash is working with sql_last_value as an integer, but it reads old records also. I can see duplicate records in ES. I checked .logstash_jdbc_last_run file, it contains 0. Don't know where I am going wrong. I am using follwing configuration also use_column_value => true tracking_column => "SEQ_NO" record_last_run => true – Vishwas Tyagi Dec 05 '16 at 10:05

1 Answers1

2

The answer is no. As a proof you can look at the source code for the jdbc input plugin.

The declaration of the tracking_column_type field highlights the following:

  # Type of tracking column. Currently only "numeric" and "timestamp"
  config :tracking_column_type, :validate => ['numeric', 'timestamp'], :default => 'numeric'

This means that this field will only be allowed to contain two values, namely timestamp and numeric.

Val
  • 207,596
  • 13
  • 358
  • 360
  • `If tracking column value rather than timestamp, the column whose value is to be tracked config :tracking_column, :validate => :string` from the given link above, could I use just `tracking_column` only in order to compare `string` type column? – Kulasangar Nov 08 '16 at 16:39
  • 1
    you should try it, no big risk :-) – Val Nov 08 '16 at 16:42
  • Alright, so is there any other way to do the comparison rather than using `> :sql_last_value` since it's a `string` value where you can't check for a greater value ? – Kulasangar Nov 08 '16 at 16:43
  • It's definitely ok to [compare strings in SQL](http://stackoverflow.com/questions/8083781/is-it-safe-to-compare-strings-with-greater-than-and-less-than-in-mysql), you should try it out and see if it works for you. – Val Nov 08 '16 at 16:58