0

I am facing an issue while running Logstash with a configuration that uses the JDBC input plugin to synchronize data from a MySQL database to Elasticsearch. Here are the details of the problem:

Following is my elastic_mysql_sample.conf (configuration) file:

input {
  jdbc {
    clean_run => true
    jdbc_driver_library => "D:/softs/mysql-connector-java-8.0.30.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/elastic_mysql_sample"
    jdbc_user => "root"
    jdbc_password => "root"
    statement => "SELECT * FROM elastic_mysql_sample.employee_sample WHERE logstash_tracking_id > :sql_last_value"
    use_column_value => true
    tracking_column => "logstash_tracking_id"
    tracking_column_type => "numeric"
    jdbc_default_timezone => "Asia/Kathmandu"
  }
}

filter {
  date {
    match => ["@timestamp", "YYYY-MM-DD HH:mm:ss,SSS"]
    timezone => "Asia/Kathmandu"
    target => "@timestamp"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200/"]
    index => "employee_idx"
  }
}
  • I am using Logstash version 8.8.2, and the bundled JDK is being used for Logstash.
  • The MySQL database connection details and Logstash configuration are set correctly.
  • However, when I run the command .\logstash.bat -f ./config/elastic_mysql_sample.conf from bin, I encounter the following error:

[2023-07-25T12:23:41,023][ERROR][logstash.agent] Failed to execute action
 {:action=>LogStash::PipelineAction::Create/pipeline_id:main, 
:exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to 
configure plugins: (ArgumentError) Cannot determine timezone from 
nil\n(secs:1690267112.402,utc~:\"2023-07-25 
06:38:32.40199995040893555\",ltz~:nil)\n(etz:nil,tnz:\"NPT\",tziv:\"2.0.6\",
tzidv:\"1.2023.3\",rv:\"2.6.8\",rp:\"java\",win:true,rorv:nil,astz:nil,eov:\
"1.2.7\",eotnz:\"???\",eotnfz:\"???\",eotlzn:\"???
\",\ndebian:nil,centos:nil,osx:nil)\nTry setting `ENV['TZ'] = 
'Continent/City'` in your script (see 
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)", :backtrace=>
["org.logstash.config.ir.CompiledPipeline.<init>
(CompiledPipeline.java:120)", 
"org.logstash.execution.AbstractPipelineExt.initialize(AbstractPipelineExt.java:186)", 
"org.logstash.execution.AbstractPipelineExt$INVOKER$i$initialize.call(AbstractPipelineExt$INVOKER$i$initialize.gen)", 
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:846)", 
"org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1229)",
 "org.jruby.ir.instructions.InstanceSuperInstr.interpret(InstanceSuperInstr.java:131)", 
"org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)", 
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)", 
"org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:128)", 
"org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:115)", 
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:329)", 
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:911)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", 
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:329)", 
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", 
"org.jruby.ir.instructions.CallBase.interpret(CallBase.java:549)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)", 
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)", 
"org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:92)", 
"org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:238)", 
"org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:225)", 
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:226)", 
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:393)", 
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:206)", 
"org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)", 
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)", 
"org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)", 
"org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:66)", 
"org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", 
"org.jruby.runtime.Block.call(Block.java:143)", 
"org.jruby.RubyProc.call(RubyProc.java:309)", 
"org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:107)", 
"java.base/java.lang.Thread.run(Thread.java:833)"]

I tried setting up my timezone and timestamp format. I also changed date filter. But I get the same error again and again. I am looking for generic solution as I am new to ELK stack. Please let me know if there is something new for the solution or to do the above task.

Satya Raj Awasthi
  • 1
  • 2
  • Not sure if it could be related to your issue, but in the past I ran into a datatype error with date fields from jdbc and the `date` filter expecting a string. You can use a mutate to convert the date column from the db to a string to see if that helps you. – superstienos Aug 20 '23 at 16:09

0 Answers0