0

In the logstash configuration files with PostgreSQL queries already existed. After adding the config file with mysql, after some time the data ceases to arrive in the elastic, tcpdump on the outgoing port also shows the absence of data, while requests to other servers are made. There are no errors in the logs. When debug is enabled, it shows that the config is re-read and that's it. On another server, where this config is only one, logstash works fine. In what there can be an error? Where to look. Tell me please.

input {
  jdbc {
    jdbc_driver_library => "/etc/logstash/mysql-connector-java-5.1.46-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://xx.xx.xx.xx:3306/database"
    jdbc_user => "user"
    jdbc_password => "*************"
    schedule => "0-59 * * * *"
    statement => "SELECT * FROM `database`.table WHERE calldate > :sql_last_value"
    tracking_column => "calldate"
    tracking_column_type => "timestamp"
    use_column_value => true
    add_field => { "typetable_id" => "table" }
  }
}
output {
        if [typetable_id] == "table" {
            elasticsearch {
                    hosts => "xx.xx.xx.xx:9200"
                    index => "data_index"
                    user => "elastic"
                    password => "***********"
              }
       }
}
TIgor
  • 3
  • 5
  • Have you missed to configure last_run_metadata_path? Also, just curious why are you using 0-59 for minute instead of using * which will do exactly same. – nitzien Sep 18 '18 at 14:21
  • Ok, thanks, but it does not solve the problem – TIgor Sep 18 '18 at 14:26
  • Can you please try to run logstash from command line with --log.level debug --config.debug – nitzien Sep 18 '18 at 14:32
  • Can you check in Logstash's logs run queries and run one manually to see if it returns anything? Maybe `:sql_last_value` is not updating correctly. – Michael Dz Sep 19 '18 at 10:53
  • Thank you. I found a solution. Increased the parameters -xms -xmx. After processing all the data, reduced these values to the original ones. – TIgor Sep 19 '18 at 13:22

0 Answers0