I have problem with data being lost when using a jdbc driver to update elasticsearch from a mysql database. My river is below:
curl -XPUT 'http://localhost:9200/_river/river_index_type/_meta' -d '
{
"type": "jdbc",
"jdbc": {
"strategy": "simple",
"driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://localhost/tt",
"user": "user",
"password": "password",
"sql": "SELECT p.product_id AS _id, ... FROM product p ... WHERE ...",
"poll": "5m",
"autocommit": true
},
"index": {
"type": "type",
"index": "index"
}
}
Initially everything works fine, but later, instead of 1200 results in my index i only have 800-900, and each five minutes the count changes. I don't understand what the problem is. Any help would be appreciated.