1

I am using a loop in scala to query an Oracle table every 10 second, since Oracle table get continuously insertion. I create a select request then I create n json string containing n line from oracle that I push into Elasticsearch. After that I create a delete request to erase the n line from Oracle table that I have inserted into ES. I developped a completely beginner approach. So can you suggest me a better approach to load in real time or micro batch data from Oracle to ES and delete from Oracle. I heard about logstach or SreamSets. Do you have any idea? Thanks

a.moussa
  • 2,977
  • 7
  • 34
  • 56
  • Do you have any issues with the current approach? I would suggest a pattern where you 1. Mark the records to be selected (with an UPDATE); 2. Select those marked records (select); 3, Delete the marked records. This way you don't accidentally delete what you have loaded. What loads the Oracle table? Can it load Elasticsearch directly? – Nick.Mc Aug 21 '17 at 09:45

0 Answers0