0

This may sound like an opinion question, but it's actually a technical one: Is there a standard process for maintaining a simple data set?

What I mean is this: let's say all I have is a list of something (we'll say books). The primary storage engine is MySQL. I see that Solr has a data import handler. I understand that I can use this to pull in book records on a first run - is it possible to use this for continuous migration? If so, would it work as well for updating books that have already been pulled into Solr as it would for pulling in new book records?

Otherwise, if the data import handler isn't the standard way to do it, what other ways are there? Thoughts?

Thank you very much for the help!

Mr Mikkél
  • 2,577
  • 4
  • 34
  • 52
  • Solr's DIH can perform Import,Update and Delete. There is a very good series from the solr.pl guys. [Have a read there](https://www.google.de/search?q=site:solr.pl/en+dataimport+handler). – cheffe Jun 08 '15 at 06:52
  • Make an answer, cheffe, and I'll accept it. This is exactly what I needed. Thank you! – Mr Mikkél Jun 08 '15 at 15:45

2 Answers2

0

If you want to update documents from within Solr, I believe you'll need to use the UpdateRequestHandler as opposed to the DataImportHandler. I've never had need to do this where I work, so I don't know all that much about it. You may find this link of interest: Uploading Data With Index Handlers.

If you want to update Solr with records that have newly been added to your MySQL database, you would use the DataImportHandler for a delta-import. Basically, how it works is you have some kind of field in MySQL that shows the new record is, well, new. If the record is new, Solr will import it. For example, where I work, we have an "updated" field that Solr uses to determine whether or not it should import that record. Here's a good link to visit: DataImportHandler

TMBT
  • 1,183
  • 10
  • 17
0

The question looks similar to the one which we are doing, but not with SQL. Its with HBase(hadoop stack DB). However there we have Hbase indexer, which after mapping DB with Solr, listens to the events in hbase(DB) for new rows, and then executes code to fetch those values from DB and add in Solr. Not sure if there is such for SQL. However the concept looks similar. IN SQL I know about triggers which can listen to inserts and updates. At that even, you can trigger something to execute the steps of adding them in continuosly manner.

Ramzy
  • 6,948
  • 6
  • 18
  • 30