3

We are using Hibernate Search (Lucene Engine) to enable fuzzy search for text some data we are storing in SQL Server database and consumed by a search service written in Java 8. The data source for the search is a table with moderate edit/update frequency. What we need is that for any change made while the index is rebuilt, we want to ensure that search function is still running and accessible instead of getting locked by process of index being built.

In short, how to temporarily use existing index while a new is being built and replace it on completion.

Shubham Mishra
  • 341
  • 1
  • 9
  • Let's ask the most important question: why do you need to rebuild the index? Is it corrupted? Did you update your mapping? Or maybe you just don't want to or cannot enable [automatic indexing](https://docs.jboss.org/hibernate/search/5.10/reference/en-US/html_single/#_automatic_indexing) for some reason? Depending on that, some strategies make sense and some do not. – yrodiere Aug 02 '18 at 06:15
  • @YoannRodière our system is built in a way that multiple services are using same database tables. The service that does write operations on the said table and service that does search operation using hibernate search are completely isolated from each other. So the index built by Hibernate are stale for search service and hence need to be rebuilt as we found that new data added to tables wouldn't be returned by our search service. – Shubham Mishra Aug 02 '18 at 09:17

1 Answers1

2

Not an answer to your question, but that could help you anyway: from what I understood in your comments, you don't really need to fully rebuild the index, you just need to source update events from a different place than the application executing the search queries.

If your "updating" application uses Hibernate ORM, you can solve the issue without fully rebuilding the index: build the index automatically and incrementally in the "updating" application (using Hibernate Search in "automatic indexing" mode without ever doing any query), and make sure the index is made available to the "searching" applicaton.

The latter can be achieved either:

  • by using the experimental Elasticsearch integration and connecting both applications to the same cluster: one will update it, the other will use it to search.
  • or by using the Lucene integration with the filesystem-master/filesystem-slave directory providers, which will allow you to periodically (and without downtime) copy the index from the "updating" application (master) to the "searching" application (slave). Note that if multiple applications update the index, you will need to use the JMS or JGroups backend to redirect all index updates to a master node and avoid conflicts. Beware, configuring JMS or JGroups won't be obvious.
yrodiere
  • 9,280
  • 1
  • 13
  • 35