1

I'm building a web application that involves very frequent database updates (10s-100s per second maybe), and also requires the same rate of search operations. (i.e. Users post information and then that information then needs to be full-text searched very regularly, and very soon after its submission)

The current plan is to use MySQL as the database, but build an index over the searchable data in Solr to return "item id" which would then be retrieved from the MySQL db.

Is anyone able to provide me with any insight/info on the ability of Solr to support my needs?

Thanks!

DJSunny
  • 1,970
  • 3
  • 19
  • 27

1 Answers1

3

It's not going to be easy, as Solr was originally designed to be performant with low updates frequencies. However, major improvements have been made with Solr+RankingAlgorithm and soft commits (now available in Solr 4.0 nightly builds). If those will meet the performance requirements of your project is for you to find out (there are also lots of other factors involved, from available hardware to search requirements involving other Solr components).

Mauricio Scheffer
  • 98,863
  • 23
  • 192
  • 275
  • "performant with low updates frequencies" so what are the possible ways to improve performance? Im thinking eventual consistency. – Jesvin Jose Sep 12 '11 at 06:57
  • @aitchnyu : that *is* how you achieve eventual consistency: by updating (technically in Solr, committing) less frequently. See also http://stackoverflow.com/questions/7011247/solr-real-time-indexing http://stackoverflow.com/questions/7221061/what-is-the-best-practice-to-implement-solr-in-ecom-applications – Mauricio Scheffer Sep 12 '11 at 12:47
  • How "less frequent" is "less frequently"? i.e. what is a reasonable update/add frequency for Solr? – DJSunny Sep 12 '11 at 13:13
  • @csjohn: again, see http://wiki.apache.org/solr/NearRealtimeSearch for some numbers, though that depends on many other factors, like index size, distribution, faceting requirements, etc. – Mauricio Scheffer Sep 12 '11 at 13:19
  • @csjohn: I'd say 1–10 minutes is reasonable for Solr `autoCommit`. – Nick Zadrozny Sep 12 '11 at 22:57