0

I have millions of documents with the following fields:
name (string), start version (int), end version (int).

I need to query efficiently all records which answers the query:
Select all documents where version >= "start version" and version<="end version"

Running the above query took 50-100 ms while similar query by tagging each version took only 15 ms.

My question is how efficient can Solr handle such queries?
Any alternate solutions will be welcomed.
The fields values / types can be changed if needed.

Avner Levy
  • 6,601
  • 9
  • 53
  • 92
  • What exactly do you mean by "full scan" here(in scope of solr)? And how did you measured that there was a "full scan"? – rchukh Jan 17 '14 at 19:30

1 Answers1

0

I suppose that you will not find a solution significantly better than Solr. The scenario you describe is one of the target scenarios in Solr. I would check these things before deciding how to proceed:

  • Check if you have an index on the specific field. You can see/define indexes on Solr using its configuration files. You add indexes in the Schema.xml file. In the SolrConfig.xml you could also define low level behaviour of indexes.

  • Check if caching is enabled. This could actually speed up your realistic use scenarios. You could change the respective settings in SolrConfig.xml.

  • If after the above checks/changes you still are not satisfied with the Solr performance, I would check some NoSQL database solution. Perhaps mongoDB which is a document oriented storage providing full index support, could perform better.

Hope I helped!

Pantelis Natsiavas
  • 5,293
  • 5
  • 21
  • 36