0

Sphinx's proximity-enabled ranker uses a slightly modified BM25 ranker (statistical bag-of-words) + a longest-word-substring match formula strongly favoring the latter, while Solr uses some other statistical ranking function (not BM25, but similar) + a boost if desired for word bigrams (this is similar to the LWS approach). I think both of these don't model a human's view of relevance, whereby relevance doesn't fall off a cliff when words in an answer aren't necessarily adjacent or in the same order.

Simple examples:

Query: Bob Jones

Body: . . . . Jones, Bob . . . . (looks relevant to me, but this will fall back to statistical-only)

-or-

Body: . . . . Bob MiddleName Jones . . . . (same)

I know there is a cost to this, but I can't be the only one who noticed that essentially both Solr and Sphinx will fall back to the bag-of-words statistical ranker if the words are out of order or separated by a word, which could even be a stop word in some cases.

Thoughts? What if I want to rank either of the cases above higher than those where the words just appear somewhere in the document? Or am I wrong and does Solr or Sphinx do this?

Jaimie Sirovich
  • 1,088
  • 2
  • 10
  • 14

1 Answers1

0

In solr there is proximity based ranking. check http://wiki.apache.org/solr/SolrRelevancyCookbook#Term_Proximity

KaKa
  • 1,543
  • 12
  • 18
  • AFAIK this would only improve case #2 (Bob MiddleNameJones). Both of the approaches listed at that link will not address the first case (reversal), and will not work in coordination with statistical ranking. Another point: Given query: Common_Word Other_Common_Word Rare_Word, if the first 2 are close to the other terms but the last word is not, it should matter less. – Jaimie Sirovich Jul 14 '11 at 04:44