How to implement the BM25F with a set of corpus and queries in Python using Whoosh indexing tool?

Asked Nov 20 '17 at 21:32

Active Nov 20 '17 at 22:29

Viewed 435 times

So, I am working on a search engine. We have two indexed corpora and some relevant queries based on the corpora. After running the query , we are getting top 10 results. Now I am trying to rank/score the results according to their relevance. I was looking up the BM25F algorithm and I can't figure out how to use the algorithm in Whoosh.

with ix.searcher(weighting = whoosh.scoring.BM25F()) as searcher:
        query_text = QueryParser("stem_text", ix.schema, 
termclass=query.Variations, group= qparser.OrGroup).parse(
            "what is something pretty astonishing?")
        results = searcher.search(query_text, sortedby= scores, limit = 10)
        for hit in results:
            print(hit["stem_text"])

I would like to apply the BM25F algorithm on the results.

edited Nov 20 '17 at 22:29

asked Nov 20 '17 at 21:32

Annesha Chowdhury

It'd help to show what you've already tried. – Dominik Stańczak Nov 20 '17 at 21:41
I edited my question(Added some relevant code)! – Annesha Chowdhury Nov 20 '17 at 22:33
Maybe this might help https://github.com/alvations/shiva-something/blob/master/ngram_context.py – alvas Nov 21 '17 at 01:59
Did using the default `ix.searcher()` return what you need? Try that first, then update the weighting parameter to BM25F later. – alvas Nov 21 '17 at 02:00
Yep ix.searcher() returned the results. But I want to sort the results based on how relevant it is to the Query. Any idea how to rank the results? – Annesha Chowdhury Nov 21 '17 at 02:52

How to implement the BM25F with a set of corpus and queries in Python using Whoosh indexing tool?

0 Answers0