In a Lucene / Lucene.net search, how do I count the number of hits per document?

Question

When searching a bunch of documents, I can easily find the number of documents which match my search criteria:

Hits hits = Searcher.Search(query);
int DocumentCount = hits.Length();

How do I determine the total number of hits within the documents? For example, let's say I search for "congress" and I get 2 documents back. How can I get the number of times "congress" occurs in each document? For example let's say "congress" occurs 2 times in document #1 and 3 times in document #2. The result I'm looking for is 5.

score 7 · Accepted Answer · edited Jul 17 '13 at 19:21

7

This is Lucene Java, but should work for Lucene.NET:

List docIds = // doc ids for documents that matched the query, 
              // sorted in ascending order 

int totalFreq = 0;
TermDocs termDocs = reader.termDocs();
termDocs.seek(new Term("my_field", "congress"));
for (int id : docIds) {
    termDocs.skipTo(id);
    totalFreq += termDocs.freq();
}

edited Jul 17 '13 at 19:21

M.Babcock

18,753
6
54
84

answered Feb 12 '10 at 15:34

bajafresh4life

12,491
5
37
46

@bajafresh4life: What about if the phrase was two words like "apple tree"? – Keltex Feb 12 '10 at 17:15
Do you want the # of times the phrase appears in each doc or each individual word? – bajafresh4life Feb 12 '10 at 19:02

score 0 · Answer 2 · answered Feb 16 '10 at 23:41

This is Lucene Java also. If your query/search criteria can be written as a SpanQuery, then you can do something like this:

IndexReader indexReader = // define your index reader here
SpanQuery spanQuery = // define your span query here
Spans spans = spanQuery.getSpans(indexReader);
int occurrenceCount = 0;
while (spans.next()) {
    occurrenceCount++;
}
// now occurrenceCount contains the total number of occurrences of the word/phrase/etc across all documents in the index

In a Lucene / Lucene.net search, how do I count the number of hits per document?

2 Answers2

Linked