Lucene Autocomplete with multiple words using Shingle filter

Question

I am trying to make a Lucene autocomplete using Lucene's Dictionary and spellcheck classes, but so far only successful in making it work for single terms.

I googled and found out that we need to make use of Shingle Matrix filter to get the work done.. Can someone experienced with Lucene show me a way to do it ?

All I need is it has to generate words for autocomplete with phrases. For example, if I have a doc like this : "This is a long line with very long rant with too many words in it", Then I should be able to generate words like "long line", "long rant", "many words" etc...

Possible ?

Thanks.

possible duplicate of [How to do query auto-completion/suggestions in Lucene?](http://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene) — Bo Persson, Mar 11 '12 at 19:04
See http://stackoverflow.com/questions/24968697/how-to-implements-auto-suggest-using-lucenes-new-analyzinginfixsuggester-api/25301811#25301811 for a complete example of how to do autocomplete with Lucene. — John Wiseman, Aug 14 '14 at 07:21

score 0 · Answer 1 · edited Nov 20 '14 at 19:45

You can write your own Analyzer implementing TokenStream function in inheriting Lucene.Net.Analysis.Analyzer class. There u can use this shingleFilter to get multiword from the tokenstream Code Stream:

public override Lucene.Net.Analysis.TokenStream TokenStream(String fieldName, System.IO.TextReader       
reader)
   { 
      Lucene.Net.Analysis.TokenStream tokenStream = new     
      Lucene.Net.Analysis.Standard.StandardTokenizer(Lucene.Net.Util.Version.LUCENE_30, reader);  
      tokenStream = new Lucene.Net.Analysis.Shingle.ShingleFilter(tokenStream, maxShingleSize);
      return tokenStream;
   }

max Shingle size identifies max length of multi word unit

score 0 · Accepted Answer · answered Mar 31 '11 at 12:25

0

writer = new IndexWriter(dir, new ShingleAnalyzerWrapper(new StandardAnalyzer( Version.LUCENE_CURRENT, Collections.emptySet()),3), false, IndexWriter.MaxFieldLength.UNLIMITED);

This did the job for me...

answered Mar 31 '11 at 12:25

Shrinath

7,888
13
48
85

2

Hi how did you get this to work? I've been trying to get multiple word did you mean suggestion feature. Can you outline the steps you took to implement this or even better post some more code? – Naz Mar 08 '12 at 20:30

Lucene Autocomplete with multiple words using Shingle filter

2 Answers2