how to index and search phrase query with special characters in lucene.net?

Question

I tried to search multiple words and special characters like "Engineering & Construction" using phrasequery and added in to boolean query but its not getting any result.The way i'm indexing the query is

doc.Add(new Field("Industry","Engineering & Construction", Field.Store.YES, Field.Index.ANALYZED,Field.TermVector.WITH_POSITIONS_OFFSETS));

For Searching:

var booleanQuery = new BooleanQuery();
PhraseQuery phrasequery = new PhraseQuery();
phrasequery.Add(new Term("Industry","Engineering & Construction"));
booleanQuery.Add(phraseQuery, BooleanClause.Occur.MUST);

the booleanQuery contains {+Industry:"Engineering & Construction"} even though its not getting desired result.

I'm using StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_29) @goalie7960 — Dinesh_Dini, Jul 12 '13 at 14:26
I know StandardAnalyzer will strip out the "&" symbol. Do you even need to analyze the input or will you always have the full/beginning part of the phrase to search for? — goalie7960, Jul 12 '13 at 15:39

score 1 · Answer 1 · answered Jul 12 '13 at 15:55

1

This

phrasequery.Add(new Term("Industry","Engineering & Construction"));

Produces a single term, Engineering & Construction, but the index will have two terms, engineering and construction, in sequence (the & will be removed by the analyzer). Constructing a phrasequery manually like this requires you to understand the tokens, and add each term separately, like:

phrasequery.Add(new Term("Industry","engineering"));
phrasequery.Add(new Term("Industry","construction"));

Of course, the easier way is to use a query parser;

Query phraseQuery = queryparser.parse("Industry:Engineering & Construction");
booleanquery.add(phraseQuery);

answered Jul 12 '13 at 15:55

femtoRgon

32,893
7
60
87

I'm using ` Analyzer analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_29); QueryParser queryParser = new QueryParser(Lucene.Net.Util.Version.LUCENE_29, "", analyzer);` .The querparser result was {+(Industry:engineering construction)}.it eliminates the special character. @femtoRgon – Dinesh_Dini Jul 13 '13 at 04:24
I need to search exact phrase including the special characters.@goalie7960 – Dinesh_Dini Jul 13 '13 at 04:27
If you need the special character maintained, you are using the wrong Analyzer. Use [KeywordAnalyzer](http://lucene.apache.org/core/3_0_3/api/all/org/apache/lucene/analysis/KeywordAnalyzer.html). – femtoRgon Jul 13 '13 at 06:23
By using WildcardQuery and setting Field.Index.NOT_ANALYZED for indexing, its working perfectly for my criteria.This link could be useful [link](http://stackoverflow.com/questions/12165488/lucene-net-and-partial-starts-with-phrase-search?answertab=active#tab-top) – Dinesh_Dini Jul 13 '13 at 07:15
1

Yes, using `NOT_ANALYZED` will work as well, though if you want an exact match, you should use a `TermQuery` instead of `WildcardQuery`. Using a `WildcardQuery` to acquire an exact match doesn't make sense. – femtoRgon Jul 13 '13 at 08:28
Thank you @femtoRgon.By changing KeywordAnalyzer,the phrase query works perfectly. – Dinesh_Dini Jul 13 '13 at 08:52

Dinesh_Dini · Accepted Answer · 2013-07-13T08:38:10.673

1

For Indexing :

doc.Add(new Field("Industry","Engineering & Construction", Field.Store.YES, Field.Index.NOT_ANALYZED));

For Searching :

TermQuery query = new TermQuery(new Term("Industry", "Engineering & Construction"));
booleanQuery.Add(query, BooleanClause.Occur.MUST);

It was helpful for my criteria.It searches the exact phrase with special characters.

edited Jul 13 '13 at 08:38

answered Jul 13 '13 at 07:46

Dinesh_Dini

419
8
20

Field.Index.NOT_ANALYZED would also mean that you can't search for this field with "free" text as the words engineering or construction since it will only work for exact match. So not a good option if the input is user input from ie. a textbox since the user have to get upper/lower case right to get a match. – Markus Knappen Johansson Jul 21 '21 at 12:53

how to index and search phrase query with special characters in lucene.net?

2 Answers2