Using stemming in a SOLR query

Question

I've set up SOLR, and added a document to the example 'collection1'.

<doc>
    <str name="id">3007WFP</str>
    <str name="name">Fishing</str>
    <str name="type">Ladies</str>
</doc>

I can query it ok in the interface using

name:*fishing*

but I would like to get stemming to work, so that I can e.g. type fish and get this document due to the word fishing being stemmed. I'd also like to be able to find Ladies when Lady is searched for.

However I have tried the query

name:fish

and I get no results. I didn't add the wildcard as I know that will match, and only want to test the stemming function.

I've changed the schema type of both the name and type fields to text_en which I believe includes stemming - restarted SOLR, and reindexed (clicked optimise).

Is there something I'm missing or doing wrong, is the query syntax different when you want to use stemming?

Another strange issue is after the change from text_general to text_en a search for

name:Fishing

produces no results, even though it should be an exact match..

Could you help me with the text_general and text_en description? — JHS, Feb 20 '13 at 15:30

score 2 · Answer 1 · answered Feb 20 '13 at 15:42

Clicking optimize won't re-index the documents. It will simply merge the various segments in your existing index, which means your index is still old. So once you re-post i.e. re-index your documents, name:fish should match.

BTW, you can look into what the analyzer is doing at http://localhost:8983/solr/#/collection1/analysis. You can choose the fieldType and see what happens at index time. For example, using this tool you can see that for text_en type,

fishing -> fish
ladies -> ladi

so a search for type:lady won't match this document. If you also specify your query at 'Field Value (Query)' it will highlight the matches, if there are any.

score 0 · Answer 2 · answered Feb 20 '13 at 15:54

You will have to delete the documents and re-add them so that the new fieldType changes take effect.

For details on how stemming works and the different type of stemming available you can have a look here.

In your case the language is English so you could use the PorterStemFilterFactory.

Using stemming in a SOLR query

2 Answers2