SOLR 6.4.1 Suggester is stubbornly case-sensitive, how to make case-insensitive?

Question

I've tried everything under the sun (well it is called solr after all) to make solr Suggest case-insensitive, but it stubbornly continues to be case-sensitive.

This returns a suggestion of Mexican:

http://localhost:8983/solr/mycollection/autocomplete?suggest.q=Mex

This returns 0 results:

http://localhost:8983/solr/mycollection/autocomplete?suggest.q=mex

To further diagnose I tried a lower case /select search against my suggestions field, which successfully returned docs containing "Mexican":

http://localhost:8983/solr/mycollection/select?q=suggestions:mex*

But no such luck using lowercase with the Suggester. It's as though my <filter class="solr.LowerCaseFilterFactory"/> has no effect when used by the Suggester.

I of course did a full config upload, collection reload, data re-index, and suggester rebuild before testing. I'm on SOLR 6.4.1 running in cloud mode. Any ideas? Diagnostic tips?

schema.xml

 <fieldType name="textSuggest" class="solr.TextField" positionIncrementGap="100">
      <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
 </fieldType>

<field name="recipe" type="text_general" indexed="true" stored="true" multiValued="false" />

<field name="suggestions" type="textSuggest" indexed="true" stored="true" multiValued="true" />

<copyField source="recipe" dest="suggestions"/>

solrconfig.xml

  <searchComponent class="solr.SuggestComponent" name="suggest">
    <lst name="suggester">
      <str name="name">foodsuggester</str>
      <str name="lookupImpl">WFSTLookupFactory</str>
      <str name="dictionaryImpl">DocumentDictionaryFactory</str>
      <str name="field">suggestions</str>
      <str name="buildOnStartup">false</str>
      <str name="buildOnCommit">false</str>
      <str name="storeDir">suggester_wfst_dir</str>
      <str name="suggestAnalyzerFieldType">textSuggest</str>
    </lst>
  </searchComponent>

  <requestHandler name="/autocomplete" class="solr.SearchHandler" startup="lazy">
      <lst name="defaults">
        <str name="suggest">true</str>
        <str name="suggest.dictionary">foodsuggester</str>
        <str name="suggest.count">10</str>
      </lst>
      <arr name="components">
        <str>suggest</str>
      </arr>
  </requestHandler>

mjlowky · Answer 1 · 2017-02-28T06:30:22.540

1

It seems the WFSTLookupFactory lookup implmentation is case sensitive.

You can use FuzzyLookupFactory, if you don't have any specific reason for using WFSTLookupFactory.

<str name="lookupImpl">FuzzyLookupFactory</str>

edited Feb 28 '17 at 06:30

answered Feb 28 '17 at 04:12

mjlowky

1,183
12
19

Hey you're right! FuzzyLookupFactory indeed does the trick! The only thing is, I don't want fuzzy results, I only wanted exact case-insensitive suggestions. This is big progress though, thanks. – Magnus Feb 28 '17 at 12:16
1

OK got fuzzy factory returning only exact case-insensitive matches by adding 40 which means that the first 40 chars must match exactly (disregarding case). It works perfectly and still allows for shorter exact matches such as in my case. – Magnus Feb 28 '17 at 14:03

score 1 · Accepted Answer · edited Sep 07 '17 at 10:23

The WFSTLookupFactory apparently does not take the suggestAnalyzerFieldType parameter and it is ignored. You could use the AnalyzingLookupFactory, which will analyze the text according to the suggestAnalyzerFieldType. So if you only want the lower case to be analyzed in the suggester you can use the suggestAnalzerFieldType, and indicate that you want to use the suggestText field type for analysis through the suggestAnalyzerFieldType.

SOLR 6.4.1 Suggester is stubbornly case-sensitive, how to make case-insensitive?

2 Answers2