I have a multivalued solr field, which is only used to filter the search results. This field contains a number of groups and if the user belongs to one of this groups then this document will be in his search resalt.
<field name="group" type="string" indexed="true" multiValued="true" stored="false"/>
Since I have tens of thousands of groups, this causes performance problems.
Is there any better way to handle fields with this many values than multivalued fields?
One possible way to handle this is to transform the data into a single field with a custom character around each value, For Ex. if groups contain group1, group2, group3 as multivalued data, transform them to a single field containing single value of "||group1|| ||group2|| ||group3||" (double pipe being the custom character). Then you can use the same custom character around the group name during query time like for ex fq=group:"||group1||" to filter on group1. If you choose this approach, make sure to change the field type to the one that supports partial match and white space tokenizer.
I am not sure if this will increase performance. But I suggest try increasing
maxFieldLength (if you are using <solr4.x)
which is removed in Solr 4.0 if you are using Solr 4.X or newer,
try including filter for fieldType definition and increase maxTokenCount value.
<filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="10000"/>
