ElasticSearch/Elastica: searching for exact term that includes "and" or other stopwords

Question

I'm trying to get ES QueryString to match a search term that includes "and" within it, but everything I've tried so far (trying different analyzers, tokenziers, filters) has not worked. In MySQL terms, what I want is:

WHERE field LIKE '%abbot and costello%'

I've tried various configurations, this is what I'm currently using (slight improvement in that it matches "abbot " (with trailing space), but still not matching anything with "and" in it:

$eI->create(array(
    'analysis' => array(
        'analyzer' => array(
            'indexAnalyzer' => array(
                'type' => 'custom',
                'tokenizer' => 'SQLedgeNGram',
                'filter' => array(
                    'lowercase',
                ),
            ),
            'searchAnalyzer' => array(
                'type' => 'custom',
                'tokenizer' => 'SQLedgeNGram',
                'filter' => array(
                    'lowercase', 
                ),
            )
        ),
        'tokenizer' => array(
            'SQLedgeNGram' => array(
                'type' => 'edgeNGram',
                'min_gram' => 2,
                'max_gram' => 35,
                'side' => 'front'   
            ),
            'standardNoStop' => array(
                'type' => 'standard',
                'stopwords' => ''   
            )   
        )
    )
), true
);

Here's my test case field value:

Abbott and Costello - Funniest Routines, Vol.

Trying the various analyzers, I just can't seem to get it to match anything that includes "and".

Results:

searching [abbot] 
 @       searchAnalyzer          total results: 1
 @       standard                total results: 1
 @       simple                  total results: 1
 @       whitespace              total results: 1
 @       keyword                 total results: 1


searching [abbot ] 
 @       searchAnalyzer          total results: 1
 @       standard                total results: 1
 @       simple                  total results: 1
 @       whitespace              total results: 1
 @       keyword                 total results: 1


searching [abbot and c] 
     searchAnalyzer          total results: 0
     standard                total results: 0
     simple                  total results: 0
     whitespace              total results: 0
     keyword                 total results: 0


searching [abbot and cost] 
     searchAnalyzer          total results: 0
     standard                total results: 0
     simple                  total results: 0
     whitespace              total results: 0
     keyword                 total results: 0


searching [abbot and costello] 
     searchAnalyzer          total results: 0
     standard                total results: 0
     simple                  total results: 0
     whitespace              total results: 0
     keyword                 total results: 0


searching [abbot costello] 
     searchAnalyzer          total results: 0
     standard                total results: 0
     simple                  total results: 0
     whitespace              total results: 0
     keyword                 total results: 0

score 1 · Accepted Answer · answered Apr 16 '13 at 18:01

1

You have typo in the query (missing second t in abbott). You also don't need to run search through ngrams. The search tokenizer can be keyword and it will still work for phrases shorter than 35 characters. And, by the way, edgeNGram will only give you trailing wildcards. For leading and trailing wildcards you need to use nGram filter.

answered Apr 16 '13 at 18:01

imotov

28,277
3
90
82

I know it was a simple answer, but thank you for taking the time to point out the problem, this caused me a lot of grief.. checked everything but the most obvious :( – Eva Apr 17 '13 at 03:52

ElasticSearch/Elastica: searching for exact term that includes "and" or other stopwords

1 Answers1