Synonym analyzer not working

Question

Here are my settings:

{
"countries": {
"aliases": {},
"mappings": {
  "country": {
    "properties": {
      "countryName": {
        "type": "string"
      }
    }
  }
},
"settings": {
  "index": {
    "creation_date": "1472140045116",
    "analysis": {
      "filter": {
        "synonym": {
          "ignore_case": "true",
          "type": "synonym",
          "synonyms_path": "synonym.txt"
        }
      },
      "analyzer": {
        "synonym": {
          "filter": [
            "synonym"
          ],
          "tokenizer": "whitespace"
        }
      }
    },
    "number_of_shards": "5",
    "number_of_replicas": "1",
    "uuid": "7-fKyD9aR2eG3BwUNdadXA",
    "version": {
      "created": "2030599"
    }
  }
},
"warmers": {}
}
}

My synonym.txt file is in the config folder inside the main elasticsearch folder.

Here is my query:

query: {
    query_string: {
        fields: ["countryName"],
        default_operator: "AND",
        query: searchInput,
        analyzer: "synonym"
        }
     }

The words in synonym.txt are: us, u.s., united states.

So this doesn't work. What's interesting is that search works as normal, except for when I enter any of the words in the synonym.txt file. So for example, when I usually type in us into the search, I would get results. With this analyzer, us doesn't give me anything.

I've done close and open to my ES server, and still it doesn't work.

EDIT

An example of a document:

{
    "_index": "countries",
    "_type": "country",
    "_id": "57aabeb80057405968de152b",
    "_score": 1,
    "_source": {
      "countryName": "United States"
    }

Example of searchInput (this is coming from the front-end):

united states

EDIT #2:

Here is my updated index config file:

{
"countries": {
    "aliases": {},
    "mappings": {},
    "settings": {
        "index": {
            "number_of_shards": "5",
            "creation_date": "1472219634083",
            "analysis": {
                "filter": {
                    "synonym": {
                        "ignore_case": "true",
                        "type": "synonym",
                        "synonyms_path": "synonym.txt"
                    }
                },
                "analyzer": {
                    "synonym": {
                        "filter": [
                            "synonym"
                        ],
                        "tokenizer": "whitespace"
                    }
                }
            },
            "country": {
                "properties": {
                    "countryName": {
                        "type": "string",
                        "analyzer": "synonym"
                    },
                    "number_of_replicas": "1",
                    "uuid": "50ZwpIVFTqeD_rJxlmd59Q",
                    "version": {
                        "created": "2030599"
                    }
                }
            },
            "warmers": {}
        }
    }
}
}

When I try adding documents, and doing a search on said documents, the synonym analyzer does not work for me.

EDIT #3

Here are 2 documents in the index:

{
"took": 3,
"timed_out": false,
"_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
},
"hits": {
    "total": 2,
    "max_score": 1,
    "hits": [{
        "_index": "stocks",
        "_type": "stock",
        "_id": "2",
        "_score": 1,
        "_source": {
            "countryName": "United States"
        }
    }, {
        "_index": "stocks",
        "_type": "stock",
        "_id": "1",
        "_score": 1,
        "_source": {
            "countryName": "Canada"
        }
    }]
}
}

Can you create a more complete example? You are missing the mapping and the test data. — Andrei Stefan, Aug 25 '16 at 19:26
Okay done. I've added the mapping with one of the fields in my `country` index, `countryName`. — MonkeyOnARock, Aug 26 '16 at 04:45
What is the `searchInput` value and give example of a document that you have in your index. — Andrei Stefan, Aug 26 '16 at 04:56
Okay, I've edited my post to include the things you've asked. — MonkeyOnARock, Aug 26 '16 at 05:02

Andrei Stefan · Accepted Answer · 2016-08-26T14:35:04.770

3

You are close, but I suggest reading thoroughly this section from the documentation to understand better this functionality.

As a solution:

PUT /countries
{
  "mappings": {
    "country": {
      "properties": {
        "countryName": {
          "type": "string",
          "analyzer": "synonym"
        }
      }
    }
  },
  "settings": {
    "analysis": {
      "filter": {
        "synonym": {
          "ignore_case": "true",
          "type": "synonym",
          "synonyms_path": "synonym.txt"
        }
      },
      "analyzer": {
        "synonym": {
          "filter": [
            "lowercase",
            "synonym"
          ],
          "tokenizer": "whitespace"
        }
      }
    }
  }
}

You need to delete the index and create it again with the mapping above. Then use this query:

  "query": {
    "query_string": {
      "fields": [
        "countryName"
      ],
      "default_operator": "AND",
      "query": "united states"
    }
  }

edited Aug 26 '16 at 14:35

answered Aug 26 '16 at 05:13

Andrei Stefan

51,654
6
98
89

I've tried for a few hours to get `"analyzer": "synonym"` added as a field, but it doesn't allow me to. I get this error: `"type": "mapper_parsing_exception",` `"reason": "analyzer [synoynm] not found for field [countryName]"` – MonkeyOnARock Aug 26 '16 at 12:40
You also need to drop the initial index, create a new one with the new settings and mappings and re-index the documents. – Andrei Stefan Aug 26 '16 at 12:46
1

Okay, I completed deleted the initial index, and re-made it. With this I was finally able to add `"analyzer": "synonym"`. However... the analyzer still isn't working. When I type **'united states'**, I don't get results with **'us'**, for example. – MonkeyOnARock Aug 26 '16 at 13:02
Ok. Provide those documents and I can test it. – Andrei Stefan Aug 26 '16 at 13:05
Okay, I've edited my OP to reflect the current state of my config file. – MonkeyOnARock Aug 26 '16 at 14:06
I was asking for test documents... It matters what you index, because ES will add the synonyms at indexing time. So, **provide some documents that I can test**. – Andrei Stefan Aug 26 '16 at 14:17
Okay, see Edit #3 in my OP. I've added the documents in my index. – MonkeyOnARock Aug 26 '16 at 14:24
Your mapping is completely wrong. The `country` mapping should be under `mappings`, but you have nothing there (`"mappings": {}`). I asked you to use my code, but it seems you didn't. – Andrei Stefan Aug 26 '16 at 14:34
Yeah, the `"mappings": {}` is empty, that's a mistake (I have 20 tabs open with different versions of all this, and copied the wrong one). My current configuration is a copy/paste of your original suggestion. After I insert the 2 documents, and do a search for **'us'** (as an example), I still get no results. – MonkeyOnARock Aug 26 '16 at 14:53
Well, please allow me to doubt your statement. I have used the code I provided to test this and it works for me, including the search for `us`. I tried to help you, but you don't seem to pay attention to my suggestions. Best of luck! – Andrei Stefan Aug 26 '16 at 14:59
Please be aware: Elasticsearch does NOT support "string" as type anymore. This has been replaced with type "text" https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html – Alexander Falk Aug 07 '19 at 09:22

score 0 · Answer 2 · answered Aug 25 '16 at 15:41

0

Have you deleted/created the index after pushing the txt ?

I think you should remove the "synonyms": "" if you are using "synonyms_path"

answered Aug 25 '16 at 15:41

Vincent Chalmel

590
1
6
28

I just tried deleting/creating index, and I still have the exact problem(s) from my original post. – MonkeyOnARock Aug 25 '16 at 15:45
Do you stille have both `"synonyms"` and `"synonyms_path"` in your mapping ? – Vincent Chalmel Aug 26 '16 at 07:25

Synonym analyzer not working

2 Answers2

Linked