How to dynamically change synonyms for ElasticSearch

Question

My synonyms are stored in a database and, when the synonyms are changed in the database, I want to update any values in the index which may be changed as a result of the synonym change.

There are two parts to this that I can think of. One, figuring out which documents to re-index. Two, figuring out how to tell ElasticSearch that the synonyms have changed. I am struggling with the 2nd one - telling ElasticSearch that the synonyms have changed.

A similar question has been asked - see Change dynamically elasticsearch synonyms - but from reading the answers in that issue, I have not been able to figure out what I need.

Currently, my configuration file looks something like the following:

index :
  analysis :
    analyzer :
      myanalyzer :
        filter: [standard, mysynonymfilter]
filter :
  mysynonymfilter :      
    type : synonym
    synonyms : synonyms.txt
    ignore_case : false
    expand : true
    format : solr

My idea was to do something like the following:

curl -XPUT 'http://127.0.0.1:9200/foo/_settings'  -d '
{
    "filter" : {
        "synonym" : {
            "type" : "mysynonymfilter",
            "synonyms" : [
                "cosmos, universe"
            ] 
        }
    }
}
'

but that doesn't seem to do what I want. That is, the index settings do not get updated as far as I can tell.

Is what I am trying to do possible? And if so, any idea what I am doing wrong?

Also, I am fairly sure I could get this to work by updating the synonym file (if I have to use a file), but that's a bit more complicated and something I'd like to avoid.

Thanks for your help, Eric

score 11 · Answer 1 · answered Nov 21 '14 at 16:56

11

It turns out that you can tell ElasticSearch programmatically that the synonyms have changed. That is, it is not necessary to update the synonym file. Here's the basic steps that are necessary:

Close the index.
Update the index settings with the new synonym list. To be safe, I am updating all of the analyzers, tokenizers and char filters for the index (not just the synonym filter) - but I am not sure that is necessary.
Open the index.

answered Nov 21 '14 at 16:56

Eric

1,945
3
23
33

2

how fast this method is ?? – Sudhanshu Gaur Nov 12 '15 at 21:37
1

Why is this weird way of doing it having 10 upvotes? – Raul Lapeira Herrero May 30 '20 at 10:30

score 7 · Answer 2 · answered Jan 17 '20 at 19:45

I know this is an old thread but as of ES 7.5 they have added a new feature to update synonyms. Have a look at their documentation.

You need to issue a POST api like this POST /twitter/_reload_search_analyzers

This would reload all the search analyzers, also ensure that the synonym token filter have the updateable flag set to true like this "updatedable": true.

PS: This feature is part of X-Pack and comes under the basic license which is free.

score 3 · Answer 3 · answered Jul 12 '17 at 07:04

I know this is an old thread, but in case it helps someone. The answer can be found here:

If you specify stopwords inline with the stopwords parameter, your only option is to close the index and update the analyzer configuration with the update index settings API, then reopen the index.

Updating stopwords is easier if you specify them in a file with the stopwords_path parameter. You can just update the file (on every node in the cluster) and then force the analyzers to be re-created by either of these actions:

Closing and reopening the index (see open/close index), or Restarting each node in the cluster, one by one

It's worth noting that this procedure for stopwords is the recommendation from [the docs on Using Synonyms](https://www.elastic.co/guide/en/elasticsearch/guide/current/using-synonyms.html) for updating synonyms as well: "See [Updating Stopwords](https://www.elastic.co/guide/en/elasticsearch/guide/current/using-stopwords.html#updating-stopwords) for techniques that can be used to refresh the synonyms list." — Carl G, Jan 18 '18 at 02:06

score 2 · Answer 4 · answered Aug 28 '13 at 07:13

2

There is a project for reloading the synonym file lindstromhenrik/elasticsearch-analysis-file-watcher-synonym Although I don't know if it works in the latest versions. Maybe you can start by using the plugin and expanding synonyms at query time, since at query time you will have all the synonyms updated instead of reindexing documents that you guess that should be updated because of changes in the synonyms file.

answered Aug 28 '13 at 07:13

moliware

10,160
3
37
47

Thanks for the response, but I believe I know how to solve my issue if I update the synonym file directly, but my question was intended to deal with not having to update the synonym file (and instead updating the synonyms via a REST command). – Eric Sep 04 '13 at 03:33

score 1 · Answer 5 · answered Oct 29 '15 at 03:04

1

You've flipped mysynonymfilter and synonym in your final curl command. The type should be synonym.

answered Oct 29 '15 at 03:04

Navneet

9,590
11
34
51

How to dynamically change synonyms for ElasticSearch

5 Answers5

Linked