Different Elasticsearch results for the same query

Question

I've setup Elasticsearch with 1 cluster á 4 nodes. Number of shards per index: 1; Number of replicas per index: 3

When I call a simple query like the following one multiple times I get different results (different total hits and different top 10 documents):

http://localhost:9200/index_name/_search?q=term

Different data on each shard? I like to have all shards up to date. What can I do?

This is the result of /_cluster/health:

{
  "cluster_name" : "secret",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 4,
  "number_of_data_nodes" : 4,
  "active_primary_shards" : 24,
  "active_shards" : 96,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

As a temporary solution I rebuild the index through Ruby gem tire: ModelName.rebuild_index

But I need a long-term solution.

More details to your setup would be nice. Some information missing here is for example number of replicas per index, number of shards per node would also be nice to know. — Torsten Engelbrecht, Jun 18 '14 at 14:09
Number of shards per index: 1 Number of replicas per index: 3 Where do I get the number of shards per node from? Cannot see it in my elasticsearch.yml. — Murdoch, Jun 18 '14 at 14:20
Sorry, was shard per index. Seeing those I also don't really understand why you have a problem with this to be honest. — Torsten Engelbrecht, Jun 18 '14 at 14:35
Can you post the output of cluster health? http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-health.html — John Petrone, Jun 18 '14 at 14:58
Looks like a split brain problem. I have a similar problem with two nodes even after recreating the index. Did you find out the root cause for this? Can you add some information on the workaround you used? Possibly related topics: http://stackoverflow.com/questions/27723587 http://stackoverflow.com/questions/28899341/elasticsearch-different-results-for-searching-same-index-with-5-or-1-shards http://stackoverflow.com/questions/29368791/elasticsearch-different-results-in-hits-total-and-num-docs https://groups.google.com/forum/#!topic/elasticsearch/goFGavMj0mQ — s.Daniel, May 22 '15 at 12:41
I had a similar problem that has been solved here: http://stackoverflow.com/questions/30402023/what-causes-different-search-results-for-same-elastic-search-query-on-two-nodes/30402153 — s.Daniel, May 27 '15 at 08:22
If total hits are different, then you should investigate split-brain (see if there is a network partition). The node you ran `/_cluster/health` on believes it's part of a 4-node cluster. But run `/_cluster/health` on the other three to see if their stories agree. It's possible that the others cannot see the entire cluster. — Peter Dixon-Moses, Aug 12 '15 at 17:24
It's not unusual for your top hits to be different when querying different cluster nodes, since document-frequency is (by default) calculated at the node level per shard. Try re-running your query with the slower `&search_type=dfs_query_then_fetch` to test this (top docs should be identical). — Peter Dixon-Moses, Aug 12 '15 at 17:28

Sam Critchley · Answer 1 · 2020-02-10T10:34:38.147

We ran into a similar problem and it turned out to be because Elasticsearch round-robins between different shards when searching. Each shard returns a slightly different _score because of slightly different indexing due to the way ES handles deleted documents in an index. In our case this meant similar results often placed slightly lower or higher in the results order, and, when combined with pagination (using from and size in the search query) it meant the same results were turning up on two separate "pages" or not at all from page to page.

We found an Elasticsearch article on consistent scoring which explains this quite neatly and implemented a preference parameter to ensure that we always get the same scores for a particular search by querying the same shards:

http://localhost:9200/index_name/_search?q=term&preference=blablabla

We also thought about using sorting, but Elasticsearch sorts results with the same scores by an internal Lucene document ID, ensuring that results with the same scores are always returned in the same order.

maximus ツ · Answer 2 · 2014-06-18T14:41:08.960

6

This is because you don't have specified sort order and size. So every time you query you get random first 10 records as default size for result set by elasticsearch server is 10.

You can add sorting in following way with curl,

curl -XPOST 'localhost:9200/_search' -d '{
 "query" : {
   ...
  },
   "sort" : [
     {"price" : {"order" : "asc", "mode" : "avg"}}
   ]
}'

Check here for for more info specially from and size with sort which is most mostly used for pagination.

update:

Though default sort is score DESC sometime it not works when records don't have relevant _score, as per http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_sorting.html#_sorting

edited Jun 18 '14 at 14:41

answered Jun 18 '14 at 14:27

maximus ツ

7,949
3
25
54

3

I should not have different total hits each time even without specifying sort and size. When I don't specify sort then I get the prefered default sort which is "score DESC". – Murdoch Jun 18 '14 at 14:32
2

`score DESC` is correct but trouble is when two records has same score. – maximus ツ Jun 18 '14 at 14:35
check this out http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_sorting.html when records don't have meaningful score it starts smelling. – maximus ツ Jun 18 '14 at 14:37

score 1 · Answer 3 · edited May 23 '17 at 11:54

1

This question helped me, as the answer says,

One of the possible reasons could be distributed IDF, by default Elastic uses local IDF on each shard, to save some performance which will lead to different idfs across the cluster.

ES doc here

edited May 23 '17 at 11:54

Community

1
1

answered Apr 24 '17 at 09:41

shellbye

4,620
4
32
44

Different Elasticsearch results for the same query

3 Answers3

Linked