3

I am using Elasticsearch 7.9.0

I was updating the document very frequently. So I was getting the below exception

Elasticsearch exception [type=version_conflict_engine_exception, reason=[111]: version conflict, required seqNo [4348], primary term [2]. current document has seqNo [4427] and primary term [2]]

Then I have given a delay of 1 second between each update.(I can't give more then that)

But still the problem exists. How can we solve this. Please help me. Thanks.

Manonandan S K
  • 462
  • 1
  • 5
  • 17

2 Answers2

6

This issue happens because of the versioning of document in elasticsearch. This feature exists in order to prevent concurrent changes to the same documents by tasks that runs simultaneously. When you try to update a document that is already being updated by another task you might run into this issue.

If you want to track the update process of documents by your updates you may want to use the Task management API by elastic: https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html

Also you might want to check this documentation on Index API as it explains further: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html

Roy Levy
  • 640
  • 1
  • 11
  • 24
  • Yes. I understood the issue. In my case the document refresh interval is 1 sec (Default). So once The documnet get updated the version changes. If I try to update within that second it the version of the document is changed. so it will not update. That is the reason for giving the delay between update. – Manonandan S K Aug 18 '21 at 14:38
  • Here I am doing bulk update. say 1000 documents per query. and the second query may have some documents from first query. Here the issue comes. – Manonandan S K Aug 18 '21 at 14:39
  • Okay, still 1 second is not much time for the tasks to finish before updating again. Try investigating with the Tasks API that I've posted. Try writing a script that queries Task API every second to see if the tasks of update do finish the job. – Roy Levy Aug 18 '21 at 14:44
  • Let me know if you find out anything about the update timings – Roy Levy Aug 18 '21 at 15:32
  • 1
    Here you mean we have to check weather the document update process is completed or not by hitting another request to Elasticsearch if it is completed then we have to go for next iteration of updating the same document. else we ahve to wait till it completes. Is it right? – Manonandan S K Aug 19 '21 at 08:25
  • Exactly. Checking the in between update requests timing. – Roy Levy Aug 19 '21 at 09:38
  • Sounds good. But do you think will it consume more time? What is the maximum update time you have observed? – Manonandan S K Aug 19 '21 at 09:41
  • It will take the same amount of time but you won't get the error. Just when the update task finishes you can run the next update and so on. – Roy Levy Aug 19 '21 at 10:11
  • By the way if its just one document that you are updating you may want to consider just adding the same document with the changes and delete every now and then the accumulated documents. – Roy Levy Aug 19 '21 at 10:13
  • Thank you. And one more do you know how to do this? https://stackoverflow.com/questions/64114456/elasticsearch-high-level-java-client-version-6-0-1-to-implement-script-based-sor – Manonandan S K Aug 19 '21 at 10:34
  • I don't quite understand the script in that post - does the user need to return if the number was found in params? But if it doesn't find it doesn't return anything? Also you would need to use ctx syntax if you are willing to insert the field to the document. – Roy Levy Aug 19 '21 at 12:10
  • Yes. I am pretty familier with ctx syntax. There the question is he is using script inside sort. My doubt was not about the query syntax, but how to implement it in java rest client. Any way I am able to implement it. I will add the answer there. Thank you. – Manonandan S K Aug 20 '21 at 05:38
  • Not quite sure about this one, but seems to me that it would be related: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-put-stored-script.html – Roy Levy Aug 20 '21 at 07:12
0

I received nearly the same error in OpenSearch configuration but it wasn't due to too frequent updates like in OP's case.

In my case, I was unknowingly trying to update an existing Role in the domain. My requests were trying to create a 'new' Role when it already existed. When I tried to do this, I received the error.

My resolution was to create a Role with an entirely new name and then update that.

Wesley Cheek
  • 1,058
  • 12
  • 22