Elasticsearch high level REST client - Indexing has latency

Question

we have started using the high level REST client finally, to ease the development of queries from backend engineering perspective. For indexing, we are using the client.update(request, RequestOptions.DEFAULT) so that new documents will be created and existing ones modified.

The issue that we are seeing is, the indexing is delayed, almost by 5 minutes. I see that they use async http calls internally. But that should not take so long, I looked for some timing options inside the library, didn't find anything. Am I missing anything or the official documentation is missing for this?

Answer 1

Since refresh_interval: 1 in your index settings, it means it is never refreshed unless you do it manually, which is why you don't see the data just after it's been updated.

You have three options here:

A. You can call the _update endpoint with the refresh=true (or refresh=wait_for) parameter to make sure that the index is refreshed just after your update.

B. You can simply set refresh_interval: 1s (or any other duration that makes sense for you) in your index settings, to make sure the index is automatically refreshed on a regular basis.

C. You can explicitly call index/_refresh on your index to refresh it whenever you think is appropriate.

Option B is the one that usually makes sense in most use cases.

Answer 2

Several reference on using the refresh wait_for but I had a hard time finding what exactly needed to be done in the rest high level client. For all of you that are searching this answer:

   IndexRequest request = new IndexRequest(index, DOC_TYPE, id);
   request.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);