0

I am working on the Elastic Search (v7.10) and see that the statistic metric "indexing.index_failed" has increased. But I want to know the reasons why it failed.

In my application, I used the Rest High-level Client and have caught the exception. But I found that there were no exception have been thrown. So does the index fail affect the insertion?

private IndexResponse insertData(Data data) {
    try {
        IndexRequest indexRequest = new IndexRequest();
        indexRequest.index("index-name");
        indexRequest.id(data.getId());
        Map<String, Object> jsonMap = mapper.convertValue(data, new TypeReference<>() {});
        indexRequest.source(jsonMap);
        return esClient.index(indexRequest, RequestOptions.DEFAULT);
    } catch (Exception ex) {
        log.error("Error: Data={} ", data, ex);
    }
}

In my opinion, I think because the work loads on the ES is high, which cause the replication from primary shard is timeout. Have anyone investigate on this? Thank you!

I have checked these document & threads:

Phúc Đỗ Vương
  • 308
  • 1
  • 9
  • What does your catch clause log when that's the case? You should find the answer in your logs. Usually it can be the result of optimistic concurrency control issues, mapping issues or indexing rejections due to sending too many requests (i.e. HTTP 429) – Val Jul 19 '23 at 04:00
  • There weren't any log throw out. Just only the metric increase. I don't think the mapping is the problem, because I just insert to the same index. The weird thing is that the log does not show anything. – Phúc Đỗ Vương Jul 19 '23 at 04:05
  • There must be some error coming back as explained in [this issue](https://github.com/elastic/elasticsearch/issues/80802#issuecomment-973170819) – Val Jul 19 '23 at 06:25
  • Thank you, it seems like that problem is coming from the version conflict exception. After checking the Elastic Search log on version conflict and compare with the Grafana chart, I found that they are the same. – Phúc Đỗ Vương Jul 19 '23 at 06:42
  • Great, glad you figured it out. Make sure to provide an answer with your findings, as that might help other people ;-) – Val Jul 19 '23 at 06:53
  • Oh you can post your answer and I will mark it the correct answer for you. – Phúc Đỗ Vương Jul 19 '23 at 12:08

1 Answers1

1

Usually, indexing failures are the result of optimistic concurrency control issues, mapping issues, bad values in your documents (keyword length exceeded, bad date format, etc) or indexing rejections due to sending too many requests (i.e. HTTP 429).

An open issue shows one of the problem that could cause this, and this error should be seen in your error logs on the client side (i.e. your catch section).

Val
  • 207,596
  • 13
  • 358
  • 360