Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://example.us-east-1.es.amazonaws.com:443/_bulk"}

Question

Logstash Config file

input {
 elasticsearch {
 hosts => ["https://staing-example.com:443"]
 user => "userName"
 password => "password"
 index => "testingindex"
 size => 100
 scroll => "1m"
 }
}

filter {

}


output {
 amazon_es {
 hosts => ["https://example.us-east-1.es.amazonaws.com:443"]
 region => "us-east-1"
 aws_access_key_id => "access_key_id"
 aws_secret_access_key => "access_key_id"
 index => "testingindex"
}
}

Using Logstash to transferred from one elastic search server to amazon elastic search

For the above config, Logstash is continuously throwing

2019-10-10T16:00:51,232][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://example.us-east-1.es.amazonaws.com:443/_bulk"} [2019-10-10T16:00:52,127][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://example.us-east-1.es.amazonaws.com:443/_bulk"} [2019-10-10T16:00:52,317][ERROR][logstash.outputs.elasticsearch] Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://example.us-east-1.es.amazonaws.com:443/_bulk"}

Don't know the reason why it is happening

run logstash with --debug flag and post results – dejanmarich Oct 10 '19 at 11:00 — dejanmarich, Oct 10 '19 at 11:00

Thilak · Accepted Answer · 2019-11-18T08:32:17.903

2

Below solution solves the issue for me Click on the configured cluster in Amazon Elastic Search configure cluster

Now click on Advanced Option Where you can able to see the Allow APIs that can span multiple indices and bypass index-specific access policies - check in the policy Advance Option

edited Nov 18 '19 at 08:32

answered Nov 15 '19 at 07:24

Thilak

126
2
12

This didn't work for me as the cluster was configured like so from the get go...here are other things to consider with the original elasticsearch plugin: https://discuss.elastic.co/t/error-elasticsearch-encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400/138267/501717 – eco Jun 04 '21 at 16:06

Encountered a retryable error. Will Retry with exponential backoff {:code=>400, :url=>"https://example.us-east-1.es.amazonaws.com:443/_bulk"}

1 Answers1