EMR on EKS | Spark job retries 5 times if failed

Asked Mar 21 '23 at 14:21

Active Mar 21 '23 at 14:27

Viewed 71 times

I am using the release: emr-5.33.0-latest

Whenever a spark job fails, it tries to restart 5 times. I am not able to find where this behavior is configured or if its something default, and how to disable this.

I also tried the below by using 6.9.0 emr release. https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/jobruns-using-retry-policies.html

Even after configuring

"retryPolicyConfiguration": {
    "maxAttempts": 1
}

It retried it 5 times if it failed.

Does anyone know how to debug this and disable this behaviour?

Thanks

edited Mar 21 '23 at 14:27

asked Mar 21 '23 at 14:21

hsnsd

1,728
12
30

What kind of job is it? There a variety of technologies supported on EMR. For example, Spark has its own configuration, including retries. – Aleksey Tsalolikhin Mar 21 '23 at 14:25
1

@AlekseyTsalolikhin my bad. Its a spark job – hsnsd Mar 21 '23 at 14:27
See https://stackoverflow.com/questions/38709280/how-to-limit-the-number-of-retries-on-spark-job-failure – Aleksey Tsalolikhin Mar 21 '23 at 14:30
You could try setting `spark.task.maxFailures`. See https://spark.apache.org/docs/latest/configuration.html – Aleksey Tsalolikhin Mar 21 '23 at 14:55

EMR on EKS | Spark job retries 5 times if failed

0 Answers0