1

In my stderr logs of EMR Step, I see the state changing from ACCEPTED to RUNNING but after some time it changes again to ACCEPTED. Does this mean another application has been submitted to the same cluster or there is issue with the code running in the step? Please note that this does not happen with every execution of the same code. At times it does change from RUNNING to FINISHED state. How can I avoid this difference in the behavior of the same job with multiple executions? How to make sure that the step moves from RUNNING to FINISHED always and does not go back to ACCEPTED? Sorry if the description is not much clear. Also I am not able to get much help on the internet search. Sample logs are shown below:

22/11/15 03:48:23 INFO Client: Application report for application_1668478752419_0001 (state: RUNNING)
22/11/15 03:48:24 INFO Client: Application report for application_1668478752419_0001 (state: RUNNING)
22/11/15 03:48:25 INFO Client: Application report for application_1668478752419_0001 (state: RUNNING)
22/11/15 03:48:26 INFO Client: Application report for application_1668478752419_0001 (state: RUNNING)
22/11/15 03:48:27 INFO Client: Application report for application_1668478752419_0001 (state: ACCEPTED)
seou1
  • 446
  • 1
  • 5
  • 21
  • This usually happens when your application fails, and that is in retries. Can you check your `spark.yarn.maxAppAttempts`? – Thiago Baldim Nov 15 '22 at 05:15
  • yeah, no retries set. Thanks – seou1 Nov 15 '22 at 14:53
  • One thing to add, I do start new EMR containers to run the same job while one container is running. But I think this should not impact as they must be independent. – seou1 Nov 15 '22 at 15:02

0 Answers0