0

we have a spark process that takes around 22 min in Spark 2.2.1 on EMR 5.12.1 and took 7h (Yes, from 22 min to 7h!!) in Spark 2.3.0 on EMR 5.14.0 (same thing happens in 5.13.0). Both run with exactly the same data, same cluster size and same application, the only difference is Spark / EMR version, have anyone else seen something like this? Is there something i can provide to figure this out? This is what we get on the logs:

Container killed by YARN for exceeding memory limits. 6.6 GB of 6.6 GB physical memory used. Consider boosting spark.yarn.executor.memoryOverhead.

Of course already tried many variants of spark.executor.memory and spark.yarn.executor.memoryOverhead, but the best I can get is 1h42 min, really far from the 22 min. Ill really appreciate any help with this!! Thanks

Gustavo Pintore
  • 1
  • 2
  • Can you add Spark UI images for both https://stackoverflow.com/questions/40716849/spark-yarn-kills-containers-for-exceeding-memory-limits – vaquar khan Jun 22 '18 at 21:16
  • Hi @vaquarkhan, I dont have that information, we shutdown the cluster after running out taks, i just have some info from EMR console (for newer versions, it doesnt work in 5.12.1) Is that wat you want? – Gustavo Pintore Jun 22 '18 at 21:43
  • Ya looking info to reproduce issue – vaquar khan Jun 22 '18 at 21:50

0 Answers0