I am submitting my spark job on yark using unix scripts and spark-submit commands. I am checking the status of spark job in if else block of unix and throwing error, however i observed that if spark job failed or incomplete on yarn still it shows as successful and unix script is not able to get the correct status of the job and exit the script.
i checked the other answers on Stack overflow but couldn't get the desired one.
Can anyone please help if implemented it successfully.
