0

I'm triying to submit Spark job to Oozie using Spark's SparkSubmit Java class as suggested in

launching a spark program using oozie workflow but Java action failed with error message

Main class [org.apache.oozie.action.hadoop.JavaMain], exit code [1]

Is there any way to access the execution error log / stack trace so that I can see what really happened? Any comment / answer is appreciated.

Community
  • 1
  • 1
Muqtafi Akhmad
  • 35
  • 1
  • 7

1 Answers1

3

To get to a java action log, you could use oozie's web console to find the hadoop job Id of that action. And then use Hadoop's Yarn WebUI to look into that hadoop job's mapper log.

With command line interface, the above steps would be:

  1. Run oozie cmd to get hadoop job id:

    oozie job -info <your job id here>

    and look for "External Id" in the output, such as : job_200904281535_0254

  2. Run yarn cmd to get hadoop job log:

    yarn logs -applicationId <your app id here>

    you can get app id by replacing "job" with "application" in job id, such as application_200904281535_0254

Paul H.
  • 1,074
  • 1
  • 8
  • 8