how to get spark-sbumit logging result

Question

When i submit spark job too terminal, it's has logging result like image in terminal.

How can i get it and set it to value or object?

Please paste your logs as plain text in a code block, not as an image. — joanis, Sep 15 '22 at 13:42
And also please clarify your question, even looking at your image, I'm really not sure what you're asking. — joanis, Sep 15 '22 at 13:43

score 0 · Answer 1 · answered Sep 17 '22 at 15:58

0

You can pipe spark-submit command to the log file like this:

spark-submit ... > log.txt

answered Sep 17 '22 at 15:58

Leonsia

score 0 · Answer 2 · answered Sep 18 '22 at 10:28

0

You can see logs from application and you can always write logs to .text files

yarn logs -applicationId <application ID> [OPTIONS]

Application ID can be found through cluster UI

answered Sep 18 '22 at 10:28

kartheek

2 Answers2