Is there a way to submit spark jobs via web ui?

Question

I switched from flink to spark and I am missing the option to submit jobs via web UI.

Until now I submitted all spark jobs per command line.

Is there a a way to submit jobs via the spark web interface?

Answer 1

Yes, there is one way you can trigger Spark job through a web application.

I tried this long back and I don't have exact documentation with me right now.

I will try to explain the steps as far as I remember, you can give a shot

You can use oozie to do the same

1. Create spark job and zip into a fat jar and place this into hdfs

2. Create shell script to submit this spark job and place this file into hdfs

3. Create job.properties and workflow.xml specific to oozie

4. Place workflow.xml file into hdfs

5. Submit oozie job using job.properties

6. Deploy UI application using which you want to trigger spark job into oozie/webapp/ROOT

7. Web application will be using oozie rest services to interact with the running oozie job

8. Now access your web application through browser and may be based on some button click you can submit the spark job

Update:

Found spark-jobserver library which helps to submit and monitor Apache Spark jobs through REST interface.

Hope this helps you...

As always, accept or upvote if my answer was helpful to you... :)

Answer 2

Is there a a way to submit jobs via the spark web interface?

No.

---

I would submit my job via the command line and then monitor it via the Spark Web Interface. Except of course if you would use oozie.