2

While submitting a Apache Spark batch job using Livy server,it uploads the jar file(containing application) everytime i.e., for every batch job submission.This seems to increase the job submission time.Is there a way to refer the jar present in the spark master local directory?

Community
  • 1
  • 1
Parithi
  • 51
  • 3

1 Answers1

0

I am not sure whether you are talking about uploading the jar files from Livy client to Livy server or from Livy server to Spark cluster. If that is from Livy client to server then I believe putting those jar files in /usr/lib/livy/repl_2.11-jars would do the job.

Averell
  • 793
  • 2
  • 10
  • 21