0

I have the following but I am unable to successfully submit a job through Livy in cluster mode.

here are my settings

spark-defaults.conf

spark.master yarn

livy.conf

livy.spark.master=yarn
livy.spark.deploy-mode = cluster
livy.server.recovery.mode = recovery
livy.server.recovery.state-store = zookeeper
livy.server.recovery.state-store.url = localhost:2181

Anything wrong with this conf?

I am using apache livy 0.4 and spark 2.3.0

user1870400
  • 6,028
  • 13
  • 54
  • 115
  • what is the error you get? and are you using a livy api or the command line – Kiran Balakrishnan Mar 29 '18 at 17:12
  • I don't see any errors although I am not sure if I am looking at the right logs. which log should I look at? I am using the java livy api and submitting jobs programatically – user1870400 Mar 29 '18 at 20:27
  • Check whether your Livy server is up and running..this would be normally @ port 8998 and you have mentioned the same host and port in the program.if starting from backend then it should be started using a Livy user.secondly if the above seems to be fine,check the resource manager as to whether the spark jobs are being aubmitted – Kiran Balakrishnan Mar 30 '18 at 09:50

0 Answers0