2

I was submitting the play application to spark 2.1 standalone cluster . In play application postgres dependency is also added and application works on local spark libraries. But at run time on standalone cluster it gives me error : 

o.a.s.s.TaskSetManager - Lost task 0.0 in stage 0.0 (TID 1, 172.31.21.3, executor 1): java.lang.ClassNotFoundException: org.postgresql.Driver

I have placed following in spark-defaults.conf directory

spark.executor.extraClassPath   /home/ubuntu/downloads/postgres/postgresql-9.4-1200-jdbc41.jar 
spark.driver.extraClassPath     /home/ubuntu/downloads/postgres/postgresql-9.4-1200-jdbc41.jar

Still executors unable to pick the driver. Am i missing something? Need help . Thanks.

Rishikesh Teke
  • 408
  • 3
  • 10

1 Answers1

1

place postgresql-9.4-1200-jdbc41.jar to your SPARK_DIRECTORY/jars folder and restart the spark cluster

learner
  • 344
  • 2
  • 22