2

I've been wracking my brain over this for the last two days trying to get it to work. I have a local Spark installation on my Mac that I'm trying to attach a debugger to. I set:

SPARK_JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005

Then I submit my job to spark-submit and launch my debug configuration in eclipse which is configured as a Socket Attach Remote Debugging session. The debugger attaches, my job resumes and executes, but none of my breakpoints are ever hit, no matter what I do.

The only way I can get it to hit a breakpoint is by attaching to a spark-shell, creating a Java Exception breakpoint and issuing

throw new java.lang.Exception()

The debugger will not stop at normal breakpoints for me.

I created a standalone Hellow World scala app and was able to attach to it and have it stop at a regular breakpoint without any issues.

Environment: Mac OS, latest Eclipse, latest Scala IDE, Spark 1.3.1, Scala 2.10.5

Thanks in advance.

llevar
  • 755
  • 8
  • 24

1 Answers1

0

I had a similar issue and there were 2 things that fixed my problem - 1. The .jar file and source not a little out of sync for me , so had to recompile and redeploy. 2. Next On the JAVAOPTS I had a suspend=n. After correcting these two it worked for me .

Rahul J
  • 91
  • 1
  • 2