3

I just ran across an unexpected behaviour in Intelli J IDEAs (v.2016.2) "Evaluate Expression" window:

While this works, both as compiled code and using the "Evaluate Expression" window

  val df1 = spark.read.json("/examples/src/main/resources/people.json")
  df1.columns.foreach(println) // outputs "age" and "name" to console

Another Dataset[Row] (DataFrame) that involves some Columns generated by expressions only works as compiled code, while in the "Evaluate Expression" window the following exception is displayed:

 com.intellij.debugger.engine.evaluation.EvaluateException: 
 Type mismatch Can't assign org.apache.spark.sql.Dataset to scala.runtime.ObjectRef

Both dataframes are of class class org.apache.spark.sql.Dataset. According to Intelli Js documentation the only limitations are:

  • A method can be invoked within the Expression Evaluation dialog only if the debugger has stopped at a breakpoint, but has not been paused.
  • Expression Evaluation can only be "single-level". In other words, if IntelliJ IDEA stops at a breakpoint within a method called from the Expression Evaluation, you cannot use the Expression Evaluation feature again.
  • If a method invoked within Expression Evaluation has a breakpoint inside its body, this breakpoint will be ignored.

That is apparently only half the truth.

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
Boern
  • 7,233
  • 5
  • 55
  • 86
  • Did you ever solve this? Running into this error now, though I swear it worked for me a few days ago. – Nick Resnick Jan 08 '19 at 15:33
  • Nope, I'm sorry. Since was back in '16 maybe this issue was fixed with a newer version? – Boern Jan 08 '19 at 15:36
  • Maybe that link can explain. https://stackoverflow.com/questions/51634079/intellij-evaluate-expression-results-in-nosuchmethodexception, I have same issue also – Mertalp Tasdelen Apr 07 '21 at 19:50

0 Answers0