1

I'm developing a software based on scala-spark and I need to test it at my local machine first. Whenever I launch my class I get an output through Intellij IDEA and I would like to store it, since Intellij can't show the complete log, it shows only "a few" lines.

Thanks in advance.

D. RoDe
  • 33
  • 6

1 Answers1

2

Spark uses log4j for its logging. The default configuration logs to console. You need to pass it a different log4j.properties file, either by changing the existing file in the conf folder, or by passing it a different file via the option -Dlog4j.configuration=file:/path/to/log4j.properties.

You can also increase IntelliJ's console output buffer so it doesn't truncate.

Joeri Sebrechts
  • 11,012
  • 3
  • 35
  • 50