1

I have a simple question - what are the times at the top of WholeStageCodegen rectangles in the Spark UI? Is it a processing time?

enter image description here

bonnal-enzo
  • 1,165
  • 9
  • 19
mLC
  • 663
  • 10
  • 22

1 Answers1

1

Yes it is the sum of the computation time of tasks composing the step.

The 3 other values in parentheses are minimum, median and maximum values of computation time spent by the tasks.

  • the fastest task took 64ms to complete
  • if you order the tasks by computation time, the one in the middle took 147ms to complete
  • the slowest took 1.3 sec to complete
bonnal-enzo
  • 1,165
  • 9
  • 19