Performance analysis using Sparklens of Spark Streaming Application

Question

I am trying to get performance analysis of a spark streaming application using sparklens. It is giving results like this

Executor count     1  ( 80%) estimated time 01m 29s and estimated cluster utilization 18.55%
Executor count     2  (100%) estimated time 01m 28s and estimated cluster utilization 9.36%
Executor count     3  (150%) estimated time 01m 28s and estimated cluster utilization 6.26%
Executor count     4  (200%) estimated time 01m 28s and estimated cluster utilization 4.69%
Executor count     6  (300%) estimated time 01m 28s and estimated cluster utilization 3.13%
Executor count     8  (400%) estimated time 01m 28s and estimated cluster utilization 2.35%
Executor count    10  (500%) estimated time 01m 28s and estimated cluster utilization 1.88%

Even though If I increase executors the estimated time is same.

Are these recommendations correct?

Answer 1

Why don't you try? Likely you are running a single task. Single task will not benefit from additional cores, unless you figure out a way to break it up into parallel tasks.