0

I have a small program in SparkSQL that in my local machine with Mac OS X uses 100% of 4 cpus (when using --master local[*] option). But when I execute the same code with the same option on a virtual machine based on Debian Jessie, it only uses 1 cpu. Looks like it is unable to assign more than 1 cpu, no matter what --master option you send.

I'm using spark-2.1.0-bin-hadoop2.7 with java 8 on both cases.

Is there any known issue about that?

Thanks in advance,

raimonbosch
  • 418
  • 1
  • 5
  • 14
  • how many core did you let your VM to use ? – dumitru Sep 06 '17 at 13:29
  • I our VM we have 4 cores available, @dumitru – raimonbosch Sep 06 '17 at 13:31
  • Seems that the performance is restricted because this disk is working via NFS. But it is curious that the restriction is exactly bound to 1 cpu. Shouldn't be a way to use all available performance althought you have some degree of degradation because the disk? – raimonbosch Sep 08 '17 at 09:41

0 Answers0