5

I have this many hundreds of cell long Mathematica file and I want to use parallel evaluation. I have a 2 processor x 4 core each machine with 16 Gb memory. My Mathematica license allows me to run at most 2 master kernels with 1 of the master could have 4 slave kernels (this is my interpretation after I played with it for a while).

I used to run my code in two master kernels in two different notebooks. To speed up things further, I tried to encapsulate a few cells with ParallelEvaluate[] and it seemed to work. Then I also have 4 copies of my code running unaware of each other through one of the master kernels, which is fine. (I am basically trying to run as many copies of my code/mathkernel in parallel as possible. I am not shooting for anything truly parallel yet).

Since my code is too long and complicated, I do not want to edit every cell again to make them evaluate in parallel. Is there anything magical I can put in the beginning of my notebook so every cell evaluated after that will be by default ParallelEvaluate[ ... cell contents.... ]?

Smi
  • 13,850
  • 9
  • 56
  • 64
Hsn
  • 291
  • 2
  • 6
  • 1
    Are you really sure that you want to `ParallelEvaluate[]` all your cells? Did you already experiment with it on your code? Hint: You'll probably get lists where you are getting a single result now ... – Dr. belisarius Aug 04 '11 at 02:10
  • Yes I am getting lists of tables and that is ok. I will manually combine those tables in the master kernel and analyze the results. I am just trying to figure out the easiest way to use all mathkernels to speed up the computation. – Hsn Aug 04 '11 at 03:58
  • I was trying to experiment with $Pre=ParallelEvaluate;, but I can't use Mma right now (HW problems) perhaps you want to give it a try – Dr. belisarius Aug 04 '11 at 04:19
  • 1
    You could try `$Pre = If[Head@# =!= Unset, ParallelEvaluate@#, #, #] &;` and then `Unset[$Pre]` to reset ... sorry I can't test it – Dr. belisarius Aug 04 '11 at 05:17
  • $Pre=ParallelEvaluate does exactly what I want and it is the magical thing I was looking for. Now, when I do $Pre=Identity to turn it off so I can go back to my master kernel, mathematica still tries to evaluate that in slave kernels instead of master and fails. I ended up solving it as follows: SetSharedVariable[parallelcontrol]; parallelcontrol = ParallelEvaluate; $Pre := parallelcontrol; ... everything is evaluated in slaves here ... ; parallelcontrol = Identity; .... everything go back to be evaluated on master only ... Thanks again for your help. I edited my earlier comment already. – Hsn Aug 04 '11 at 05:20
  • But does the form @belisarius gave, `$Pre = If[Head@# =!= Unset, ParallelEvaluate@#, #, #] &;`, not automatically take care of the problem by explicitly not parallel evaluating `Unset[$Pre]`? – acl Aug 04 '11 at 11:09
  • If I use just $Pre=ParallelEvaluate; and run $KernelID, it gives me {1,2,3,4}. If I do $Pre = If[Head@# =!= Unset, ParallelEvaluate@#, #, #] &; and run $KernelID, it gives me {0,0,0,0}. So there is some problem with belisarius's second suggestion. What I do which is based on $Pre=ParallelEvaluate; works fine. – Hsn Aug 04 '11 at 21:12
  • @Hsn Post yourself that as an answer, and accept it if it works – Dr. belisarius Aug 07 '11 at 00:17

1 Answers1

2

As suggested by belisarius, $Pre=ParallelEvaluate does exactly what I want. One problem was when I do $Pre=Identity to turn it off so I can go back to my master kernel, mathematica still tries to evaluate that in slave kernels instead of master and fails. I ended up solving it as follows: SetSharedVariable[parallelcontrol]; parallelcontrol = ParallelEvaluate; $Pre := parallelcontrol; ... everything is evaluated in slaves here ... ; parallelcontrol = Identity; .... everything go back to be evaluated on master only ... Here is a sample run on my laptop which has 2 cores:

LaunchKernels[]

{KernelObject[1, "local"], KernelObject[2, "local"]}

$KernelID

0

ParallelEvaluate[$KernelID]

{1, 2}

SetSharedVariable[parcontrol]; $Pre := parcontrol; parcontrol = ParallelEvaluate

Null[ParallelEvaluate]

$KernelID

{1, 2}

parcontrol = Identity

{Identity, Identity}

$KernelID

0

parcontrol = ParallelEvaluate

ParallelEvaluate

$KernelID

{1, 2}

Hsn
  • 291
  • 2
  • 6