python simple parallel computation with joblib

Question

I took example from joblib tutorial. Here is how my code looks like:

from math import sqrt
from joblib import Parallel, delayed
import multiprocessing

test = Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in range(10))
print(test)

It produces the following error message:

Attempting to do parallel computing without protecting your import
on a system that does not support forking. To use parallel-computing 
in a script, you must protect you main loop using 
"if __name__ == '__main__'". Please see the joblib documentation on 
Parallel for more information

And it runs for too long. What am I missing?

Did you read the documentation that the error message is telling you to read? — BrenBarn, Aug 31 '15 at 19:26

score 4 · Accepted Answer · answered Aug 31 '15 at 19:45

What the error message and BrenBran are telling you is that a) you should read the error message, and b) you should organise your code something like:

from math import sqrt
from joblib import Parallel, delayed
import multiprocessing

if __name__ == '__main__':
    test = Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in range(10))
    print(test)

HTH barny

score 0 · Answer 2 · answered Mar 06 '18 at 18:47

0

I had to add backend="threading" to barny's code to run without error:

from math import sqrt
from joblib import Parallel, delayed
import multiprocessing

if __name__ == '__main__':
    test = Parallel(n_jobs=2, backend="threading")(delayed(sqrt)(i ** 2) for i in range(10))
print(test)

answered Mar 06 '18 at 18:47

P__2

66
5

1

I was reviewing the docs and if we use backend="threading" then it may lead to Global interpreter lock of python. – Aman Raparia Dec 07 '18 at 07:49

python simple parallel computation with joblib

2 Answers2