from joblib import Parallel, delayed
def func(arg1, arg2, arg3):
# do some processing and return result
return result
def func2():
arg1 = 'value1'
arg2 = 'value2'
elems = ['a','b','c','d','e']
resultsList = Parallel(n_jobs=4)(delayed(func)(arg1, arg2, elem) for elem in elems)
Calling func2
in a django view throws the following warning:
"Multiprocessing backed parallel loops cannot be nested below threads, setting n_jobs=1"
There's a related question, but the workaround here is to rename the thread name to 'MainThread', which seems odd and won't work in my case.
Is it possible to use joblib to parallelize for loops in django views? Is there a better way to do this?