I have the next function that is called from a separated code and I want to run it in parallel using multiprocessing:
def GridParallel(vec_main, vec_egf, vec_lp):
VR_list = []
Synthetics = []
for i in vec_main:
'''
List with VR vectors for each iteration of egf values, the size of the list is
given by the len of vec_egf
'''
temp_VR = []
temp_Synth = []
for j in vec_egf:
for k in vec_lp:
synth = Synthetic(Frequency=frequency_vector, LongPeriod_amp=k,
Corner_egf=j , Corner_main=i,
gamma=gamma, fall_off=fall_off)
VR = Variance_Reduction(data=data, model=synth)
temp_VR.append(VR)
temp_Synth.append(synth)
VR_list.append(temp_VR)
Synthetics.append(temp_Synth)
VR_array = np.asarray(VR_list)
VR_max_values = np.amax(VR_array, axis=1)
# Corner index
corner_index = VR_max_values.argmax()
#corner_index = np.nonzero(VR_max_values == np.max(VR_max_values))
# Best corner frequency for the main event
Corner_main_shock = float(vec_main[corner_index])
# Variace reduction corresponding to the main event
Best_fit = float(VR_max_values[corner_index])
# Indexing for extracting spectral ratios in gridsearch
VR_max_values_index = VR_array.argmax(axis=1)
Synthetics = np.asarray(Synthetics)
N_Synthetics = []
rows, cols, lenn = Synthetics.shape
temp = np.arange(0, rows, 1)
for ii in temp:
spec = Synthetics[ii, VR_max_values_index[ii]]
N_Synthetics.append(spec)
N_Synthetics = np.asarray(N_Synthetics)
return N_Synthetics, VR_max_values, Corner_main_shock, Best_fit
However, when I run:
pool = mp.Pool(processes=Ncores)
results = pool.apply_async(GridParallel, args=(vec_main, vec_egf, vec_lp, ))
pool.close()
results_ = results.get()
pool.join()
I get the next error:
AttributeError: Can't pickle local object 'GridSearchCorner.<locals>.GridParallel'
The input variables: vec_* are all numpy ndarrays (1d).
Also, how I can recover the return objects from the function?
I am trying to understand multiprocessing, so I apologize in advance if the question is too simple. Any help will be appreciated. For this I am working on python 3.5.
Thanks!!