I want to move from MATLAB to open-source alternatives as scipy and numpy. However, I have some problems with the speed. I am aware that sometimes multi-core operations can be slower than single core due to overheading, however, I am trying to do a process that was actually speeded-up by MATLAB.
I have a function that does some math on each pixel of an 2D matrix. I use 3 nested loops to do that.
def reconstruct2D(frame, parameters):
"""
Does some nested for loop operations on 2D data
"""
for channel_i in range(nr_cols): # for every channel
for y in range(nr_rows):
for x in range(nr_cols):
# Do some calculations here
what I normally did in MATLAB was to call this function for each 3rd dimnsion of a 3D matrix.
parfor frameNo = 1:N
result(:,:,frameNo) = reconstruct2D(rawFrame(:,:,frameNo), parameters);
end
Then it becomes four times faster, when 4 core is active. However, when I try the same thing by using Joblib, it still does it in an order.
import numpy as np
from scipy import signal
from joblib import Parallel, delayed
def reconstruct2D(frame, parameters):
# Same as above
if __name__ == '__main__':
print('Main Loop is running...')
Parallel(n_jobs=4)(delayed(reconstruct2D(frame[:, :, indx], parameters)) for indx in range(N))
print('Main Loop is finished...')
Processing time of a single frame is also much slower in Python. It takes 1.8s in MATLAB and 19s in Python.
I have two questions basically:
- Does anybody have an idea about the reason why a single frame processing is 10 times slower in Python?
- Why joblib calculates frames in an order and not concurrently?
I am using Python 3.5 in Windows7 64bits hardware with 4 cores.
