0

I have an algorithm myAlgo() which uses a parameter par1 in order to analyze a set of data (about 1000 .mat files). The path to the .mat files is some cell array I pass also to myAlgo(). The myAlgo() function contains classes and other functions. For every value of par1 I have to test all 1000 .mat files. So it would be a lot faster if I could use a parallel loop since I have an independent (?) problem.

I use the following code with parfor:

par1 = linespace(1,10,100);
myFiles % cell array with the .mat file location
myResult = zeros(length(par1),1);

parfor k=1:length(par1)
    myPar = par1(k);
    myResult(k) = myAlgo(myPar, myFiles);
end

% do something with myResult

.

function theResult = myAlgo(myPar, myFiles)
    for ii=1:length(myFiles)
        tempResult = initAlgo(myPar, myFiles(ii));
    end
    theResult = sum(tempResult);
end

So for every parameter in par1 I do the same thing. Unfortunately the processing time does not decrease. But if I check the workload of the CPU (i5), all cores are quiet active.

Now my question: Is it possible, that parfordoes not work in this case, because every worker initialized by parfor needs to access the folder with the 1000 .mat files. Therefore they can not do their job on the same time. Right? So is there a way handle this?

Gilfoyle
  • 3,282
  • 3
  • 47
  • 83

1 Answers1

1

First of all, check if you've got a license for the parallel computing toolbox (PCT). If you do not have one, parfor will behave just like a normal for loop WITHOUT actually parallel processing (for compatibility reasons)..

Second, make sure to open a parpool first.

Another problem may be that you are using parallel processing for the outer loop with 100 iterations, but not for the larger inner loop with 1000 iterations. You should rephrase your problem as one big loop that allows parfor to parallelize the 100*1000=100000 tasks, not just the 100 outer loops. This excellent post explains the problem nicely and offers several solutions.

Community
  • 1
  • 1
mzunhammer
  • 178
  • 1
  • 7
  • You are right. That makes it probably slower. But actually `parfor` does not even work in the example above. Is this due to the case, that every process needs access to the .mat files at the same time? – Gilfoyle Dec 17 '16 at 10:54
  • That should make it a lot slower. Did you try changing the loop structure ? To my knowledge parallel file reading should be no issue, according to this post it may even be faster in some cases: https://de.mathworks.com/matlabcentral/answers/20863-parfor-file-reading What do you mean with "does not even work"? Was there an error message? BTW Have you initialized parallel processing using parpool? – mzunhammer Dec 17 '16 at 17:50
  • There is no error. But if I measure the time of 'for' and 'parfor', than there is no significant difference. If I use an example of the MATLAB website which compares both loops, than there is a huge difference in computing time. So this works. But I can not initialize parallel processing using parpool. If I try that, I get an license error message. – Gilfoyle Dec 17 '16 at 18:56
  • Ok, that explains why, see answer above. – mzunhammer Dec 18 '16 at 11:36