Is there a more elegant way to write these loops?

Question

I have a script that requires a handful of parameters to run. I'm interested in exploring the results as the parameters change, so I define a few scan arrays at the top, wrap the whole code in multiple for loops and set the parameters values to the current scan values.

This is error prone and inelegant. The process for changing the code is: 1) reset scan variables at the top, 2) comment out eg b = scan2(j2) and 3) uncomment b=b0.

What's a better method to allow variables to be set to arrays, and subsequently run the code for all such combinations? Example of my code now:

close all
clear all

%scan1 = linspace(1,4,10);
scan1 = 0;
scan2 = linspace(0,1,10);
scan3 = linspace(-1,0,10);

for j3 = 1:length(scan3)
  for j2 = 1:length(scan2)
    for j1 = 1:length(scan1)

      a = a0;
      %b = scan2(j2);
      b = b0;
      %c = c0;
      c = scan3(j3);
      d = scan2(j2);

      %(CODE BLOCK THAT DEPENDS ON variables a,b,c,d...)

    end

  end

end

There is no general rule at this point. The vectorization of a code depends pretty much on what exactly is going on in the loops, e.g. matrix multiplication, summation or something else. — freude, Jan 17 '16 at 14:54
Hmm. I'm not actually trying to vectorize this, I just want better organization. I could for example make a 1xN array for each parameter at the top, avoiding the commenting out business, but I'm a bit concerned about memory usage... and it seems unnecessary to have a triple for loop when often I only loop on 1 scan. — anon01, Jan 17 '16 at 14:55
I try to understand your code, is there a specific reason to enumerate the `scan` variables different than your parameters? Couldn't you rename them to `scana` `scanb` etc and the iterators to `ja` `jb` etc? — Daniel, Jan 17 '16 at 15:09
@Daniel That's a possibility, but then I'd need n nested loops, one for each a,b,c... — anon01, Jan 17 '16 at 15:11

score 4 · Accepted Answer · edited May 23 '17 at 11:50

4

Based on this idea to use one for loop to simulate multiple loops, I tried to adapt it to your case. While fulfilling a good memory efficiency and usability, this solution is slower than using individual for loops.

%define your parameters
p.a = 1;
p.b = linspace(1,4,4);
p.c = linspace(11,15,5);
p.d = linspace(101,104,4);
p.e = 5;
iterations=structfun(@numel,p);
iterator=cell(1,numel(iterations));
for jx = 1:prod(iterations)
    [iterator{:}]=ind2sub(iterations(:).',jx);%.'
    %This line uses itertor to extract the corresponding elemets of p and creates a struct which only contains scalars.
    q=cell2struct(cellfun(@(a,b)(a(b)),struct2cell(p),iterator(:),'uniform',false),fieldnames(p));
    %__ (CODE THAT DEPENDS ON q.a to q.e here) __

end

For the scenarios I tested it adds an computation overhead below 0.0002s per iteration which is 0.0002.*prod(iterations)s in total.

edited May 23 '17 at 11:50

Community

1
1

answered Jan 17 '16 at 15:44

Daniel

36,610
3
36
69

Very cool! I assume this is slower because of the `structure`? In my case, speed is not an issue for the wrapper loops: I have code inside that takes orders of magnitude more time, so this would be an option. – anon01 Jan 17 '16 at 15:48
Cell arrays, struct, calling ind2sub, all these three things add an overhead compared to a pure for loop of roughly 0.002s per iteration. – Daniel Jan 17 '16 at 15:54
1

Do you mean `ind2sub(iterations(:).',j)` instead of `ind2sub(iterations(:).',3)` in the first line of the loop? – Andras Deak -- Слава Україні Jan 17 '16 at 18:23

score 0 · Answer 2 · answered Jan 17 '16 at 15:17

0

One method is to make a single vector that contains all the parameter combinations, using ndgrid. For a sufficiently large parameter scans this may become a memory concern, but otherwise is at least much cleaner, requiring only a single loop and no re-assignments later in the code:

a0vec = 1;
b0vec = linspace(1,4,4);
c0vec = linspace(11,15,5);
d0vec = linspace(101,104,4);
e0vec = 5;

[a0s,b0s,c0s,d0s,e0s] = ndgrid(a0vec,b0vec,c0vec,d0vec,e0vec);    
N = numel(a0s);

for j = 1:N

    a0 = a0s(j);
    b0 = b0s(j);
    c0 = c0s(j);
    d0 = d0s(j);
    e0 = e0s(j);

    %__ (CODE THAT DEPENDS ON a0 - e0 here) __

end

Would still like to see your suggestions!

answered Jan 17 '16 at 15:17

anon01

10,618
8
35
58

Since your profile says `python`: how about using that language and a generator, to avoid storing all those values in memory?:) On a related note: what kinds of stuff goes on in the loop? Something complicated, or perhaps vectorizable? – Andras Deak -- Слава Україні Jan 17 '16 at 15:26
@AndrasDeak good suggestion! And a quick look suggests that matlab doesn't have generator-like syntax. As for memory, I just ran the numbers and 100x100x100 grid x 20 doubles takes up less than .2 gig, so this will work in my case. The code is ~1,000 lines, this is really a question of good coding practice! – anon01 Jan 17 '16 at 15:37
Indeed, I'm certain MATLAB can't support anything like generators. If your parameter space is always that small, your answer seems quite reasonable. – Andras Deak -- Слава Україні Jan 17 '16 at 16:50

Is there a more elegant way to write these loops?

2 Answers2

Linked