Sum each element across a structure containing oddly sized arrays [MATLAB]

Question

I've spent some time researching this, but am likely missing something simple. I have intermediate Matlab knowledge.

I have a structure of data saved from an experiment;

1x30 struct array with fields:
pixelStats: [118.6892 3.1370e+003 -0.1418 2.1195 -25.9308 259.3778]
  pixelLPStats: [5x2 double]
  autoCorrReal: [9x9x5 double]
   autoCorrMag: [4-D double]
      magMeans: [18x1 double]
 cousinMagCorr: [4x4x5 double]
 parentMagCorr: [4x4x4 double]
cousinRealCorr: [8x8x5 double]
parentRealCorr: [8x8x4 double]
   varianceHPR: 21.5116

Where each sub-field ('arrays'? terminology is likely incorrect I'm sorry) is a different size. There are 30 entries (trials) that I need to average across so that the shape of the above is retained, but the values for each cell within each field are averaged across the 30 trials. I also need the variance (I am trying to calculate Z scores to see if other trials are significantly different from this population).

First I tried to do this in a for loop, where I get the fieldnames, then for each field I break in and try to add the values for each element across the 30 trials, though this strategy doesn't work for fields with differently sized dimensions (e.g. I'd need to variably create differently sized for loops according to dimensions of current field).

Then I tried to convert the data structure to a cell array, which works;

cell_in = struct2cell(s_params_in)

cell_in = 10x1x30  607440  cell

...and then I tried something I found in my searching that uses cellfun, though to be honest I am not entirely sure of how it works. I had to add the UniformOutput to deal with unequal sized fields.

cellfun( @(cell_in) sum(cell_in(:)), cell_in,'UniformOutput',false)

This summed all values within each 'field' (first dimension), collapsing across elements within them, for all 30 trials. e.g.

ans(:,:,30) = 

[3.4911e+003]
[    10.9414]
[2.0854e+009]
[2.6877e+007]
[1.3612e+004]
[2.3328e+006]
[8.4917e+004]
[7.3826e+008]
[8.2038e+003]
[    22.4030]

I tried playing around with how I called the cell_in(:) part, but could not get the result I was looking for i.e. an array summed across the dimension for trials, whilst retaining the other dimensionality.

I have also tried to convert that to a matrix and then I could use something like

sum([data(:)])

And play around with reshaping to get the original size back, though that also seems difficult and I'd probably have to do it per fields and manually input the dimensions. The cell2mat doesn't work anyhow on account of the inconsistent dimensions.

So now I'm at the limit of my knowledge and am reaching out for help, can anyone solve this?

Cheers, Alex

score 1 · Accepted Answer · edited May 23 '17 at 11:57

A working code.

clear;clc

% create an example struct array
field1 = 'f1';  value1 = rand(1,10);
field2 = 'f2';  value2 = {rand(3,4,2), rand(3,4,2)};
field3 = 'f3';  value3 = {pi, pi.^2};
field4 = 'f4';  value4 = {magic(3), magic(3).^2};
S = struct(field1,value1,field2,value2,field3,value3,field4,value4);
clearvars field* value*

N = length(S);
T = struct();
U = T;

% enumerate over all fields (all should be numbers)
FN = fieldnames(S);
for ii=1:length(FN)
    fn = FN{ii};
    ss = {S.(fn)};
%     convert cell array of N-dim matrices into one (N+1)-dim matrix
    ssdim = ndims(ss{1});
    TT = cell2mat( reshape(ss, [ones(1,ssdim),N]) );
%     here you can do what you want
    T.(fn) = mean(TT,ndims(TT));
    U.(fn) = var(TT,0,ndims(TT));
end
clearvars ii ssdim ss fn TT

for ii=1:length(FN)
    disp(FN{ii})
    disp(T.(FN{ii}))
    disp(U.(FN{ii}))
end
clearvars ii N

The basic steps are addressed in code comments. TT is 30 trials of one particular type of experimental data. Say you are in the iteration for field pixelLPStats. Each trial has a matrix assigned by this name, which is a 5x2 double array. So, TT will be a 5x2x30 array. You can do what you want with it. Then let the loop bring you to the next field.

I think this time I got the right answer?

>> ( S(1).f2 + S(2).f2 )/2 == T.f2

ans(:,:,1) =

     1     1     1     1
     1     1     1     1
     1     1     1     1


ans(:,:,2) =

     1     1     1     1
     1     1     1     1
     1     1     1     1

>>

One more way to convert a cell array of N-dimension matrices into one (N+1)-dimension matrix - How to average over a cell-array of arrays?

Edit* struggling to fit in my resposne, I will post a reply to my original — Alex, Jul 21 '14 at 04:47

score 0 · Answer 2 · answered Jul 21 '14 at 05:01

@Yvon

Thanks for your reply, the code looks great and i have taken the time to go through it. I will be keeping this one on file as it has a few shortcuts I wasn't aware of. unfortunatelyit doesn't seem to have solved my problem, likely due to my inexperience and inability to manipulate your generous contribution to make it work for me. Grovelling aside, my results are as below

pixelStats: [1x6x30 double]
  pixelLPStats: [1x2x30 double]
  autoCorrReal: [1x9x150 double]
   autoCorrMag: [4-D double]
      magMeans: [1x1x30 double]
 cousinMagCorr: [1x4x150 double]
 parentMagCorr: [1x4x120 double]
cousinRealCorr: [1x8x150 double]
parentRealCorr: [1x8x120 double]
   varianceHPR: [1x1x30 double]

To begin with, the mean/variance parts don't seem to work when I run through the code step by step, it just populates that field with an mxmx30 array instead of an mxnx1 array of each element's mean. Also concerning is the dimensions of the above output. Notice that the dimension representing the 30 trials has been added within each field, and some it seems to have multiplied 30 by the last dimension. For example, comapre these fields, the top one is the original, the bottom is from your output:

cousinMagCorr: [4x4x5 double]

vs.

cousinMagCorr: [1x4x150 double]

notice the final dimension is multiplied by 30? I ran into this problem trying to loop through the fields and convert the cells to matrices; they worked for 2d, but not higher configurations, which came out like your output where the last dimension is multiplied by n trials.

it is likely I wasn't clear; I hope to get something like the very first entry in the structure, where all values within each field correspond to the mean for that values across al 30 trials. I am interested in each individual value, rather than collapsing across dimensions within a field.

Thanks again, Alex

If it still doesn't work, could you bring me an example of `autoCorrReal`? — Yvon, Jul 21 '14 at 05:06
I did a quick fix. I was wrong with the code converting cell arrays. Try the new code :) — Yvon, Jul 21 '14 at 05:09
Phenomenal! You've really gone above and beyond and I am very grateful. This bugged me over the weekend and is crucial to my experiments, and very time sensitive, hence me reaching out instead of learning. Thank you very much for you input. — Alex, Jul 21 '14 at 05:17

Sum each element across a structure containing oddly sized arrays [MATLAB]

2 Answers2