Cumulative sum over index in MATLAB

Question

Consider the following matrix, where the first column is the index, the second - is values, the third - is the cumulative sum which resets once the index changes:

1     1     1     % 1   
1     2     3     % 1+2
1     3     6     % 3+3
2     4     4     % 4
2     5     9     % 4+5
3     6     6     % 6
3     7    13    % 6+7
3     8    21    % 13+8
3     9    30    % 21+9
4    10    10    % 10
4    11    21    % 10+11

How can one get the third column avoiding loops?

I try the following:

  A = [1 1;...                 % Input
       1 2;...
       1 3;...
       2 4;...
       2 5;...
       3 6;...
       3 7;...
       3 8;...
       3 9;...
       4 10;...
       4 11];
  CS = cumsum(A(:,2));         % cumulative sum over the second column

  I = [diff(data(:,1));0];     % indicate the row before the index (the first column)  
                               % changes
  offset=CS.*I;                % extract the last value of cumulative sum for a given 
                               % index

  offset(end)=[]; offset=[0; offset] %roll offset 1 step forward

  [A, CS, offset]

The result is:

ans =

 1     1     1     0
 1     2     3     0
 1     3     6     0
 2     4    10     6
 2     5    15     0
 3     6    21    15
 3     7    28     0
 3     8    36     0
 3     9    45     0
 4    10    55    45
 4    11    66     0

So the problem would have been solved, if there were a trivial way to transform the fourth column of the matrix above into

O =

 0
 0
 0
 6
 6
15
15
15
15
45
45

Since CS-O gives the desired output.

I would appreciate any suggestions.

Answer 1

cumsum and diff based method and as such might be good with performance -

%// cumsum values for the entire column-2
cumsum_vals = cumsum(A(:,2));

%// diff for column-1
diffA1 = diff(A(:,1));

%// Cumsum after each index
cumsum_after_each_idx = cumsum_vals([diffA1 ;0]~=0);

%// Get cumsum for each "group" and place each of its elements at the right place
%// to be subtracted from cumsum_vals for getting the final output
diffA1(diffA1~=0) = [cumsum_after_each_idx(1) ; diff(cumsum_after_each_idx)];

out = cumsum_vals-[0;cumsum(diffA1)];

---

Benchmarking

If you care about performance, here are some benchmarks against the other solutions based on accumarray.

Benchmarking code (with comments removed for compactness) -

A = ..  Same as in the question

num_runs = 100000; %// number of runs

disp('---------------------- With cumsum and diff')
tic
for k1=1:num_runs
    cumsum_vals = cumsum(A(:,2));
    diffA1 = diff(A(:,1));
    cumsum_after_each_idx = cumsum_vals([diffA1 ;0]~=0);
    diffA1(diffA1~=0) = [cumsum_after_each_idx(1) ; diff(cumsum_after_each_idx)];
    out = cumsum_vals-[0;cumsum(diffA1)];
end
toc,clear cumsum_vals  diffA1 cumsum_after_each_idx out

disp('---------------------- With accumarray - version 1')
tic
for k1=1:num_runs
    result = accumarray(A(:,1), A(:,2), [], @(x) {cumsum(x)});
    result = vertcat(result{:});
end
toc, clear result

disp('--- With accumarray - version 2 (assuming consecutive indices only)')
tic
for k1=1:num_runs
    last = find(diff(A(:,1)))+1; %// index of last occurrence of each index value
    result = A(:,2); %// this will be cumsum'd, after correcting for partial sums
    correction = accumarray(A(:,1), A(:,2)); %// correction to be applied for cumsum
    result(last) = result(last)-correction(1:end-1); %// apply correction
    result = cumsum(result); %// compute result
end
toc, clear last result correction

disp('--- With accumarray - version 2 ( general case)')
tic
for k1=1:num_runs
    last = find(diff(A(:,1)))+1; %// index of last occurrence of each index value
    result = A(:,2); %// this will be cumsum'd, after correcting for partial sums
    correction = accumarray(A(:,1), A(:,2), [], @sum, NaN); %// correction
    correction = correction(~isnan(correction)); %// remove unused values
    result(last) = result(last)-correction(1:end-1); %// apply correction
    result = cumsum(result);
end
toc

Results -

---------------------- With cumsum and diff
Elapsed time is 1.688460 seconds.
---------------------- With accumarray - version 1
Elapsed time is 28.630823 seconds.
--- With accumarray - version 2 (assuming consecutive indices only)
Elapsed time is 2.416905 seconds.
--- With accumarray - version 2 ( general case)
Elapsed time is 4.839310 seconds.

Answer 2

Use accumarray with a custom function:

result = accumarray(A(:,1), A(:,2), [], @(x) {cumsum(x)});
result = vertcat(result{:});

This works irrespective of index changes being by a step of 1 (as in your example) or not.

---

The following approach is faster as it avoids cells. See @Divakar's excellent benchmarking in his answer (and see his solution, which is the fastest):

1. If index changes always correspond to an increase by 1 (as in your example):

   last = find(diff(A(:,1)))+1; %// index of last occurrence of each index value
   result = A(:,2); %// this will be cumsum'd, after correcting for partial sums
   correction = accumarray(A(:,1), A(:,2)); %// correction to be applied for cumsum
   result(last) = result(last)-correction(1:end-1); %// apply correction
   result = cumsum(result); %// compute result
   

2. If the index value can change by more than 1 (i.e. there may be "skipped" values): this requires a small modification that slightly slows things down.

   last = find(diff(A(:,1)))+1; %// index of last occurrence of each index value
   result = A(:,2); %// this will be cumsum'd, after correcting for partial sums
   correction = accumarray(A(:,1), A(:,2), [], @sum, NaN); %// correction
   correction = correction(~isnan(correction)); %// remove unused values
   result(last) = result(last)-correction(1:end-1); %// apply correction
   result = cumsum(result);
   

Answer 3

Your strategy is actually what I may have done. Your last step could be achieved this way: (Remember however that your approach assumes consecutive indices. You could of course change this via offset=[0; CS(1:end-1).*(diff(A(:,1))~=0)];, but would still need sorted indices.)

I = find(offset);
idxLastI = cumsum(offset~=0);
hasLastI = idxLastI~=0; %// For the zeros at the beginning
%// Combine the above to the output
O = zeros(size(offset));
O(hasLastI) = offset(I(idxLastI(hasLastI)));
out = CS-O;

This should be comparable to Divakar's cumsum-diff approach.