1

I am trying to derive a function for calculating a moving/rolling correlation for two vectors and speed is a high priority, since I need to apply this function in an array function. What I have (which is too slow) is this:

Data1 = rand(3000,1);
Data2 = rand(3000,1); 

function y = MovCorr(Data1,Data2)

[N,~] = size(Data1);

correlationTS = nan(N, 1);

for t = 20+1:N
    correlationTS(t, :) = corr(Data1(t-20:t, 1),Data2(t-20:t,1),'rows','complete');
end
    y = correlationTS;
end

I am thinking that the for loop could be done more efficiently if I knew how to generate the roling window indices and then applying accumarray. Any suggestions?

Mace
  • 1,259
  • 4
  • 16
  • 35
  • 1
    Isn't this similar to convolving the elementwise product with `ones(1,21)`? As I'm not perfectly familiar with it: do you have a formula for the *moving correlation*? – knedlsepp Feb 20 '15 at 09:37
  • Not sure what you mean by " the elementwise product with ones(1,21)"? The function above `MovCorr` produces the right values. It is just too slow. It produces a 21 observations moving correlation between the two vectors, Data1 and Data2. – Mace Feb 20 '15 at 09:42
  • BTW: Your above code doesn't even run (because of the `Data1(.,2)`) – knedlsepp Feb 20 '15 at 09:42
  • Argh, sorry.... It should work now – Mace Feb 20 '15 at 09:43
  • Also: It's 22 observations. – knedlsepp Feb 20 '15 at 09:46
  • Ah, yes, have changed it now. Thanks. I want (3000-20) correlations. These should be calculated on moving data... – Mace Feb 20 '15 at 09:46
  • I guess you should be able to hack this together using [movingstd](http://www.mathworks.com/matlabcentral/fileexchange/9428-movingstd-x-k-windowmode-) from the file-exchange and a moving average filter. – knedlsepp Feb 20 '15 at 10:09
  • I dont have the code now, but you should check http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance. Section : Online algorithm. This shows how to calculate mean and variance of a variable when a new variable is introduced. You should be able to calculate updated mean and variance by removing last value and inserting new value in two lines. Extend it to formula of correlation. The result will not be pretty but it will fast. – lonstud Feb 20 '15 at 10:11
  • @knedlsepp I have looked at movingstd from the file-exchange, and it seems very fast indeed. Unfortunately, I have a hard time understanding what filter does, but it seems like a good way to go... – Mace Feb 20 '15 at 13:10

1 Answers1

3

Following the advice from @knedlsepp, and using filter as in the movingstd, I found the following solution, which is quite fast:

function Cor = MovCorr1(Data1,Data2,k)
y = zscore(Data2);
n = size(y,1);

if (n<k)
    Cor = NaN(n,1);
else
    x = zscore(Data1);
    x2 = x.^2;
    y2 = y.^2;
    xy = x .* y;
    A=1;
    B = ones(1,k);
    Stdx = sqrt((filter(B,A,x2) - (filter(B,A,x).^2)*(1/k))/(k-1));
    Stdy = sqrt((filter(B,A,y2) - (filter(B,A,y).^2)*(1/k))/(k-1));
    Cor = (filter(B,A,xy) - filter(B,A,x).*filter(B,A,y)/k)./((k-1)*Stdx.*Stdy);
    Cor(1:(k-1)) = NaN;
end
end

Comparing with my original solution the execution times are:

tic
MovCorr(Data1,Data2);
toc
Elapsed time is 1.017552 seconds.

tic
MovCorr1(Data1,Data2,21);
toc
Elapsed time is 0.019400 seconds.
Mace
  • 1,259
  • 4
  • 16
  • 35