5

I would like to generate covariance matrices (and mean vectors) using a rolling window. But in all my attempts rollapply stacks the covariance matrices from cov and runs out of pre-allocated space (e.g., if my original data have 40 observations, then rollapply can't return more than 40 rows).

Is there a way that I can get rollapply to return a list of matrices? Or to return a data.frame that is larger than the original data.frame, which I can manually split into a list? My end goal is to take a panel, split the panel into a list of individual data.frames, calculate the rolling covariances and means for each data frame, then use these lists of covariances and means downstream to compare to a bunch of individuals.

Here is some code. My problem is that my.fun won't return data from all covariance matrix caluclations. Is my best option to code my own rollapply? Or my own cov that returns a vector that I convert back to a matrix? Thanks!

library("zoo")
data.df <- data.frame(sic = rep(1:10, each = 40),
                      year = rep(1:40, len = 10*40),
                      one = rnorm(10*40),
                      two = 2*rnorm(10*40),
                      three = 3*rnorm(10*40))
data.list <- split(data.df, data.df$sic)
data.list <- lapply(data.list, zoo)
my.fun <- function(x) {
    x <- x[, c("one", "two", "three")]
    rollapply(x,
              width = 10, 
              FUN = cov,
              by.column = F, 
              align = "right")
}
cov.list <- lapply(data.list, FUN = my.fun)
Richard Herron
  • 9,760
  • 12
  • 69
  • 116
  • Can you try to make your goal more clear? Covariances between what exactly? It doesn't appear in the code And how many groups did you imagine that data.list has? This produces a zoo-series: `my.fun(data.list[[1]])`. Is that what you expected from the sic==1 group? – IRTFM Mar 30 '12 at 17:59
  • @DWin I would like a covariance matrix for each sic for each year based on a 10-year moving window. There is nothing sacred about having a `zoo` object here, I was just already familiar with how to use a `rollapply` to generate a scalar. – Richard Herron Mar 30 '12 at 18:43

3 Answers3

2

After glancing at the rollapply.zoo code, I don't think there's a way to make it do what you want. Rolling your own function isn't that difficult though (pun intended).

rollcov <- function(x, width=10) {
  len <- NROW(x)
  add <- rep(1:(len-width)-1,each=width)
  seq.list <- split(rep(1:width,len-width)+add, add)
  lapply(seq.list, function(y) cov(x[y,]))
}

rollcov(data.list[[1]][,c("one","two","three")],10)
all <- lapply(data.list, function(x) rollcov(x[,c("one","two","three")],10))
Joshua Ulrich
  • 173,410
  • 32
  • 338
  • 418
  • That does it! Thanks! I also realized that I can trick `rollapply` into returning a vector, then bend that vector back into a matrix (I'll add it as an answer). – Richard Herron Mar 30 '12 at 18:47
2

I realized that I can trick rollapply into returning a vector, then bend that vector back into a matrix. The trick is using alply from the plyr package to bend the vector back into a matrix. 

library("plyr")
library("zoo")
data.df <- data.frame(sic = rep(1:10, each = 40),
                      year = rep(1:40, len = 10*40),
                      one = rnorm(10*40),
                      two = 2*rnorm(10*40),
                      three = 3*rnorm(10*40))
data.list <- split(data.df, data$sic)
data.list <- lapply(data.list, zoo)
my.fun <- function(x) {
    x <- x[, c("one", "two", "three")]
    rollapply(x,
              width = 10, 
              function(x) as.vector(cov(x)),
              by.column = F, 
              align = "right")
}
my.fun.2 <- function(x) {
    alply(x, 1, matrix, nrow = sqrt(ncol(x)))
}
cov.list <- lapply(data.list, FUN = my.fun)
cov.list.2 <- lapply(cov.list, my.fun.2)

Or, for a list of arrays I can replace alply with aaply.

Richard Herron
  • 9,760
  • 12
  • 69
  • 116
2

Remove the second data.list<- and modify my.fun so that given data.df we have the following (which is reasonably close to the original):

data.list <- split(data.df, data.df$sic)
my.fun <- function(x) {
    z <- zoo(x[, c("one", "two", "three")], x$year)
    ccov <- function(x) c(cov(x))
    r <- rollapplyr(z, width = 10, FUN = ccov, by.column = FALSE)
    toMat <- function(tt) matrix(r[tt], ncol(z))
    sapply(paste(time(r)), toMat, simplify = FALSE) # sapply(char,...) adds names
}
cov.list <- lapply(data.list, FUN = my.fun)

EDIT: Slight simplification.

G. Grothendieck
  • 254,981
  • 17
  • 203
  • 341
  • Awesome. Thanks! I couldn't figure out how to get around `alply` because I was fixed on operating over and over again on the same list. This is nice. – Richard Herron Mar 30 '12 at 19:32