How to resample and remodel n times by vectorization?

Question

here's my for loop version of doing resample and remodel,

B <- 999
n <- nrow(butterfly)
estMat <- matrix(NA, B+1, 2)
estMat[B+1,] <- model$coef

for (i in 1:B) {
resample <- butterfly[sample(1:n, n, replace = TRUE),]
re.model <- lm(Hk ~ inv.alt, resample)
estMat[i,] <- re.model$coef
}

I tried to avoid for loop,

B <- 999
n <- nrow(butterfly)

resample <- replicate(B, butterfly[sample(1:n, replace = TRUE),], simplify = FALSE)
re.model <- lapply(resample, lm, formula = Hk ~ inv.alt)
re.model.coef <- sapply(re.model,coef)
estMat <- cbind(re.model.coef, model$coef)

It worked but didn't improve efficiency. Is there any approach I can do vectorization?

Sorry, not quite familiar with StackOverflow. Here's the dataset butterfly.

colony  alt precip  max.temp    min.temp    Hk
pd+ss   0.5 58  97  16  98
sb  0.8 20  92  32  36
wsb 0.57    28  98  26  72
jrc+jrh 0.55    28  98  26  67
sj  0.38    15  99  28  82
cr  0.93    21  99  28  72
mi  0.48    24  101 27  65
uo+lo   0.63    10  101 27  1
dp  1.5 19  99  23  40
pz  1.75    22  101 27  39
mc  2   58  100 18  9
hh  4.2 36  95  13  19
if  2.5 34  102 16  42
af  2   21  105 20  37
sl  6.5 40  83  0   16
gh  7.85    42  84  5   4
ep  8.95    57  79  -7  1
gl  10.5    50  81  -12 4

In the data you showed, there is `alt` column, is that the same as `inv.alt`? — akrun, Oct 30 '18 at 00:23
"No for loop" != "vectorized". I guess the difference in running time between for-loop version and `lapply` version would not be too large. — mt1022, Oct 30 '18 at 01:35

score 1 · Accepted Answer · answered Oct 30 '18 at 03:36

(Assuming butterfly$inv.alt <- 1/butterfly$alt)

You get the error because resample is not a list of resampled data.frames, which you can obtain with:

resample <- replicate(B, butterfly[sample(1:n, replace = TRUE),], simplify = FALSE)

The the following should work:

re.model <- lapply(resample, lm, formula = Hk ~ inv.alt)

To extract coefficients from a list of models, re.model$coef does work. The correct path to coefficients are: re.model[[1]]$coef, re.model[[2]]$coef, .... You can get all of them with the following code:

re.model.coef <- sapply(re.model, coef)

Then you can combined it with the observed coefficients:

estMat <- cbind(re.model.coef, model$coef)

In fact, you can put all of them into replicate:

re.model.coef <- replicate(B, {
    bf.rs <- butterfly[sample(1:n, replace = TRUE),]
    coef(lm(formula = Hk ~ inv.alt, data = bf.rs))
})
estMat <- cbind(re.model.coef, model$coef)

Thanks, it does fix the error. But my ultimate intention is to improve efficiency, now I know lapply is not vectorization. — ladychili, Oct 30 '18 at 15:00
@LadyChili, this might be helpful: https://stackoverflow.com/questions/25416413/is-there-a-faster-lm-function. — mt1022, Oct 31 '18 at 01:49

How to resample and remodel n times by vectorization?

1 Answers1