0

I recently discovered R's future package and have already successfully used it. However I am unable to use this package together with predictions for lasso regressions.

In the following toy example I first create some toy data, then fit a lasso regression and finally I make a prediction for new data. I do this procedure twice: once without the future package and then with the future package:

library(tidyverse)
library(glmnet)

set.seed(1)
a <- tibble(id = rep(letters[1:2], each = 100), x1 = rnorm(200), x2 = rnorm(200),
            y = 1 + 2*x2 + rnorm(200))

newdata <- as.matrix(data.frame(x1 = rnorm(200), x2 = rnorm(200)))

pred_func <- function(z) predict(z, newx = newdata)[,1]

b1 <- a %>% 
  group_by(id) %>% 
  nest(.key = data) %>% 
  mutate(lasso = map(data, function(z){glmnet(x = as.matrix(select(z, x1, x2)), y = z$y,  
                                                 intercept = T, alpha = 1)})) %>% 
  mutate(myprediction = map(lasso, pred_func))

This works just fine. Now with the future package:

library(future)

plan(multiprocess)
b2 <- a %>% 
  group_by(id) %>% 
  nest(.key = data) %>% 
  mutate(lasso = map(data, function(z){glmnet(x = as.matrix(select(z, x1, x2)), y = z$y,  
                                              intercept = T, alpha = 1)})) %>% 
  mutate(myprediction_future = map(lasso, ~future(pred_func(.x)))) %>% 
  mutate(myprediction = values(myprediction_future))

This terminates with an error (which is in german). Basically it tells me that it cannot use predict for an object of class elnet, glmnet.

I first thought that maybe there is a different predict function in the future package but this does not seem to be the case. Since I do not really understand what happens behind the curtains of the future functions I feel stuck with this one.

EDIT: here is the error message in english:

Error in mutate_impl(.data, dots) : 
Evaluation error: no applicable method for 'predict' applied to 
an object of class "c('elnet', 'glmnet')".
jay.sf
  • 60,139
  • 8
  • 53
  • 110
Cettt
  • 11,460
  • 7
  • 35
  • 58
  • 1
    Please type `Sys.setenv("Language" = "EN")` into console and rerun code, then paste error message in English into your question. – jay.sf Jul 26 '18 at 07:29
  • 1
    See [this vignette](https://cran.r-project.org/web/packages/future/vignettes/future-4-issues.html) under "Missing packages"? – Roland Jul 26 '18 at 07:43
  • @Roland. Thank you for pointing this out :) – Cettt Jul 26 '18 at 07:47
  • Note that with some experience in parallel computing with R, the root of the problem was obvious. It took me longer to look into the (unfamilar to me) future package than to identify the problem. I've seen similar errors with the parallel package. – Roland Jul 26 '18 at 07:51

1 Answers1

1

There's a predict.glmnet() in glmnet package.

Just define a new function.

pred_func2 <- function(z) predict.glmnet(z, newx = newdata)[,1]

And run.

plan(multiprocess)
b2 <- a %>% 
  group_by(id) %>% 
  nest(.key=data) %>% 
  mutate(lasso=map(data, 
                     function(z) {
                       glmnet(x=as.matrix(select(z, x1, x2)), y=z$y,  
                              intercept=TRUE, alpha=1)
                       })) %>% 
  mutate(myprediction_future=map(lasso, ~future(pred_func2(.x)))) %>% 
  mutate(myprediction=values(myprediction_future))

Yielding

> b2
# A tibble: 2 x 5
  id    data               lasso       myprediction_future      myprediction
  <chr> <list>             <list>      <list>                   <list>      
1 a     <tibble [100 x 3]> <S3: elnet> <S3: MultisessionFuture> <dbl [200]> 
2 b     <tibble [100 x 3]> <S3: elnet> <S3: MultisessionFuture> <dbl [200]>
jay.sf
  • 60,139
  • 8
  • 53
  • 110