tidy function cannot be used within future_map?

Question

I have R code below. for the last row, when I used map() function, it worked well. however, when I changed to future_map() function, I got the following error message:

"Error: Problem with mutate() column model. i model = future_map(splits, fun1). x no applicable method for 'tidy' applied to an object of class "c('lmerMod', 'merMod')""

any idea on what's wrong? thanks.

fun1 <- function(data) {
    data %>% analysis %>%
       lmer(val ~ period + (1 | id), data = .) %>% tidy
 }
 
 plan(multisession)
 
 raw %>%
    nest(data = -c(analyte, var)) %>%
    mutate(boot = future_map(data, ~ bootstraps(
       data = .x,
       times = 5,
       strata = id
    ),
    .progress = T)) %>%
    unnest(boot) %>%
    mutate(model =future_map(splits, fun1))

homgran · Answer 1 · 2021-12-13T22:56:39.250

I experienced exactly the same problem with one of my scripts. In order to get future_map to work properly with tidy, I needed to explicitly reference the broom package (i.e. I needed to use broom::tidy in place of tidy). In your example, you are attempting to extract summary statistics from a mixed model, so the code should run without error if we modify fun1 to be as follows:

fun1 <- function(data) {
data %>% analysis %>%
   lmer(val ~ period + (1 | id), data = .) %>% broom.mixed::tidy
}

UPDATE (13-Dec-2021):

After a bit more reading, I now understand that the problem, as described in the original post, is due to the broom.mixed package not being attached in the R environment(s) where the future is evaluated. Instead of modifying fun1 (which is a very hacky way of resolving the problem), we should make use of the .options argument of future_map to guarantee that broom.mixed is attached (and all associated functions are available) in the future environments. The following code should run without error:

fun1 <- function(data) {
          data %>%
          analysis %>%
          lmer(val ~ period + (1 | id), data = .) %>%
          tidy
        }

plan(multisession)

raw %>%
  nest(data = -c(analyte, var)) %>%
  mutate(boot = future_map(data, ~ bootstraps(data = .x,
                                              times = 5,
                                              strata = id),
                           .progress = T)) %>%
  unnest(boot) %>%
  mutate(model = future_map(splits,
                            fun1,
                            .options = furrr_options(packages = "broom.mixed")))

My take-home from this is that it's probably good practice to always list the packages that we need to use (as a character vector) using the .options argument of future_map, just to be on the safe side. I hope this helps!

thank you for sharing your experience. I replaced "tidy" with "broom::tidy" in my code, but still saw the same error message. I agree with you that this may be bugs from furrr. anyway, thanks a lot. — YanLi, Dec 10 '21 at 18:21
I've just realised that, since you're extracting summary statistics from a mixed model, you'll need to use the `tidy` function from the broom.mixed package. Does your code work as expected if you replace `tidy` with `broom.mixed::tidy`? — homgran, Dec 12 '21 at 11:21
omg, yes, it worked when I used "broom.mixed::tidy". I am so happy to get this issue resolved! thank you so much!!!! — YanLi, Dec 13 '21 at 22:14
No problem at all -- happy to help. :) I've actually figured out a better way of fixing this, which doesn't require modifying the `fun1` function. There is an option for `future_map` that allows you to specify which packages must be loaded in each of the R environments. Let me update my answer with this new information. — homgran, Dec 13 '21 at 22:33

tidy function cannot be used within future_map?

1 Answers1