Progress bar within drake functions

Question

I'm trying to implement progress bars within function for use in a drake-r project. I am using the progress package for the progress_bar R6 class. The following example produces the expected progress bar:

library(dplyr)
library(purrr)
library(progress)

data <- mtcars %>%
    split(.$carb)

n <- length(data)

pb <- progress::progress_bar$new(total = n)

data <- data %>%
    map(~{pb$tick()
      Sys.sleep(2)
      lm(mpg ~ wt, data = .x)
      })

If I put this into my drake workflow, a new progress bar displays for each iteration:

fit_lm <- function() {
  data <- mtcars %>%
    split(.$carb)

  n <- length(data)

  pb <- progress::progress_bar$new(total = n)

  data <- data %>%
    map(~{pb$tick()
      Sys.sleep(2)
      lm(mpg ~ wt, data = .x)
      })

  return(data)
}

plan <- drake_plan(
  models = fit_lm()
)

make(plan)

Console output:

How can I modify the function to display only one progress bar that updates on each iteration?

For reproducibility, `drake` intercepts messages so you can retrieve them with `diagnose(your_target)$messages`. This has the side effect of delaying all the messages for a target until is is completed. But stdout (`print()` and `cat()`) go straight to the console, so you can still use `txtProgressBar()`. — landau, Dec 10 '19 at 21:34
There is a solution at https://community.rstudio.com/t/including-a-progress-bar-in-a-drake-plan-step/42516. `dplyr` has a progress bar, apparently, which you can print from and `drake` does not intercept. — landau, Dec 10 '19 at 21:42
Update: you progress bar should show up now if you use development `drake` (the GitHub version, `remotes::install_github("ropensci/drake")). — landau, Feb 22 '20 at 13:32
@landau I didn't expect that! I'm eager to try it out. Thanks! — mpschramm, Feb 25 '20 at 15:20

landau · Accepted Answer · 2019-12-11T18:45:20.883

As I mentioned before, drake captures messages for reproducibility, so there is friction with the progress package. But as Adam Kowalczewski pointed out at https://community.rstudio.com/t/including-a-progress-bar-in-a-drake-plan-step/42516, dplyr has a progress bar of its own, and you can make it print to stdout with pb$tick()$print(). This worked for me:

library(drake)
library(dplyr)
library(purrr)

fit_lm <- function() {
  data <- mtcars %>%
    split(.$carb)
  n <- length(data)
  pb <- progress_estimated(n = n)
  data <- data %>%
    map(~{
      pb$tick()$print()
      Sys.sleep(2)
      lm(mpg ~ wt, data = .x)
    })
  return(data)
}

plan <- drake_plan(
  models = fit_lm()
)

make(plan)

Progress bar within drake functions

1 Answers1