0

I would like to use dynamic files for my drake plan. I have followed drake's documentation on dynamic files.

However, drake does not seem to incorporate the file paths created into its cache.

In the following reprex, plan_a follows the documentation, but ends up rebuilding the targets figure_export_path and hence figure_export on every run. plan_b does not rebuild anything on the second run, but its code is inconsistent with drake's documentation.

What is the correct way to use dynamic files in a situation like mine?

library(drake)
library(tidyverse)

foo <- function(bar) {
  y <- pmax(1:3, bar)
  qplot(x = 1:3, y = y, geom = "point")
}

plan_a <- drake::drake_plan(
  parameter = 1:3,

  figure = target(
    foo(parameter),
    dynamic = map(parameter)
  ),

  figure_export_path = target(
    paste(parameter, 'pdf', sep = '.'),
    dynamic = map(parameter),
    format = "file" # This line seems to invalidate the cache
  ),
  
  figure_export = target(
    ggsave(figure_export_path, figure),
    dynamic = map(figure_export_path, figure)
  )
)

plan_b <- drake::drake_plan(
  parameter = 1:3,
  
  figure = target(
    foo(parameter),
    dynamic = map(parameter)
  ),
  
  figure_export_path = target(
    paste(parameter, 'pdf', sep = '.'),
    dynamic = map(parameter),
    # format = "file" # This line seems to invalidate the cache
  ),
  
  figure_export = target(
    ggsave(figure_export_path, figure),
    dynamic = map(figure_export_path, figure)
  )
)

rerun(2, drake::make(plan = plan_a))
#> ▶ target parameter
#> ▶ dynamic figure
#> > subtarget figure_0b3474bd
#> > subtarget figure_b2a5c9b8
#> > subtarget figure_71f311ad
#> ■ finalize figure
#> ▶ dynamic figure_export_path
#> > subtarget figure_export_path_0b3474bd
#> Warning: missing dynamic files for target figure_export_path_0b3474bd:
#>   1.pdf
#> > subtarget figure_export_path_b2a5c9b8
#> Warning: missing dynamic files for target figure_export_path_b2a5c9b8:
#>   2.pdf
#> > subtarget figure_export_path_71f311ad
#> Warning: missing dynamic files for target figure_export_path_71f311ad:
#>   3.pdf
#> ■ finalize figure_export_path
#> ▶ dynamic figure_export
#> > subtarget figure_export_25d05a3f
#> Saving 7 x 5 in image
#> > subtarget figure_export_4e67ff77
#> Saving 7 x 5 in image
#> > subtarget figure_export_5ac7bd49
#> Saving 7 x 5 in image
#> ■ finalize figure_export
#> ▶ dynamic figure_export_path
#> > subtarget figure_export_path_0b3474bd
#> > subtarget figure_export_path_b2a5c9b8
#> > subtarget figure_export_path_71f311ad
#> ■ finalize figure_export_path
#> ▶ dynamic figure_export
#> > subtarget figure_export_e284629c
#> Saving 7 x 5 in image
#> > subtarget figure_export_7278fc00
#> Saving 7 x 5 in image
#> > subtarget figure_export_72064f81
#> Saving 7 x 5 in image
#> ■ finalize figure_export
#> [[1]]
#> NULL
rerun(2, drake::make(plan = plan_b))
#> ▶ dynamic figure_export_path
#> > subtarget figure_export_path_0b3474bd
#> > subtarget figure_export_path_b2a5c9b8
#> > subtarget figure_export_path_71f311ad
#> ■ finalize figure_export_path
#> ▶ dynamic figure_export
#> > subtarget figure_export_5a8b1281
#> Saving 7 x 5 in image
#> > subtarget figure_export_05ed8067
#> Saving 7 x 5 in image
#> > subtarget figure_export_3c33e0b9
#> Saving 7 x 5 in image
#> ■ finalize figure_export
#> ✓ All targets are already up to date.
#> [[1]]
#> NULL

Created on 2020-08-03 by the reprex package (v0.3.0)

robust
  • 594
  • 5
  • 17
  • 1
    `plan_b` is the correct one. `figure_export_path` determines the path but does not actually write the file yet, so it should not have `format = "file"`. In what way does `plan_b` contradict the documentation? – landau Aug 04 '20 at 02:04
  • Thank you. I had a wrong understanding of how `format = "file"` works. My wrong understanding was that I needed to declare the paths as targets *before* using them to write to a file. Your answer also makes the documentation clearer to me. With the "curse" of understanding it now, I am unsure what can be changed to make the documentation clearer. But maybe it would have helped if the example plan in the documentation writes to more than just one file, so that the plan has a context in which using dynamic files makes more sense. – robust Aug 04 '20 at 17:27

0 Answers0