Give a character string to define a file dependency in drake

Question

I am learning drake to define my analysis workflow but I have trouble getting data files as dependencies.

I use the function file_in() inside drake_plan() but it only works if I give the path to the file directly. If I give it with the file.path() function or with a variable storing that file path, it doesn't work.

Examples:

# preparation
library(drake)

path.data <- "data"
dir.create(path.data)
write.csv(iris, file.path(path.data, "iris.csv"))

Working plan:

# working plan
working_plan <-
  drake_plan(iris_data = read.csv(file_in("data/iris.csv")),
             strings_in_dots = "literals")
working_config <- make(working_plan)
vis_drake_graph(working_config)

This plan works fine, and the file data/iris.csv is considered as a dependency

Working plan

Not working plan:

# not working
notworking_plan <-
  drake_plan(iris_data = read.csv(file_in(file.path(path.data, "iris.csv"))),
             strings_in_dots = "literals")
notworking_config <- make(notworking_plan)
vis_drake_graph(notworking_config)

Here it is trying to read the file iris.csv instead of data/iris.csv.

Working but problem with dependency:

# working but "data/iris.csv" is not considered as a dependency
file.name <- file.path(path.data, "iris.csv")
notworking_plan <-
  drake_plan(iris_data = read.csv(file_in(file.name)),
             strings_in_dots = "literals")
notworking_config <- make(notworking_plan)
vis_drake_graph(notworking_config)

This last one works fine but the file is not considered a dependency, so drake doesn't re-run the plan if this file is changed.

Not working drake plan

So, is there a way to tell drake file dependencies from variables?

Update: you may be interested in dynamic files: https://github.com/ropensci/drake/pull/1178. Brand new in development `drake` (the GitHub version, `remotes::install_github("ropensci/drake")). — landau, Feb 22 '20 at 13:34

score 5 · Answer 1 · answered May 08 '19 at 16:56

Per tidyeval, if you add !! in front of the file.path(), it will be evaluated and not quoted.

Also, in the new version of drake, the strings_in_dots = "literals" argument is deprecated.

library(drake)
path.data <- "data"
dir.create(path.data)
write.csv(iris, file.path(path.data, "iris.csv"))

# now working
notworking_plan <-
  drake_plan(iris_data = read.csv(file_in(!!file.path(path.data, "iris.csv"))))
notworking_plan
#> # A tibble: 1 x 2
#>   target    command                           
#>   <chr>     <expr>                            
#> 1 iris_data read.csv(file_in("data/iris.csv"))

^{Created on 2019-05-08 by the reprex package (v0.2.1)}

Great first contribution, Jeanette! Welcome to stackoverflow! — kennyB, May 08 '19 at 17:40

score 2 · Accepted Answer · answered Mar 30 '18 at 13:36

2

After a answer from the developpers on Github, the code in file_in() is not evaluated so that's why it is not possible to use file.path in it.

answered Mar 30 '18 at 13:36

norival

49
6

One workaround might be `do.call`. A silly example: `library(paste0("MA","SS"))` fails, but `do.call(library, list(paste0("MA","SS")))` works. Not sure if that helps in your dependency tracking; never heard of drake. – Frank Mar 30 '18 at 14:09

Give a character string to define a file dependency in drake

Examples:

Working plan:

Not working plan:

Working but problem with dependency:

2 Answers2