Manually add dependency in drake workflow?

Question

Let's say I have a drake plan where I create a SQL table in an external database, and after that job, I download from some table that depends on the initial job. My plan might look like this

drake_plan(up_job = create_sql_file('some_input.csv'), 
           down_job = download_from_sql('my_code.sql')

is there a way of manually forcing down_job to be downstream of up_job? There's nothing inherent in create_sql_file or download_from_sql that drake would be able to parse to infer the relationship but I'd still like to manually apply it.

Thanks!

Answer 1

To have down_job depend on up_job, either up_job or a file_out() created by up_job should be mentioned in the command of down_job.

Example using the up_job return value

library(drake)

plan <- drake_plan(
  db_path = create_sql_db_from(file_in("some_input.csv")), 
  down_job = download_from_sql(db = db_path, file_in("my_code.sql"))
)

plan
#> # A tibble: 2 x 2
#>   target   command                                                    
#>   <chr>    <chr>                                                      
#> 1 db_path  "create_sql_db_from(file_in(\"some_input.csv\"))"          
#> 2 down_job "download_from_sql(db = db_path, file_in(\"my_code.sql\"))"

config <- drake_config(plan)
vis_drake_graph(config)

Example with file paths

library(drake)

plan <- drake_plan(
  up_job = create_sql_db_from(file_in("some_input.csv"), file_out("db_path")),
  down_job = download_from_sql(file_in("db_path"), file_in("my_code.sql"))
)

plan
#> # A tibble: 2 x 2
#>   target   command                                                         
#>   <chr>    <chr>                                                           
#> 1 up_job   "create_sql_db_from(file_in(\"some_input.csv\"), file_out(\"db_…
#> 2 down_job "download_from_sql(file_in(\"db_path\"), file_in(\"my_code.sql\…

config <- drake_config(plan)
vis_drake_graph(config)

^{Created on 2019-01-25 by the reprex package (v0.2.1)}