0

I have a rule in a snakemake workflow that might produce an empty output file, depending on the input. That output file is then used in another rule whose command will be totally ok with an empty file as input, as long as the path exists for that input file, and still complete successfully with the correct result. Is there a way I can write the workflow so that snakemake doesn't consider the first rule to have failed because it produced an empty file?

Edit with some output from snakemake:

RuleException:
CalledProcessError in line 123 of /home/ckern/All_Species_Chromatin_Model/Snakefile:
Command ' set -euo pipefail;  grep E6$ Model_10/Cattle_Cortex_10_segments.bed > Model_10/Cattle_Cortex_10_E6.bed ' returned non-zero exit status 1.
  File "/home/ckern/All_Species_Chromatin_Model/Snakefile", line 123, in __rule_split_states
  File "/share/apps/conda3/miniconda3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Removing output files of failed job split_states since they might be corrupted:
Model_10/Cattle_Cortex_10_E6.bed
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Chris_Rands
  • 38,994
  • 14
  • 83
  • 119
Colin
  • 10,447
  • 11
  • 46
  • 54
  • 2
    snakemake only checks the existence of the file and the timestamps i believe, so it should be fine to have empty files- why do you think otherwise? – Chris_Rands Jun 21 '19 at 09:35
  • I've included the error from the workflow that I'm getting in the original question. I've copied the grep command that it runs into the command line, and it runs without error and produces an empty file. Once that empty file exists, I can run the workflow and it will continue fine. So it seems like it won't have an issue with an empty file when checking input dependencies, but it does when verifying the output of a rule? Unless there is something else going on. I don't see a specific error from the grep command. – Colin Jun 21 '19 at 09:55
  • Also, I should add that this rule has completed successfully for many other input files, so it's not a general error with the syntax of the command or something. It seems to only fail when the command will not produce any output. – Colin Jun 21 '19 at 09:56
  • 7
    I think this is because `grep` returns an exit status of `1` rather than `0` when nothing is found, so snakemake thinks the jobs failed, you can try adding `|| true` to the end of your `grep` command to always force a status of `0` – Chris_Rands Jun 21 '19 at 10:25

0 Answers0