2

I've got a ruffus pipeline in Python 2.7, but when I call it with -n or --just_print it still runs all the actual tasks instead of just printing the pipeline like it's supposed to. I:
* don't have a -n argument that would supercede the built-in (although I do have other command-line arguments)
* have a bunch of functions with @transform() or @merge() decorators
* end the pipeline with a run_pipeline() call

Has anyone else experienced this problem? Many thanks!

Chris_Rands
  • 38,994
  • 14
  • 83
  • 119
J. C. Szamosi
  • 23
  • 3

1 Answers1

1

As of ruffus version 2.4, you can use the builtin ruffus.cmdline which stores the appropriate flags via the cmdline.py module that uses argparse, for example:

from ruffus import *
parser = cmdline.get_argparse(description='Example pipeline')
options = parser.parse_args()

@originate("test_out.txt")
def run_testFunction(output):
        with open(output,"w") as f:
            f.write("it's working!\n")

cmdline.run(options)

Then run your pipeline from the terminal with a command like:

python script.py --verbose 6 --target_tasks run_testFunction --just_print

If you want to do this manually instead (which is necessary for older version of ruffus) you can call pipeline_printout() rather than pipeline_run(), using argparse so that the --just_print flag leads to the appropriate call, for example:

from ruffus import *
import argparse
import sys

parser = argparse.ArgumentParser(description='Example pipeline')
parser.add_argument('--just_print', dest='feature', action='store_true')
parser.set_defaults(feature=False)
args = parser.parse_args()

@originate("test_out.txt")
def run_testFunction(output):
        with open(output,"w") as f:
            f.write("it's working!\n")

if args.feature:
    pipeline_printout(sys.stdout, run_testFunction, verbose = 6)
else:
    pipeline_run(run_testFunction, verbose = 6)

You would then run the command like:

python script.py --just_print
Chris_Rands
  • 38,994
  • 14
  • 83
  • 119
  • That's certainly a functional hack, but doesn't it defeat the purpose of ruffus having those built-in arguments? Or are the built-in arguments just suggestions? – J. C. Szamosi Jan 27 '17 at 14:02
  • @J.C.Szamosi There are no builtin arguments, you need to build your own constructs. I have seen code from the author of the ruffus package, he also does it something like this – Chris_Rands Jan 27 '17 at 14:08
  • That's very helpful, thank you! I think the documentation about predefined options (http://www.ruffus.org.uk/tutorials/new_tutorial/command_line.html) is a little misleading, so I'll get in touch with them about that. – J. C. Szamosi Jan 30 '17 at 14:42
  • @J.C.Szamosi Actually I was not aware of these new features in ruffus 2.4; see my updated answer and +1 for the question! – Chris_Rands Jan 30 '17 at 15:50
  • Ah! It was `cmdline.run(options)` that I was missing. I'm a little unsure about how that interacts with my normal habit of renaming argparse's variables for use in my scripts (i.e. `infile = options.i`), but I'll play with that. Many thanks! – J. C. Szamosi Jan 31 '17 at 17:17