3

I am writing a command-line script, mycli, with two subcommands:

  • mkcli init (to initialize an empty project with a .configrc file)
  • mkcli run (to run the main logic of the script).

In general, mycli run should not work if a .configrc file is not found in the working directory. However, my users should be able to look at the help message for run:

$ mycli run --help
Usage: mycli run [OPTIONS]

Options:
  --dryrun  Run in read-only mode
  --help    Show this message and exit.

However, this does not work if .configrc does not exist because FileNotFoundError is raised in the group command cli (and run is never reached). I can get the init subcommand to fire without first finding a .configrc file by using ctx.invoked_subcommand (see below), but I see no way to ensure that the run subcommand will always fire if it is invoked with --help.

If a user runs mkcli run and no .configrc file is found, my script exits with run "mycli init" first. But mycli run --help should work even if there is no .configrc. How can I do this? Or can anyone suggest a better way to handle init?

@click.group()
@click.pass_context
def cli(ctx):

    ctx.obj = {}
    if ctx.invoked_subcommand != "init":
        config = yaml.load(open(".configrc").read())
        ctx.obj.update({key: config[key] for key in config})

@cli.command()
@click.pass_context
def init(ctx):
    print("Initialize project.")

@cli.command()
@click.option("--dryrun", type=bool, is_flag=True, help="Run in read-only mode")
@click.pass_context
def run(ctx, dryrun):
    print("Run main program here.")
Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
Tom Baker
  • 683
  • 5
  • 17

1 Answers1

2

I would suggest changing the order in which your init code is run. That can be done with a...

Custom Class:

class LoadInitForCommands(click.Group):

    def command(self, *args, **kwargs):

        def decorator(f):
            # call the original decorator
            cmd = click.command(*args, **kwargs)(f)
            self.add_command(cmd)
            orig_invoke = cmd.invoke

            def invoke(ctx):
                # Custom init code is here
                ctx.obj = {}
                if cmd.name != "init":
                    config = yaml.load(open(".configrc").read())
                    ctx.obj.update({key: config[key] for key in config})

                # call the original invoke()
                return orig_invoke(ctx)

            # hook the command's invoke
            cmd.invoke = invoke
            return cmd

        return decorator

Using the Custom Class:

Pass the Custom Class to click.group() using cls parameter like:

@click.group(cls=LoadInitForCommands)
def cli():
    """"""

How does this work?

This works because click is a well designed OO framework. The @click.group() decorator usually instantiates a click.Group object but allows this behavior to be over-ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride the desired methods.

In this case, we hook the command() decorator and in that hook, we override the invoke() for the command. This allows the init file to be read after the --help flag has already been processed.

Note this code is meant to make it easy to have many commands for which the --help would be available before the init is read. In the example in the question there is only one command that needs the init. If this always the case, then this answer might be appealing.

Test Code:

import click
import yaml

@click.group(cls=LoadInitForCommands)
def cli():
    """"""

@cli.command()
@click.pass_context
def init(ctx):
    print("Initialize project.")


@cli.command()
@click.option("--dryrun", type=bool, is_flag=True,
              help="Run in read-only mode")
@click.pass_context
def run(ctx, dryrun):
    print("Run main program here.")


if __name__ == "__main__":
    commands = (
        'init',
        'run --help',
        'run',
        '--help',
    )

    import sys, time

    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for cmd in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + cmd)
            time.sleep(0.1)
            cli(cmd.split(), obj={})

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc, (click.ClickException, SystemExit)):
                raise

Results:

Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> init
Initialize project.
-----------
> run --help
Usage: test.py run [OPTIONS]

Options:
  --dryrun  Run in read-only mode
  --help    Show this message and exit.
-----------
> run
Traceback (most recent call last):
  File "C:\Users\stephen\AppData\Local\JetBrains\PyCharm 2018.3\helpers\pydev\pydevd.py", line 1741, in <module>
    main()
  File "C:\Users\stephen\AppData\Local\JetBrains\PyCharm 2018.3\helpers\pydev\pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Users\stephen\AppData\Local\JetBrains\PyCharm 2018.3\helpers\pydev\pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Users\stephen\AppData\Local\JetBrains\PyCharm 2018.3\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/stephen/Documents/src/testcode/test.py", line 77, in <module>
    cli(cmd.split(), obj={})
  File "C:\Users\stephen\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\stephen\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\stephen\AppData\Local\Programs\Python\Python36\lib\site-packages\click\core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:/Users/stephen/Documents/src/testcode/test.py", line 26, in invoke
    config = yaml.load(open(".configrc").read())
FileNotFoundError: [Errno 2] No such file or directory: '.configrc'
Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
  • Wow - this works great, thanks! I tried adding an option to `init` (`init --with-examples`) and this works too (as it should). But I do not understand _why_ it works. How is `decorator(f)` called and where does `f` come from? – Tom Baker Jan 14 '19 at 08:02
  • Decorators can be confusing if you have not used them much. `decorator(f)` is called when you do `@cli.command()`. In short: `cli.command()` returns `decorator`, and then the `@` calls `decorator(f)` where `f` is the function being decorated. This might help: https://www.learnpython.org/en/Decorators – Stephen Rauch Jan 14 '19 at 13:27
  • The [sourcecode](https://github.com/pallets/click/blob/master/click/core.py#L1227) makes this (almost) clear. I found two things confusing: 1) the arbitrary name of the `decorator` function (`foobar` works just as well), 2) the syntax of `click.command(*args, **kwargs)(f)` - I do not believe I have ever seen a function call with two sets of parens (and am still unsure how that parses). Do I correctly understand that every time `Group.command()` is run, the `decorator` function is passed the decorated function as argument 'f'? Thank you for packaging your answer with tests - very helpful! – Tom Baker Jan 16 '19 at 02:08
  • 1) As you noted, the name is arbitrary and lasts only long enough to return the function, so why not call it what it is? If Python had a richer lambda mentality, like some other languages, then maybe you could simply return an anonymous function when it was defined. 2) Your understanding is correct. 3) The test cases thing is because I have answered [quite a few click questions](https://stackoverflow.com/tags/python-click/topusers), and as you noted it can be difficult to understand simply reading the code. Being able to run the exact code I did can help as the example is ported to reality. – Stephen Rauch Jan 16 '19 at 02:30