0

I have 2 files, runner.py that runs target.py with subprocess or exec.
They both have command line options.

If runner runs target with subprocess it's ok:

$ python runner.py
run target.py with subprocess...
target.py: running with dummy = False

If runner runs target code with exec (with the -e option):

$ python runner.py -e
run target.py with exec...
usage: runner.py [-h] [-d]
runner.py: error: unrecognized arguments: -e

the command line argument -e is "seen" by target.py code (which accepts only one --dummy option) and raises an error.

How can I hide args to argparse when running script with exec?

Here's the code:

runner.py

import subprocess
import argparse


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-e", "--exec", help="run with exec", action="store_true")
    args = parser.parse_args()

    target_filename = "target.py"

    if args.exec:
        print("run target.py with exec...")
        source_code = open(target_filename).read()
        compiled = compile(source_code, filename=target_filename, mode="exec")
        exec(compiled) # OPTION 1 - error on argparse
        # exec(compiled, {}) # OPTION 2 - target does not go inside "if main"
        # exec(compiled, dict(__name__="__main__")) # OPTION 3 - same error as OPTION 1 
    else:
        print("run target.py with subprocess...")
        subprocess.run(["python3", target_filename])

I tried to hide the globals with the commented options above, but without luck.
Seems related to how argparse works.

target.py

import argparse

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-d", "--dummy", help="a dummy option", action="store_true")
    args = parser.parse_args()

    print(f"target.py: running with dummy = {args.dummy}")
Guglie
  • 2,121
  • 1
  • 24
  • 43
  • Is there any reason to use `exec` over `subprocess` ? There are various cases [docs](https://docs.python.org/3/library/functions.html#exec) how `g/locals` should look like and what happens when they are not provided (Different namespace, ...). – Maurice Meyer Jun 02 '20 at 12:27
  • I think that in general it can be useful to explore and share questions or answers about unclear aspects of programming. – Guglie Jun 02 '20 at 14:22
  • Thanks @MauriceMeyer, I'm trying to understand how to handle exec in this case. The first viable reason that comes in my mind could be to preprocess the python code before running it (without creating additional files). – Guglie Jun 02 '20 at 14:27
  • You can use `parse_known_args` if you want to ignore unrecognized arguments. – hpaulj Jun 02 '20 at 15:25

1 Answers1

0

There is the argparse conflict_handler option so one can write
argparse.ArgumentParser(conflict_handler='resolve') in target script.
resolve removes the conflicting options, but that doesn't handle well similar cases where the options have the same name both in runner and target or the case where you can't or don't want to change the target file.

Here's the solution I have found.
Internally argparse uses sys.argv to retrieve options set with command line.

You can directly set sys.argv = [target_filename] which removes the options, but changing sys can give a lot of other problems.

Using unittest.mock.patch (python3.4+) the sys.argv can be securely altered like this:

from unittest.mock import patch

# [...]    

source_code = open(target_filename).read()
compiled = compile(source_code, filename=target_filename, mode="exec")

# remove command args
with patch('sys.argv', [target_filename]):
    exec(compiled)

So one can also run target script code with options:

# run target
with patch('sys.argv', [target_filename]):  
    exec(compiled)

# run target with -d
with patch('sys.argv', [target_filename, "-d"]):
    exec(compiled)
Guglie
  • 2,121
  • 1
  • 24
  • 43
  • Can you elaborate on this part "You can directly set `sys.argv = [target_filename]` which removes the options, but changing `sys` can give a lot of other problems." ? Overwriting `sys.argv` shouldn't be a problem in this case, I'd think. – NichtJens May 11 '22 at 07:07
  • @NichtJens Yes probably in this simplified example it wouldn't be a problem to overwrite `sys.argv`, but this doesn't hold if you write a _real_ program where you may use `sys` in other logic – Guglie May 11 '22 at 07:44
  • Not entirely sure I get your point. If you use `sys` in other logic it would probably mean to use it for anything else, but not argument handling. Because you usually have to deal with arguments exactly once, usually at the start. Specifically, there shouldn't be any interference between argv and any other functionality in `sys`. In contrast, I would argue that `mock` is not really meant to be used like that in a *real* program... But for tests. – NichtJens May 17 '22 at 07:57