0

I want to test a Python function that executes a shell command.

According to testfixtures there are two approaches:

  • execute a real process and check the result
  • mock the subprocess module and check the expected interactions

My function is called run_in_shell. Although the subprocess module is the obvious way to implement it, my function doesn't explicitly depend on it, so I'm trying to do the "real" test.

import subprocess

def run_in_shell(command, shell_name=None):
    """Run command in default shell. Use named shell instead if given."""
    subprocess.run(command, shell=True, executable=shell_name)

This test shows that the function can execute a command with the default shell.

import pytest

def test_with_command_string(capfd):
    run_in_shell("echo 'hello'")
    cap = capfd.readouterr()
    assert cap.out.strip() == "hello"
    assert cap.err.strip() == ""

I also want to show that it can execute in the user's chosen shell such as /bin/bash.

The invocation is simple enough. This prints hello to the terminal.

run_in_shell("echo 'hello'", shell_name="/bin/bash")

Without mocking, how do I show that it executed /bin/bash to do so?

I tried ptracer to trace the system calls, but the output disappointed me.

def callback(syscall):
    name = syscall.name
    args = ",".join([str(a.value) for a in syscall.args])
    print(f"{name}({args})")

with ptracer.context(callback):
    run_in_shell("echo 'hello'")

with ptracer.context(callback):
    run_in_shell("echo 'hello'", shell_name="/bin/bash")

I was hoping to see a clone or fork call with the name of the shell, but there is nothing so clear. I don't understand the strings in the read calls, and I don't see any write calls.

hello
pipe2((23, 24),524288)
clone(18874385,0,None,261939,0)
close(24)
read(23,bytearray(b'U\r\r\n'),50000)
close(23)
wait4(262130,0,0,None)

hello
pipe2((24, 25),524288)
clone(18874385,0,None,261939,0)
futex(2,129,1,0,9693984,9694016)
futex(0,129,1,0,None,9694016)
close(25)
read(24,bytearray(b'\x1d'),50000)
close(24)
wait4(262308,0,0,None)

At this point I'm out of my depth. I've surely misunderstood what the system calls are really doing. What am I missing?

Is it possible and practical to test this using Python and PyTest? If not, I'll redefine my function to depend explicitly on one of the subprocess functions, and then I can use a mock to test that it sends the right messages to the function.

Iain Samuel McLean Elder
  • 19,791
  • 12
  • 64
  • 80

2 Answers2

2

If I understand correctly, for this check all we're trying to do is verify that whatever shell is passed in is being used.

I would keep it as simple as possible and just print the active shell's name, then verify it matches the target shell. Something like

def test_executes_in_specified_shell(capfd):
    target_shell = "/bin/bash" 
    
    print_current_shell = "ps -p $$ -oargs= && true"
    run_in_shell(print_current_shell, shell_name=target_shell)

    cap = capfd.readouterr()
    current_shell = cap.out.strip()
    assert current_shell == target_shell
Iain Samuel McLean Elder
  • 19,791
  • 12
  • 64
  • 80
Teejay Bruno
  • 1,716
  • 1
  • 4
  • 11
  • Wow, I hadn't thought of running an introspective command to test that. Can you elaborate on `-oargs=`? That's new to me and seems to be the key to this. I'm going to try it tomorrow when I'm back at my desk! – Iain Samuel McLean Elder Aug 30 '23 at 22:34
  • It doesn't work but I don't know why! `bash -c 'ps -p $$ -oargs='` gives `ps -p 553355 -oargs=`, as if `$$` gave the pid of the `ps` process. But `bash -c 'ps -p $$ -oargs= && true'` gives `bash -c ps -p $$ -oargs= && true`, which starts with `bash` so looks right. Do you know why this happens? – Iain Samuel McLean Elder Aug 31 '23 at 11:30
  • Your insight helped me find an even easier way. I'll share it in an answer. – Iain Samuel McLean Elder Aug 31 '23 at 11:31
  • [Bash replaces itself with the simple command](https://unix.stackexchange.com/questions/755419/why-doesnt-pid-refer-to-bash-here), so `ps` would return its own command. The `&& true` addition makes the command a compound and bash forks each subprocess with a new pid. – Iain Samuel McLean Elder Aug 31 '23 at 12:54
  • Glad you found a solution that works! – Teejay Bruno Aug 31 '23 at 16:23
  • I completed your example so that it is also a valid solution. Thanks again for your help. – Iain Samuel McLean Elder Sep 02 '23 at 11:55
0

As Teejay Bruno shows, you can use the command to introspect the shell. The shell already stores its name in the $0 variable, so you can just echo that and check the standard output.

def test_executes_in_specified_shell(stack, capfd):
    run_in_shell("echo $0", shell_name="/bin/bash")
    cap = capfd.readouterr()
    assert cap.out.strip() == "/bin/bash"
    assert cap.err.strip() == ""

I think $0 should work in any POSIX-compliant shell, although I can't find clear documentation for that. It certainly works in Bash and dash.

Iain Samuel McLean Elder
  • 19,791
  • 12
  • 64
  • 80