Subprocess & Python what am I doing wrong?

Question

I can't make heads or tails of this one. Trying to use subprocess in python script to scrape my wifi signal via iw. The terminal command works fine:

root@123da06:/app# iw dev wlan0 link | grep signal | awk '{print $2}'
-62

But fails when trying to run it in python:

root@123da06:/app# python3 sub.py
Traceback (most recent call last):
  File "sub.py", line 2, in <module>
    output_bytes = subprocess.check_output("iw dev wlan0 link | grep signal | awk '{print $2}'")
  File "/usr/local/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: "iw dev wlan0 link | grep signal | awk '{print $2}'"

The script couldn't be simpler:

import subprocess
output_bytes = subprocess.check_output("iw dev wlan0 link | grep signal | awk '{print $2}'")
output = output_bytes.decode("utf-8") 
print(f'Signal: {output}')

What am I doing wrong?

Could you try running with `shell=True` option like : `subprocess.check_output("iw dev wlan0 link | grep signal | awk '{print $2}'", shell=True)` — sushant, Oct 15 '21 at 18:48
Even in shell, this is better written as a two-command pipeline: `iw dev wlan 0 link | awk '/signal/ {print $2}'`. With Python, I would skip `awk` as well and process the output of `iw` in Python itself. `for line in subprocess.check_output(["iw", "dev", "wlan0", "link"]): ...`. — chepner, Oct 15 '21 at 18:55

Marshall Conover · Accepted Answer · 2021-10-15T21:57:04.987

The big thing you're running into here is that iw dev wlan0 link | grep signal | awk '{print $2}' isn't one process.

It's three processes, connected by pipes: iw, whose output is piped to grep, whose output is piped to awk.

Notably, the thing creating those pipes when you run this command is the shell - bash, most likely - and it's doing so based on the command you passed into it.

Python's subprocess commands, however, expect you to pass a path to an executable file as your first argument. You can then pass additional strings to be used as arguments for the executable you're running. Doing things the python way, then, would look roughly like: subprocess.check_output("/usr/sbin/iw", "dev", "wlan0"...). This is why python's confused - the big long string you passed to it isn't a file path.

However, you can tell python that you're passing it a big long shell command using an argument - the boolean shell arg.

Try this:

# note the "shell=True" bit
output_bytes = subprocess.check_output("iw dev wlan0 link | grep signal | awk '{print $2}'", shell=True)

and you should see it work out. Python will take your command and hand it to a shell, who will correctly execute it, as opposed to trying to use the string you passed it as a filename of an executable to run.

That did the trick, thank you! Hadn't even considered the point about three separate processes. — Don T Spamme, Oct 15 '21 at 18:59
I believe we have other answers in the knowledge base that not only describe the techniques taught in this answer, but _also_ describe how to create a separate `Popen` object for each stage in the pipeline and connect them together, to allow pipeline use without `shell=True`. — Charles Duffy, Oct 15 '21 at 21:55

Subprocess & Python what am I doing wrong?

1 Answers1