subprocess Popen - decode

Question

In my script I was using this with decoding to utf-8:

result = subprocess.run(['command'], stdout=subprocess.PIPE).stdout.decode('utf-8')

Now I need to change command, where I need to use pipelines, so according to few examples which I have found, I need to use subprocess.Popen instead of subprocess.run. So I have something like this:

    r1 = subprocess.Popen(['command1'], stdout=subprocess.PIPE)
    r2 = subprocess.Popen(['command2'], stdin=r1.stdout, stdout=subprocess.PIPE)
    r3 = subprocess.Popen(['command3'], stdin=r2.stdout, stdout=subprocess.PIPE)
    result = r3.stdout

However, in this situation I can't add to the end stdout.decode('utf-8') because I get error

AttributeError: '_io.BufferedReader' object has no attribute 'decode'

Could someone help me, how can I decode it to utf8?

sorry, I've made a mistake. Now it should be OK. But my problem is not solved. I ve just copied wrong part here. — jozinko9, Aug 31 '21 at 12:49
`r3.stdout.read()` should be text itself. You can also pass encoding or text parameter to Popen https://docs.python.org/3/library/subprocess.html#subprocess.Popen — OneCricketeer, Aug 31 '21 at 12:52

score 0 · Answer 1 · answered Aug 31 '21 at 12:58

0

If you need to read the output you can try

op = subprocess.check_output(
        command, shell=True, stderr=subprocess.STDOUT)


result = op.decode()

answered Aug 31 '21 at 12:58

This doesn't really show how to do this properly in Python. Delegating the work to the shell might be a feasible approach if you want to give up the control, but if the OP is asking about using `Popen`, that's what you should arguably be answering. (`subprocess.run()` does everything `check_output` does anyway.) – tripleee Aug 31 '21 at 13:01

score 0 · Answer 2 · answered Aug 31 '21 at 12:59

When you run Popen, you have to do the plumbing yourself.

In very brief, you want to add

stdout, stderr = r3.communicate()

to allow r3 (and by extension, the other two processes) to finish.

You should still also reap the other processes you started.

r1.wait()
r2.wait()

Finally, you can decode the text you read.

print(stdout.decode('utf-8'))

However, running complex pipelines from Python is probably something you want to avoid. Often, you can replace simple shell utilities with native Python code.

On the other hand, if you want to get away with writing as little code as possible, running everything in a single shell command with shell=True might be preferable (with the usual caveats).

subprocess Popen - decode

2 Answers2