python subprocess dd and stdout

Question

I am using subprocess to create a random file from /dev/random using unix dd. Now, if i want the data output of dd to be written to a file instead of stdout. so here's the code am using,

import subprocess
out_fd = open('test_file','w')
def os_system_dd():
   global out_fd
   out_fd.write("executing the time dd command\n")
   cmd_list = ['time','dd','if=/dev/random', 'of=/home/anand/sys_entropy_random', 'bs=1M' ,'count=5']
   a = subprocess.Popen(cmd_list,stdout=out_fd)
   a.wait()

if __name__ == '__main__':
   os_system_dd()

This doesn't print the dd output to the file and instead prints it in the stdout. Is this a specific functionality of dd command? Or am i missing some thing about how subprocess works?

+1 For succinct code and using a list to construct the argument. However, I can't reproduce your problem here; your code works fine. Does your problem still occur if the whole code is `subprocess.Popen(['time','dd','if=/dev/random', 'of=sys_entropy_random', 'bs=1M' ,'count=5']).communicate()` ? — phihag, Sep 28 '11 at 10:22
Well, i tried this `code` a = subprocess.Popen(['time','dd','if=/dev/random', 'of=/home/anand/sys_entropy_random', 'bs=1M' ,'count=5'],stdout=out_fd) `code` and still get the same error. Using communicate() may solve my problem, but i was trying to avoid the extra explicit write from python. will try and update it though. — Software Mechanic, Sep 28 '11 at 10:29
Oops, I misunderstood the question. Answered - simply redirect stderr instead of stdout. — phihag, Sep 28 '11 at 10:34

score 3 · Accepted Answer · answered Sep 28 '11 at 10:35

dd outputs its debugging information on stderr, not stdout:

import subprocess
out_fd = open('test_file','w')
def os_system_dd():
   out_fd.write("executing the time dd command\n")
   cmd_list = ['time','dd','if=/dev/random', 'of=/home/anand/sys_entropy_random',
                           'bs=1M' ,'count=5']
   a = subprocess.Popen(cmd_list,stderr=out_fd) # notice stderr
   a.communicate()

if __name__ == '__main__':
   os_system_dd()

spicavigo · Answer 2 · 2011-09-28T10:41:18.880

2

The reason nothing gets written in the file is because its written to stderr. Redirect stderr and you will get the result.

import subprocess
out_fd = open('test_file','w')
def os_system_dd():
   global out_fd
   out_fd.write("executing the time dd command\n")
   cmd_list = ['date'] #Your list
   a = subprocess.Popen(cmd_list,stdout=out_fd, stderr=out_fd)
   a.wait()

if __name__ == '__main__':
   os_system_dd()

Also, flush the buffer after writing "executing the time..."

edited Sep 28 '11 at 10:41

answered Sep 28 '11 at 10:31

spicavigo

4,116
22
28

`shell=True` just makes the code run slower and more error-prone. Why would you want to use it? – phihag Sep 28 '11 at 10:33
My bad. I tried adding different things till it worked. Now I realize that there was nothing wrong with his code except that stderr redirection. Updating my answer to reflect that. – spicavigo Sep 28 '11 at 10:38
@phihag I don't think that the slowdown is noticeable in any way. And if the command line is hard coded, there is noothing wrong with `shell=True` (except that it leaves a little bad smell). If it was a dynaming string, it would be criminal indeed... – glglgl Sep 28 '11 at 10:46

python subprocess dd and stdout

2 Answers2