Redirecting Output of PIPE to a FILE in python

Question

Following code is to redirect the output of the Pipe to a file "CONTENT" and it has some content, I want to overwrite it with output of "sort CONTENT1 | uniq ".... But I'm not able overwrite it and also i don't know weather following code is redirecting to CONTENT(ie correct or not) or not. Please help me out....

f1=open('CONTENT','w')      
sys.stdout=f1  
p1 = subprocess.Popen(["sort", "CONTENT1"], stdout=subprocess.PIPE)   
p2 = subprocess.Popen(["uniq"], stdin=p1.stdout, stdout=subprocess.PIPE)  
p1.stdout.close()  
p2.communicate()   
sys.stdout=sys.__stdout__

possible duplicate of [piping output of subprocess.Popen to files](http://stackoverflow.com/questions/2331339/piping-output-of-subprocess-popen-to-files) — Martijn Pieters, Sep 26 '13 at 06:39
@MartijnPieters Not really, this question is about connecting two processes with a pipe, then passing the final output to a file. — rerx, Sep 26 '13 at 07:33

rerx · Accepted Answer · 2013-09-26T07:45:47.810

Here is how you can catch the output of the first process and pass it to the second, which will then write its output to the file:

import subprocess
with open('CONTENT','w') as f1:
  p1 = subprocess.Popen(["sort", "CONTENT1"], stdout=subprocess.PIPE)
  p2 = subprocess.Popen(["uniq"], stdin=subprocess.PIPE, stdout=f1)
  p1_out = p1.communicate()[0] # catch output
  p2.communicate(p1_out)       # pass input

You should not tinker with sys.stdout at all. Note that you need one call to the method communicate for each process. Note also that communicate() will buffer all output of p1 before it is passed to p2.

Here is how you can pass the output of p1 line-by-line to p2:

import subprocess
with open('CONTENT','w') as f1:
    p1 = subprocess.Popen(["sort", "CONTENT1"], stdout=subprocess.PIPE)
    p2 = subprocess.Popen(["uniq"], stdin=subprocess.PIPE, stdout=f1)
    out_line = p1.stdout.readline()
    while out_line:
        p2.stdin.write(out_line)
        out_line = p1.stdout.readline()

The cleanest way to do the pipe would be the following:

import subprocess
with open('CONTENT','w') as f1:
  p1 = subprocess.Popen(["sort", "CONTENT1"], stdout=subprocess.PIPE)
  p2 = subprocess.Popen(["uniq"], stdin=p1.stdout, stdout=f1)
  p1.stdout.close()

Alternatively, of course, you could just use the facilities of the shell, which is just made for these tasks:

import subprocess
with open('CONTENT','w') as f1:
    p = subprocess.Popen("sort CONTENT1 | uniq", shell=True,
                         stdout=f1)

Reference: http://docs.python.org/2/library/subprocess.html

Thank you for your answer,But it is not overwriting CONTENT file rather it is appending.....? — Naive, Sep 26 '13 at 07:40
`open('CONTENT','w')` should open the file `CONTENT` for writing, overwriting everything that is in it. To append you would need `open('CONTENT','a')`. — rerx, Sep 26 '13 at 07:48

Redirecting Output of PIPE to a FILE in python

1 Answers1