1

I'm trying to parse sar results in python using subprocess, everytime I call the python code that calls a sar subprocess it produces a different number of lines. Here's a minimal program to reproduce this problem:

import sys
import subprocess

fn = sys.argv[1]

p = subprocess.Popen(str('sar -r -f %s' % fn).split(' '), shell=True, stdout=subprocess.PIPE)
count = 0
while p.poll() is None:
    output = p.stdout.readline()
    count += 1

print "num lines = %d" % count

Calling this program several times, produces different number of lines each time: like this

for i in {1..10}; do ./sarextractor.py /var/log/sysstat/sa02; done

Calling sar directly on the command line produces constant number of lines:

for i in {1..10}; do sar -r -f /var/log/sysstat/sa02 | wc -l; done

any idea how can this happen?

Moataz Elmasry
  • 2,509
  • 4
  • 27
  • 37

2 Answers2

1

Suppose p.poll() returns a return code but the p.stdout buffer still holds some data? Try something like this instead:

p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, bufsize=0)
    for line in p.stdout:
        count += 1

That will drain the stdout buffer of all its lines.

William Reed
  • 101
  • 4
1

I'm not sure why you're using poll here, but why not use communicate?

import sys
import subprocess

fn = sys.argv[1]

p = subprocess.Popen('sar -r -f %s' % fn.split(' '), shell=True, stdout=subprocess.PIPE)
outs, errs = p.communicate()
print("num lines = %d" % len(outs.splitlines())
simonzack
  • 19,729
  • 13
  • 73
  • 118
  • Thanks. It is working. Actually both answers are working. I'll give the right answer to the other guy with very few points :) – Moataz Elmasry Sep 26 '14 at 07:47