0

I am trying to use Python's Popen() to retrieve graph data from a multiple rrd files. Due to complexity of app where the following piece of code is utilised, I rely on rrdtool graph parameter -Z for handling missing files for me:

#!/bin/python3

import subprocess

cmd = '/opt/rrdtool/bin/rrdtool graph - -a JSONTIME -Z --width 924 --start 1486428000 --end 1486471200 DEF:foo1=ch1.rrd:flows:MAX DEF:foo2=ch2.rrd:flows:MAX  AREA:foo1#000:"ch1" AREA:foo2#606060:"ch2":STACK'
path = '/data/live/pokus/rrd/channels/'

p = subprocess.Popen(cmd, stdout=subprocess.PIPE, cwd=path, shell=True)
p.wait()
if p.returncode is not 0:
    print("Error")
else:
    print(p.stdout.read().decode(encoding="utf-8"))

Following code works as expected when both files ch1.rrd and ch2.rrd are present. When one of them is missing, whole thing hangs indefinitely until I kill the rrdtool process manually from htop. Then python detects nonzero return code and reports error.

Using shell=False and shlex.split() on cmd does not help.

When I execute the same command from bash, even with the missing files rrdtool does the job as expected.

Unfortunately I can't use rrdtool bindings for python and also I am stuck on python 3.4.5. rrdtool version is 1.6.0.

I am glab for any idea how to overcome this. I would prefer solution that does not include testing whether files exist from python and keeps the -Z parameter in rrdtool command. Also using timeout on p.wait() isn't a viable solution.

Thanks in advance

doomista
  • 501
  • 2
  • 10
  • Try capturing STDERR in your `subprocess.Popen()` call as well (i.e. `stderr=subprocess.PIPE`) – zwer Jul 04 '17 at 18:44

1 Answers1

0

Ok, I found the solution.

The reason why Python (namely p.wait) hanged was because rrdtool did not know the minimum step size (parameter -S) resulting in step size of two seconds. This way, output of the rrdtool was able to fill the OS buffers and that deadlocked p.wait. According to Python docs, Popen.communicate should be a way to go.

doomista
  • 501
  • 2
  • 10