Python monitoring progress of Handbrake

Question

So I am using handbrake and python to encode videos based on a schedule. I need to monitor the progress because I use it to estimate the encoding time. Then I can fit it to my scheduler.

I am having an issue getting the ETA and % complete from the process. Here is what I have so far

profile = ["HandBrakeCLI","-i",input,"-o","output","-e","x264"]
cp = subprocess.Popen(profile, stderr=subprocess.PIPE, bufsize=1)
for line in iter(cp.stderr.readline, b''):

  # regex match for % complete and ETA
  matches = re.match( r'.*(\d+\.\d+)\s%.*ETA\s(\d+)h(\d+)m(\d+)s', line.decode('utf-8') )

  if matches:
    print( matches.group() )

  print(line),

cp.stderr.close()
cp.wait()

It does not match, in fact I'm not entirely sure what is going on. When I run my script, I see the ETA and % complete print out

Encoding: task 1 of 1, 1.19 % (45.57 fps, avg 62.74 fps, ETA 00h08m01s)

I've tried using stdout but it doesn't work either.

I'm pretty sure the issue is that Handbrake CLI doesn't output new additional lines each time the process has progressed a bit, rather it modifies the existing line. Try printing the line immediately after `for line in inter..` and see what it actually gives you. — advance512, Jun 20 '16 at 15:16
wouldn't that give me the output after the for loop exits? so when the process has finished ? I would like to get the information as the process is going. — Yusuf Ali, Jun 20 '16 at 16:16
First of all, you're iterating over stderr, maybe you need to iterate over stdout. Second,I think that cp.stderr.readline would return complete lines from the subprocess itself, not a single repeatedly updated line. I would print the last line after the `handbreak` process finishes, to see how it look. And, as I told you, I'd print each and every lines in the for loop, to understand what I actually receive. — advance512, Jun 20 '16 at 16:20
iterating over stdout doesn't print out the Encoding information. I do not understand what you mean, can you provide an example? — Yusuf Ali, Jun 20 '16 at 16:47
Try remove lines 5-13 and instead, just output `print(line)`, and let me know when and what it actually prints. — advance512, Jun 20 '16 at 16:49
Here is the output in a pastebin (http://pastebin.com/SLuayW3q). essentially its one line with "Encoding: task 1 of 1, 98.04 % (394.61 fps, avg 385.53 fps, ETA 00h00m01s)\r" repeating over and over until the % hits 100, then it terminates with \n — Yusuf Ali, Jun 20 '16 at 17:12
The issue (I think!) seems to be that \r (carriage return, return to the beginning of the line) isn't a line separator (\n), so readline reads the entire chunk until the last character \n appears, and you get it all as one chunk. A possible solution would be reading stderr byte by byte into a string, until you reach a \r character, and then treat that string as a single line. After that start again. — advance512, Jun 20 '16 at 17:18

score 1 · Answer 1 · answered Oct 25 '17 at 10:34

You need to read from stdout, not stderr.

profile = ["HandBrakeCLI","-i",input,"-o","output","-e","x264"]
cp = subprocess.Popen(profile, stderr=subprocess.PIPE, strout=subprocess.PIPE, bufsize=1)
for line in iter(cp.stdout.readline, b''):

  # regex match for % complete and ETA
  matches = re.match( r'.*(\d+\.\d+)\s%.*ETA\s(\d+)h(\d+)m(\d+)s', line.decode('utf-8') )

  if matches:
    print( matches.group() )

  print(line),

cp.stderr.close()
cp.stdout.close()
cp.wait()

Using a progress wrapper (using clint.textui.progress.Bar) and read byte by byte (works for me):

profile = ["HandBrakeCLI","-i",input,"-o","output","-e","x264"]
cp = subprocess.Popen(profile, stderr=subprocess.PIPE, strout=subprocess.PIPE, close_fds=True)
bar = Bar(label="Encoding %s" % input, width=30, expected_size=10000, every=1)
bar.show(0)

line = ""
c = 0

while True:    
  nl = cp.stdout.read(1)
  if nl == '' and cp.poll() is not None:
     break  # Aborted, no characters available, process died.
  if nl == "\n":
     line = ""
  elif nl == "\r":
     # regex match for % complete and ETA, assuming the regex is ok.
     matches = re.match( r'.*(\d+\.\d+)\s%.*ETA\s(\d+)h(\d+)m(\d+)s', line.decode('utf-8') )

     if matches:
        print( matches.group() )
        # do something
     line = ""
  else:
     line += nl

error = cp.stderr.read()
success = "Encode done!" in error

Did not test the code, rewrote it to match the threads initial post.

Hope that helps.

score -1 · Answer 2 · answered Jan 20 '20 at 09:46

There are good bones here. However, I had to make a few modification to make it work with Python 3.7 and PyQt5. The ui lines are for a PyQt5 QProgressBar and a QLineEdit

This code has been tested. I thank all of you for your help.

def hbConvertISOtoMP4():

line = ""
inFile = #place your input file here!
oFile =  #place your output file here!

ui.progressBar.setValue(0)
profile = ["HandBrakeCLI", "-t", "1", "-i", inFile, "-o", oFile, "-e", "x264"]
cp = Popen(profile, stderr=PIPE, stdout=PIPE, close_fds=True)

ui.leProgress.setText('Loading data... Please Wait.')
ui.centralwidget.repaint()

while True:
    nl = cp.stdout.read(1)

    if nl == '' or cp.poll() is not None:
        break  # Aborted, no characters available, process died.

    elif nl.hex() == '0d' and len(line) > 30:

        # regex match for % complete and ETA, assuming the regex is ok.
        matches = re.match(r'.*(\d+\.\d+)\s%.*ETA\s(\d+)h(\d+)m(\d+)s\)', line)

        if matches:
            # do something here
            # I inserted this code for a PyQt5 Progress Bar UI
            ui.leProgress.setText(line)
            ui.centralwidget.repaint()
            pBar = matches.group().split(' ')
            ui.progressBar.setValue(int(float(pBar[5])))

        line = ""
    else:
        line += nl.decode('utf-8')

error = cp.stderr.read()

if 'Encode done!' in str(error):
    ui.progressBar.setValue(0)
    ui.leProgress.setText("Encode Done!")
else:
    ui.leProgress.setText('Error during Endoding')

Python monitoring progress of Handbrake

2 Answers2

Linked