Async Python function call upon a Bash execution (through Python Popen)

Question

I am executing a bash command (that downloads a file) inside a Python script.

def execBashCmd(bash_cmd):
     process = Popen([bash_cmd], stdout=PIPE, stderr=PIPE, shell=True)

The bash command outputs a progression (refreshed) in stdout as follow:

Downloading File - 20.66 MiB/165.31 MiB

I would like to be able to call a python function at a regular interval to update the progression (that I would catch with a regex or something like this) somewhere in my python code.

cmd = "gsutil cp -n ...."     # A Bash command downloading a file and displaying a status in stdout while busy.
code = utils.execBashCmd(cmd)
self.setProgress(BASH_CURRENT_PROGRESS)  # This line should be asynchrone and called upon the bash command lifetime

Is there a way to achieve this?

This question is new and seems similar to your own question: http://stackoverflow.com/questions/37925840/python-monitoring-progress-of-handbrake - there's a basic mechanism there that might or might not work in your case. — advance512, Jun 20 '16 at 15:18

score 0 · Answer 1 · answered Jul 14 '16 at 09:45

I would like to be able to call a python function at a regular interval to update the progression (that I would catch with a regex or something like this) somewhere in my python code.

At any time you can call e. g. os.read(process.stdout.fileno(), 4096) and get the bash command's output progression (possibly multiple updates) at that time, provided the number of bytes is big enough. If you don't want this call to block until the next progress output if there isn't yet, you can make the pipe non-blocking with

fcntl.fcntl(process.stdout, fcntl.F_SETFL, os.O_NONBLOCK)

and do e. g.

try:
    progress = os.read(process.stdout.fileno(), 4096)
except OSError:
    # no progress data
    pass

Async Python function call upon a Bash execution (through Python Popen)

1 Answers1