Run bash script in thread not blocking the main loop

Question

I have a python programm that I want a progressbar to be pulsed when I run a bash script in a thread and show some messages.

When I click a button, the thread is started launching the script, but the messages and progressbar only gets responsive when the thread is finished, showing only the last message.

Reading, I understood that I'm blocking the main loop but I can't figure it out how to solve this.

Simplified code of my program, the problem is when calling "on_fixme_button_pressed":

from gi.repository import Gtk, Gdk, Pango, GObject, GLib
import os, sys
import xml.etree.ElementTree as etree
from urllib.request import urlopen
from subprocess import Popen
import threading


UI_FILE = "remendo_gtk.ui"
#UI_FILE = "/usr/local/share/remendo_gtk/ui/remendo_gtk.ui"

GObject.threads_init()

class GUI:

    def __init__(self):

        self.builder = Gtk.Builder()
        self.builder.add_from_file(UI_FILE)
        self.builder.connect_signals(self)

        self.window = self.builder.get_object('remendo')
        self.event_treeview = self.builder.get_object('event_treeview')
        self.event_info = self.builder.get_object('event_info')
        self.progressbar = self.builder.get_object('progressbar')
        self.progressbar_lock = threading.Lock()

        self.selected_event = ''
        self.url_script = ''
        self.local_script = ''
        self.window.connect("destroy", lambda _: Gtk.main_quit())
        self.set_event_list()

        self.window.show_all()

    def pulse_progressbar(self):
        if threading.active_count() > 1:
            self.progressbar.pulse()
            return True

        self.progressbar.set_fraction(0.0)
        self.progressbar_lock.release()
        return False

    def on_fixme_button_clicked(self, button):
        self.event_info.set_label("Fixing now...")
        script = threading.Thread(target=self.run_script(), args=(self,))
        script.start()

        if self.progressbar_lock.acquire(False):
            GLib.timeout_add(250, self.pulse_progressbar)

    def run_script(self):
        try:
            self.local_script = '/tmp/%s.sh' % self.selected_event.replace(" ", "_")
            script = urlopen(self.url_script)
            localscript = open(self.local_script, 'wb')
            localscript.write(script.read())
            localscript.close()

            command = ['bash', self.local_script]
            p = Popen(command)
            p.communicate()

            #self.event_solved()
            self.event_info.set_label("My work is done. Good day!")
        except:
            self.event_info.set_label("Script failed: %s" % self.local_script)

        return False


def main():
    app = GUI()
    Gtk.main()

if __name__ == "__main__":
    sys.exit(main())

`Popen` isn't a regular function, but rather a constructor. If you want to do anything with the process you opened, you'll need to store its return-value in a variable that you can interact with and call methods on (such as `wait`). — ruakh, Jun 15 '13 at 20:14
Yes, using `.communicate()` seems to work just writting `p=Popen(command)` and then `p.communicate()`, but for some reason the popup window doesn't show up until the command ends, showing the "My work is done" message — codiaf, Jun 15 '13 at 20:54
Here's the updated code (http://pastebin.com/xRqb9uDQ). I removed the popup window to work only on the main window. The code is supposed to pulse the progressbar when a thread is running, but it keeps freezing until the thread is finished. I think I understand that it is because the main loop is blocked, but I can't figure it out how to solve it. The main problem is within the on_fixme_button_clicked call — codiaf, Jun 16 '13 at 11:34
If you don't want the main process to block, then you shouldn't be calling the `communicate` method, since by definition it blocks until the process is complete (or, as the docs put it: "Wait for process to terminate"). What you need to do is, *read the documentation*. Or heck, just read the answer that you already replied to: Johan Lundberg already told you that to investigate if the process has completed, you use the `poll` method. — ruakh, Jun 16 '13 at 18:20

Johan Lundberg · Answer 1 · 2013-06-15T19:37:01.723

1

If you use subprocess.Popen you wait for the returned handler with .wait(). You can investigate if the process has completed [and the exit code] with .poll() or interact with it (standard in/out) with .communicate().

The documentation is found here: http://docs.python.org/2/library/subprocess.html#module-subprocess

what have you tried?

edited Jun 15 '13 at 19:37

answered Jun 15 '13 at 19:21

Johan Lundberg

26,184
12
71
97

I've edited the question and put a simplified code of my program where you can see the on_fixme_button_clicked where I call run_script as thread. I have to look yet to the .wait() thing that you pointed – codiaf Jun 15 '13 at 19:33

score 0 · Answer 2 · answered Jun 16 '13 at 16:06

0

I am really not sure it will help, but I suggest you have a look at Pexpect module: http://pexpect.sourceforge.net/pexpect.html

As far, as I understand, it is a more sophisticated "launcher" for system calls.

answered Jun 16 '13 at 16:06

Phlya

5,726
4
35
54

score 0 · Accepted Answer · answered Jun 16 '13 at 21:30

0

Finally solved using GObject.timeout_add to update the state of the messages using a variable in the threaded class to know when the process is finished, and also using the .communicate() option suggested by @johan-lundberg

answered Jun 16 '13 at 21:30

codiaf

569
2
18
47

score 0 · Answer 4 · edited May 23 '17 at 10:32

There are at least two issues:

target=self.run_script() is incorrect: it runs the function immediately. Drop (), to pass the method as an argument instead of calling it.
you call GUI method (on self.event_info.set_label) without protection from a background thread. The simplest way to fix it is to call set_label using idle_add():
```
Gobject.idle_add(self.event_info.set_label, msg) # in a background thread
```
It is safe to use idle_add because there is Gobject.init_threads() at the top of the module.

Run bash script in thread not blocking the main loop

4 Answers4