0

Is it possible to have a few child processes running some calculations, then send the result to main process (e.g. update PyQt ui), but the processes are still running, after a while they send back data and update ui again? With multiprocessing.queue, it seems like the data can only be sent back after process is terminated. So I wonder whether this case is possible or not.

martineau
  • 119,623
  • 25
  • 170
  • 301
nmvhs
  • 3
  • 4

1 Answers1

1

I don't know what you mean by "With multiprocessing.queue, it seems like the data can only be sent back after process is terminated". This is exactly the use case that Multiprocessing.Queue was designed for.

PyMOTW is a great resource for a whole load of Python modules, including Multiprocessing. Check it out here: https://pymotw.com/2/multiprocessing/communication.html

A simple example of how to send ongoing messages from a child to the parent using multiprocessing and loops:

import multiprocessing

def child_process(q):
    for i in range(10):
        q.put(i)
    q.put("done")  # tell the parent process we've finished

def parent_process():
    q = multiprocessing.Queue()
    child = multiprocessing.Process(target=child_process, args=(q,))
    child.start()
    while True:
        value = q.get()
        if value == "done":  # no more values from child process
            break
        print value
        # do other stuff, child will continue to run in separate process
wolfson109
  • 898
  • 6
  • 10
  • I'm not sure I understand. Do you mean you want to terminate the process if it takes more than a certain amount of time to complete? If that is what you want then you should probably look at signal.alarm in the signal package https://docs.python.org/2/library/signal.html – wolfson109 Sep 04 '17 at 12:28
  • Thanks! So I was wondering a situation if I want to get some result from child process without waiting it terminates. Child process keeps running all the time and sending back data constantly. Is that possible? – nmvhs Sep 04 '17 at 12:28
  • Yes, you just need to run your child process in a loop that calls queue.put every time it generates a new value. The parent thread can then run a different loop that calls queue.get each time around. – wolfson109 Sep 04 '17 at 12:31
  • Ah. I know what you mean now. I think I completely missed the usage of Queue. I'll reread the article you shared. Thanks a lot! – nmvhs Sep 04 '17 at 12:44
  • Thank you soooooo much for the example. Really helpful for me to understand Queue. Cheers! – nmvhs Sep 04 '17 at 12:47
  • No worries, I actually just edited it because I spotted a bug in my code example. – wolfson109 Sep 05 '17 at 09:51