Python Multiprocessing Drawing to Tkinter Canvas

Question

I have a program which uses multiple cores to process images before drawing to a canvas for a main view (using multiprocessing). I would like to know the best way to tackle this problem.

Is it possible to have each core drawing to its own canvas which are layered on-top of each other in the same view? Is it possible to have this behavior?

You can't share GUI widgets between processes. Depending on your platform, it will either do nothing, freeze, raise exceptions, create two separate copies of the GUI, or, if you're really unlucky, unpredictably work sometimes but do one of the other things other times. — abarnert, Jun 26 '18 at 17:58
One option is to proxy canvas-drawing commands through a `Pipe` or `Queue` to have the main process execute them. Since the only canvas-drawing command you're probably using is `create_image`, this should be dead easy. In fact, you could probably just use a `Pool` or `ProcessPoolExecutor` and have background tasks that run on the child processes and `return` the image to draw. — abarnert, Jun 26 '18 at 18:01
Another option is to draw to an off-screen `Canvas` then get the (bitmap, PS, whatever) contents to pass that to the parent process to blit onto its `Canvas`. — abarnert, Jun 26 '18 at 18:02
@abarnert the only problem actually is that the images that its sending to the main drawing core is actually pre-cached, however there are alot of update per second to be made expect around 400+ updates to the canvas per second. Thus im wondering if a single 1.2GHz core enough to draw 400+ small image updates from pre-cached images? I know its hard to imagine — DiscreteTomatoes, Jun 26 '18 at 18:08
If displaying the images to the `Canvas` is your performance bottleneck, `multiprocessing` isn't going to help you. You'll probably need to switch to a different GUI library. Qt (via `PyQt5` or `PySide2`), `wx`, etc. should be able to handle a lot more. Or native platform wrappers like `pyobjc` or `PyWin32` or one of the `Xlib` wrappers. If not, you have to go directly to OpenGL (there are a few Python wrappers for that), or at least to something that wraps OpenGL directly like SDL (e.g., via `PyGame`). — abarnert, Jun 26 '18 at 18:12
Also, if you have a single core, `multiprocessing` isn't going to help _anything_ (unless you need crash isolation or multiple 32-bit virtual memory spaces or something—but it's not going to help performance), and it is adding overhead. Why not just do it all in-process, either synchronously or with threads? Sure, if the processing isn't happening in GIL-releasing libraries like NumPy, the threads will end up serialized—but if you have a single core, your processes end up just as serialized. — abarnert, Jun 26 '18 at 18:13
@abarnert so what i'm gathering is that the best way to do this is load all the pre-cached images into one core, and use an efficient language or library to get as many draws as possible? — DiscreteTomatoes, Jun 26 '18 at 18:26
Before looking for "an efficient language or library", first see whether tkinter really can't handle it. Do you have existing code that's failing? If so, do you know that rendering the images is the bottleneck? Until you know that, don't rewrite your whole program to optimize something that might not be relevant. — abarnert, Jun 26 '18 at 18:31

score 2 · Accepted Answer · answered Jun 26 '18 at 18:09

No, it is not possible. GUI widgets cannot be shared between processes. On some platforms, it just isn't possible at all; on other platforms, it would be possible, but only by doing things very differently from the way Tk does; on others, it sort of works, but the event loops are all screwed up. So, the result may be that nothing shows up, that one or both processes freezes, that the GUI doesn't respond to events, that tkinter raises an exception in the child, that tkinter creates a whole separate independent GUI, or, if you're really unlucky, that things unpredictably work sometimes but do one of the other things other times.

However, that doesn't mean there's no way to do what you want, just that you can't do it directly.

The simplest solution is to marshal your Canvas commands and pass them over a Pipe or Queue for the main process to execute.

A fully general solution isn't that hard, but in your case, it should be even simpler: all you want the background process to do is process an image and then display it. So the only Canvas command you need is create_image.

And, in fact, you can probably do with tasks on a Pool, which just return the image when they're done, with the main process doing the create_image with the results.

Mixing waiting on multiprocessing async results with a tkinter event loop is a bit of a pain, but if you use concurrent.futures, you can just attach the create_image as a callback on the Future returned by the task.

A different option is to have the background processes create off-screen Canvas objects, draw to them, then capture the results as a BitmapImage or a postscript rendering, which you can then pass to the main process to blit onto a Canvas of its own. But this is a lot more complicated; I think the other solution will probably work a lot better for you.

Python Multiprocessing Drawing to Tkinter Canvas

1 Answers1