1

I am looking to use multiprocessing or threading in my application to do some time-consuming operations in the background. I have looked at many examples, but I still have been unable to achieve what I want. I am trying to load a bunch of images, each of which takes several seconds. I would like the first image to be loaded and then have the others loading in the background and being stored in a list (to use later) while the program is still doing other things (like allowing controls on my GUI to still work). If I have something like the example below, how can I do this? And should I use multiprocessing or threading?

class myClass():
    def __init__(self, arg1, arg2):
        #initializes some parameters

    def aFunction(self):
        #does some things
        #creates multiple processes or threads that each call interestingFunc
        #continues doing things

    def interestingFunc(self):
        #performs operations

m = myClass()
tshepang
  • 12,111
  • 21
  • 91
  • 136
SFBA26
  • 870
  • 3
  • 12
  • 24

3 Answers3

1

Here's the simplest possible way to do multiple things in parallel, it'll help get you started:

source

import multiprocessing

def calc(num):
    return num*2

pool = multiprocessing.Pool(5)
for output in pool.map(calc, [1,2,3]):
    print 'output:',output

output

output: 2
output: 4
output: 6
johntellsall
  • 14,394
  • 4
  • 46
  • 40
1

You could try something like this:

from thread import start_new_thread

pictureList = [ f for f in os.listdir(r"C:\your\picture\folder")]
for pic in pictureList:
    start_new_thread(loadPicture,(pic,))

def loadPicture(pic):
    pass # do some stuff with the pictures

This is a quite simple approach, the thread returns immediatley and perhaps you'll need to use allocate_lock. If you need more capabilities, you might consider using the threading module. Be careful to pass a tuple as 2nd argument to the thread.

user2366975
  • 4,350
  • 9
  • 47
  • 87
1

You can use either approach. Have your Process or Thread perform its work and then put the results onto a Queue. Your main thread/process can then, at its leisure, take the results off the queue and do something with them. Here's an example with multiprocessing.

from multiprocessing import Process, Queue

def load_image(img_file, output_q):
    with open(img_file, 'rb') as f:
        img_data = f.read()
        # perform processing on img_data, then queue results
        output_q.put((img_file, img_data))

result_q = Queue()
images = ['/tmp/p1.png', '/tmp/p2.jpg', '/tmp/p3.gif', '/tmp/p4.jpg']

for img in images:
    Process(target=load_image, args=(img, result_q)).start()

for i in range(len(images)):
    img, data = result_q.get()
    # do something with the image data
    print "processing of image file %s complete" % img

This assumes that the order of processing is not significant to your application, i.e. the image data from each file might be loaded onto the queue in any particular order.

mhawke
  • 84,695
  • 9
  • 117
  • 138