0

I have the following function:

def rescale(file):
    img = cv2.imread(file)
    return img

I am trying to upscale a series of images, but each image takes 20 seconds to process. I want to use multithreading in order to create a faster program.

I have a list of my images :

 file_list = ['page-136.jpg',
'page-0.jpg',
'page-11.jpg',
'page-12.jpg',
'page-13.jpg',
'page-14.jpg',
'page-37.jpg',
'page-58.jpg',
'page-62.jpg',
'page-64.jpg',
'page-134.jpg',
'page-135.jpg']

and I have the following code to add multithreading:

import tqdm
from concurrent.futures import ThreadPoolExecutor, as_completed

with ThreadPoolExecutor(max_workers=1000) as executor:
    future_to_response = {
        executor.submit(Upscale, i): i for i in file_list
    }
    t = tqdm.tqdm(total=len(future_to_response))

    for future in as_completed(future_to_response):
        domain = future_to_response[future]
        result = future.result()      
        t.update()

This however does not work, and I appear to be stuck. Can someone guide me in the right direction?

  • Please describe what you mean by "this however does not work." Do you get a traceback? Please post it. Does the output differ from what you expect? Tell us how it differs. Is it not any faster? Consider using processes instead of threads, this is likely a CPU-bound operation. – Michael Ruth Dec 06 '22 at 22:39
  • 1
    A few seconds with the search engine of your choice ("python threading") would tell you about the GIL and why threading is probably a bad idea for your use case. Also, what does _"does not work"_ mean? – Daniil Fajnberg Dec 06 '22 at 22:39

2 Answers2

0

Instead of collecting the results in a container you simply assign it to a variable every iteration. When iteration stops result will only point-to the result of the last future to complete

....
    result = []
    for future in as_completed(future_to_response):
        ...
        result.append(future.result())
        ...
wwii
  • 23,232
  • 7
  • 37
  • 77
-1

You could use the threading library:

import threading
threads_lst = []
for file in file_list: 
    threads_lst.append(threading.Thread(target=Upscale, args=(file,)))
    threads_lst[-1].start()

for t in threads_lst:
    t.join()
nokla
  • 451
  • 1
  • 8