0

I am currently using worker threads in Python to get tasks from a Queue and execute them, as follows: 

from queue import Queue
from threading import Thread

def run_job(item)
    #runs an independent job...
    pass    

def workingThread():
    while True:
        item = q.get()
        run_job(item)
        q.task_done()

q = Queue()
num_worker_threads = 2

for i in range(num_worker_threads):
    t = Thread(target=workingThread)
    t.daemon = True
    t.start()

for item in listOfJobs:
    q.put(item)

q.join()

This is functional, but there is an issue: some of the jobs to be executed under the run_job function are very memory-demanding and can only be run individually. Given that I could identify these during runtime, how could I manage to put the parallel worker threads to halt their execution until said job is taken care of?

Edit: It has been flagged as a possible duplicate of Python - Thread that I can pause and resume, and I have referred to this question before asking, and it surely is a reference that has to be cited. However, I don't think it adresses this situation specifically, as it does not consider the jobs being inside a Queue, nor how to specifically point to the other objects that have to be halted.

fabio.avigo
  • 308
  • 1
  • 13
  • Possible duplicate of [Python - Thread that I can pause and resume](https://stackoverflow.com/questions/33640283/python-thread-that-i-can-pause-and-resume) – psukys Oct 08 '18 at 19:46

1 Answers1

0

I would pause/resume the threads so that they run individually. The following thread Python - Thread that I can pause and resume indicates how to do that.

jmartori
  • 384
  • 1
  • 5
  • 14
  • This is hardly any help, Jordim. I did my homework before asking here. Please refer to [How do I write a good answer](https://stackoverflow.com/help/how-to-answer) before answering. – fabio.avigo Oct 10 '18 at 16:33