1

I am creating a PyQt5 application and want to run some ML code in a separate thread when a button is clicked. I am unable to do so. This is my threading code:

newthread = QtCore.QThread()
runthread = RunThread()
runthread.moveToThread(newthread)
runthread.finished.connect(newthread.quit)
newthread.started.connect(runthread.long_run)
newthread.start()

And the class RunThread is as follows:

class RunThread(QtCore.QObject):

finished = QtCore.pyqtSignal()

def __init__(self):
    super().__init__()

@pyqtSlot()
def long_run(self):
    #ML code called in the line below
    simulation.start_simulation()
    self.finished.emit()

Running it normally doesnt work. Pycharm quits with the following error:

process finished with exit code -1073740791 (0xc0000409)

Running it in debug mode throws thousands of warnings thrown by sklearn saying:

Multiprocessing-backed parallel loops cannot be nested below threads,
setting n_jobs=1

The code runs eventually, but it takes a lot more time (at least 4x times) to do so.

Could someone please let me know what the issue here is?

Isma
  • 14,604
  • 5
  • 37
  • 51
javapyscript
  • 720
  • 8
  • 22

1 Answers1

1

The warning from sklearn is pretty explicit here. Basically you cannot train sklearn models with hyperparameter n_jobs set to anything greater than 1 when executing from a nested thread.

That being said, without seeing what is going on inside of simulation.start_simulation() it's hard to conjecture.

My best guess would be to look for anything in start_simulation() that is using multiple jobs and see what happens when you set that to n_jobs=1.

Alternatively, you may try writing your ML code as a standalone script and execute it using QProcess. That may allow you to utilize n_jobs greater than 1.

Grr
  • 15,553
  • 7
  • 65
  • 85
  • Thanks for the quick response. I was given the ML code by another dev (I am assigned to work on UI) and I just checked that n_jobs was never set, so it always took the default value 1. The ML script works standalone, and also works if I import and call it in the UI thread (thereby making the app unresponsive). Doesnt work when threaded. Will check what QProcess does. Thank you. – javapyscript Jul 11 '18 at 13:28
  • Unfortunately QProcess might not work as I would have to provide the path of the python executable, which will differ per user. I found a link on SO which might be related, but dont know if it even fixes my issue: https://stackoverflow.com/questions/27646052/multiprocessing-backed-parallel-loops-cannot-be-nested-below-threads – javapyscript Jul 12 '18 at 08:14