7

Background

I am working on face recognition following this link and I would like to build a standalone application using Python. My main.py script looks like the following.

# main.py

# Import packages and other scripts
import tkinter as tk
...

# Some functions
def initialization():
    # OpenCV and sklearn are used here
    ...

def registration():
    # OpenCV, and sklearn are used here
    ...

def face_recognition():
    # OpenCV and sklearn are used here
    ...

# Start the Tkinter GUI
window = tk.Tk()

# Initialize the face recognition model
initialization()

# Input name for registration
tk.Label(window, text = "Name").grid(...)
entry1 = tk.Entry(window)
entry1.grid(row=0, column=1)

# When the button is clicked, different command is executed
tk.Button(window, text='Registration', command=registeration).grid(...) 
tk.Button(window, text='Face Recognition', command=face_recognition).grid(...)

window.mainloop()

Using python interpret to run the script (python main.py), everything works fine.

Problem

I use Pyinstaller to convert the scripts to a single exe with this command:

pyinstaller --onefile \
            --windowed \
            --hidden-import sklearn.neighbors.typedefs \
            main.py

Then, I have two exe generated. The first one is in dist/main and the second one is in dist/main.app/Contents/MacOS/main

When running the exe, the exe keeps duplicate itself. I display the running process and found this result

$ ps
PID    TTY    TIME    CMD
...    ...    ...     /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)
...    ...    ...     /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)
...    ...    ...     /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(7)
...    ...    ...     /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(7)
...    ...    ...     /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)

I have no idea what happens to the exe since I do not import multiprocessing packages in my scripts. Any idea how to fix this problem? Thanks

Update 1

I added a --log-level ERROR when using Pyinstaller and gives this result. Not sure if it is related to my problem.

Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'Crypto.Math'
174598 INFO: MKL libraries found when importing numpy. Adding MKL to binaries
176282 ERROR: Can not find path ./libtbb.dylib (needed by /Users/user/anaconda3/lib/libmkl_tbb_thread.dylib)

Update 2

I found out that one of the packages that I am using -- imultis VideoStream involves threading. I guess it is the reason for observing from multiprocessing.semaphore_tracker as shown above even though I do not import multiprocessing explicitly.

Then I come across with this post, suggesting to add multiprocessing.freeze_support(). It can suppress the GUI window from keep duplicating, but the background processes as shown using $ ps still keep duplicating. The problem has not fixed yet.

Update 3 (Issue located but not fix)

After debugging the code for quite a while, it turns out that the causes of this infinite looping is NOT due to the threading of imultis VideoStream (I write another script to test imultis and it is completely OK). But the problem comes from importing sklearn!!! This problem has been reported in GitHub in this link.

This GitHub link also suggest to include multiprocessing.freeze_support(). Moreover one of the responses suggests to import sklearn after multiprocessing.freeze_support() is called. I try it with a simple script but the problem is still here.

Conclusion: sklearn causes the executable keeps opening. But I have no idea why it is the case, and I do not know how to solve it. Any help would be appreciated. Thanks.

Community
  • 1
  • 1
K_inverse
  • 357
  • 3
  • 16
  • 1
    may be you need to use **waitkey** in your **main.py** to wait for some keys to destroy your window. And also use **destroyAllWindows**. – Ghantey Feb 13 '19 at 04:07

3 Answers3

2

This answer does a good job explaining why this happens: https://stackoverflow.com/a/55382641/109525

First try setting this before starting your program:

export JOBLIB_MULTIPROCESSING=0

If it works, you can add a runtime hook to your program to set the environment variable automatically. See: https://pythonhosted.org/PyInstaller/when-things-go-wrong.html#changing-runtime-behavior

0x26res
  • 11,925
  • 11
  • 54
  • 108
  • I have the same issue with an app that uses `sklearn` and is built with `PyInstaller` for the Mac. I was able to work around this issues by adding this at the start of my app: `if getattr(sys, 'frozen', False): os.environ['JOBLIB_MULTIPROCESSING'] = '0'` – Matthew Self Jan 05 '21 at 04:40
1

I met the same issue. Scikit-learn provides a workaround.

https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux

I added below codes at the beginning of my script and fixed my issue:

import multiprocessing
multiprocessing.set_start_method('forkserver', force=True)
multiprocessing.freeze_support()

Hope this can also fix yours.

FredF
  • 11
  • 1
  • 1
    Thanks. now the program does not fork processes infinitely But still, when I check the background process, it shows `/path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(7)`. Any idea why there are such processes? – K_inverse Feb 18 '19 at 07:46
1

Starting my main entry point with:

from multiprocessing import freeze_support
freeze_support()

Worked for me!

user3394381
  • 41
  • 1
  • Worked perfectly! When trying to run a parallel process from within `wxpython` it just started a new UI instead. For reference, here's how I wrapped the UI function to make it work with multiprocessing: `from multiprocessing import freeze_support; from my_gui import open_gui; freeze_support(); open_gui()` Then export this script with `pyinstaller` – Justas Oct 18 '21 at 12:31