Asynchronous import in Python

Question

I need to import something large and computationally expensive but don't need it right away in my application. Is there a way to import something asynchronously in python, i.e. have whatever the import needs to do be done in the background while my script is running? Google has not been helpful so far.

Its a python module and you do `import foo` but its computationally expensive to load? You can only get so much speed up because python's cooperative multithreading only allows one thread to run at a time. — tdelaney, Apr 25 '18 at 16:06

score 3 · Accepted Answer · answered Apr 25 '18 at 16:23

You can call the python importer as a function in a different thread instead of using import foo. Since this import is computationally expensive and python only allows one thread to run at a time (unless its something like pandas that releases the GIL), you may find little benefit. Still,

import threading
import time

def begin_load_foo():
    global foo_thread
    foo_thread = threading.Thread(target=load_foo_thread)
    foo_thread.start()

def load_foo_thread():
    global foo
    print("start importing")
    foo = __import__('foo')
    print("done importing")

def wait_foo():
    print('wait')
    foo_thread.join()
    print('done')

def do_other_things():
    time.sleep(1)

begin_load_foo()
do_other_things()
wait_foo()
foo.bar()

Thank you! My import is slow mainly because it reads stuff from the disk and instantiates an object that does numpy stuff, all of which happens outside the GIL, so I should see an increase in performance regardless. — Syncrossus, Apr 26 '18 at 10:06

Asynchronous import in Python

1 Answers1