Proper way of implementing post import hooks in python-3.4+

Question

I want to modify the import behavior. I found an example with post import hook in the Book "Python Cookbook" by David Beazley and Brian K. Jones, which should suit my problem.

Since the latest edition was published at the times of python-3.3 the provided example is outdated. I had to modify it myself, so that the code is compatible with importlib after python-3.4. In the original version in PostImportFinder classfind_module(self, fullname, path = None) is defined and in PostImportLoader instead of create_module() load_module() was defined. Here is a small reproducible example:

#postimport.py
import
import importlib
import sys
from collections import defaultdict

_post_import_hooks = defaultdict(list)

class PostImportFinder:
    def __init__(self):
        self._skip=set()
    
    def find_spec(self, fullname, path = None, target = None):
        if fullname in self._skip:
            return None
        self._skip.add(fullname)
        return PostImportLoader(self)


class PostImportLoader:
    def __init__(self, finder):
        self._finder = finder
    
    def create_module(self, spec):
        importlib.import_module(spec.name)
        module = sys.modules[spec.name]
        for func in _post_import_hooks[spec.name]:
            func(module)
        self._finder._skip.remove(spec.name)
        return module
    
def when_imported(names):
    def decorate(func):
        for fullname in names:
            if fullname in sys.modules:
                print(f'importing {fullname}')
                func(sys.modules[fullname])
            else:
                _post_import_hooks[fullname].append(func)
        return func
    return decorate

sys.meta_path.insert(0,PostImportFinder)

#postimportfunc.py
from inspect import getmembers, isfunction, isclass
from postimport import when_imported


list_of_module_names = ['simple']
#Some decorator. For example purposes pretty simple
@when_imported(list_of_module_names)
def decorate(mod):
    # Decorate classes
    print(f'module {mod} imported')

#simple.py
class A:
    def __init__(self):
        self.x=42
    
    def bar(self):
        print(self.x)

#start.py
import postimportfunc
from simple import A


foo = A()
foo.bar()

When I run the start.py following error occurs:

    runfile('/home/user/reproduce/start.py', wdir='/home/user/reproduce')

  File "/usr/lib/python3/dist-packages/spyder_kernels/customize/spydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "/usr/lib/python3/dist-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/user/reproduce/start.py", line 2, in <module>
    import postimportfunc

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 971, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 914, in _find_spec

  File "/home/user/reproduce/postimport.py", line 12, in find_spec
    if fullname in self._skip:

AttributeError: 'str' object has no attribute '_skip'

So my questions are obvious: What am I doing wrong? I simply do not see how _skip and str are related, since I initialize _skip as a set. Is there another/better approach how to change postimport behavior for specified modules?

Upd: P.S. I forgot to mention, that I currently use python-3.7 .

score 1 · Accepted Answer · answered Oct 17 '21 at 22:05

The main cause was a dumb typo in the last line of postimport.py. Instead of:

sys.meta_path.insert(0,PostImportFinder)

It should be:

sys.meta_path.insert(0,PostImportFinder()) #DO NOT FORGET PARENTHESIS

But it was not the only problem. I had also to change following things:

In the PostImportFinder rename find_spec(self, fullname, path = None, targer = None) into find_module(self, fullname, path = None) and in PostImportLoader replace find_spec()with:

def load_module(self, fullname):
        importlib.import_module(fullname)
        module = sys.modules[fullname]
        for func in _post_import_hooks[fullname]:
            func(module)
        self._finder._skip.remove(fullname)
        return module

So, to sum it up: I should have sticked to the recipe from the book mentioned in the question.

Proper way of implementing post import hooks in python-3.4+

1 Answers1