How to check if any module in a Python package imports from another package?

Question

I want to ensure all modules within one package ("pkg-foo") don't import from another package ("pkg-block").

Update: I know there are many black magic ways to import modules due to Python's dynamism. However, I am only interested in checking explicit imports (e.g. import pkg.block or from pkg.block import ...).

I want to enforce this via a unit test in pkg-foo that ensures it never imports from pkg-block.

How can I accomplish this? I use Python 3.8+ and am looking to use either built-ins or perhaps setuptools.

Current Half-Baked Solution

# pkg_resources is from setuptools
from pkg_resources import Distribution, working_set

# Confirm pgk-block is not in pkg-foo's install_requires
foo_pkg: Distribution = working_set.by_key[f"foo-pkg"]
for req in foo_pkg.requires():
    assert "pkg-block" not in str(req)

However, just because pkg-block is not declared in setup.py's install_requires doesn't mean it wasn't imported within the package. So, this is only a half-baked solution.

My thoughts are I need to crawl all modules within pkg-foo and check each module doesn't import from pgk-block.

Answer 1

So my suggestion is to conceptually split this problem into two parts.

First sub-problem: determine all of the modules imported in pkg-foo. Let's use mod_foo to be some arbitrary imported module in pkg-foo

Second sub-problem: determine if any mod_foo are from pkg-block. If none of these modules are in pkg-block, pass the unit test, else, fail the unit test.

To solve the first sub-problem you can use the class modulefinder.ModuleFinder. As shown in the example from the documentation, you can do modulefinder.ModuleFinder.run_script(pathname) for each module in pkg-foo. Then you can get the module names by grabbing the keys from the dict modulefinder.ModuleFinder.modules. All of these modules will be your mod-foo modules.

To solve the second sub-problem, you can use mod_foo.__spec__ As mentioned here, mod_foo.__spec__ will be an instance of 'importlib.machinery.ModuleSpec' which is defined here. As described in the documentation just linked to, this object will have the attribute name which is:

A string for the fully-qualified name of the module.

Therefore we need to check to see if pkg-block is in the fully qualified name given by mod_foo.__spec__.name for each mod_foo.

Putting this all together, something along the lines of the following code should do what you need:

import modulefinder

def verify_no_banned_package(pkg_foo_modules, pkg_ban):
    """
    Package Checker
    :param pkg_foo_modules: list of the pathnames of the modules in pkg-foo
    :param pkg_ban: banned package
    :return: True if banned package not present in imports, False otherwise
    """

    imported_modules = set()

    for mod in pkg_foo_modules:
        mod_finder = modulefinder.ModuleFinder()
        mod_finder.run_script(mod)
        mod_foo_import_mods = mod_finder.modules.keys()
        imported_modules.update(mod_foo_import_mods)

    for mod_foo in imported_modules:
        mod_foo_parent_full_name = mod_foo.__spec__.name
        if pkg_ban in mod_foo_parent_full_name.split(sep="."):
            return False
    return True

Answer 2

I think the easiest way in your case is to mock a module that shouldn't be imported, i.e. pkg-block. In order to mock a module put it to the very beginning of sys.path, probably the easiest way to do so is to create a fake module (or temporary replace original one) in the project folder, so it would end up in your PYTHONPATH immediately. This will "redirect" all imports to your mock module. In the mock module itself put something like this:

print(f'Module {__name__} imported')


def __getattr__(name):
    print(f'Imported attribute {name} from module {__name__}')

The code above will print a line informing you that module or its attribute was imported, this would work for direct module imports and from imports, for imports inside functions, for imports with importlib (except importing by path, but I find this too esoteric). In order to simplify detection of imports in unit tests you probably want to raise an exception instead of just printing. You also don't have to have specific tests to verify that imports are not happening, instead just run your regular test suite for pkg-foo and see if any tests are failing due to import-specific exception.

Example:

Assume the following file structure:

|---main.py
|
\---pkg
    |---block.py
    |---foo.py
    |---__init__.py

where pkg/block.py

print(f'Module {__name__} imported')


def __getattr__(name):
    print(f'Imported attribute {name} from module {__name__}')

pkg/foo.py

import importlib

import pkg.block
from . import block
from .block import abc

importlib.import_module('pkg.block')


def from_function():
    from pkg.block import from_function

main.py

import pkg.foo

if __name__ == '__main__':
    pkg.foo.from_function()

then after executing main.py you'll get:

Module pkg.block imported
Imported attribute __path__ from module pkg.block
Imported attribute abc from module pkg.block
Imported attribute abc from module pkg.block
Imported attribute __path__ from module pkg.block
Imported attribute from_function from module pkg.block
Imported attribute from_function from module pkg.block