0

Scenario

From a package let's say beta:
After modifying the sys.path, importing a package alpha then reverting the sys.path. I try to import a module data_provider that exists in both alpha and beta

The issue is: the data_provider in alpha gets picked over beta even though sys.path now has no traces of alpha's directories

Question: Is that a bug, or there's some other places than sys.path (may be caches) that python looks at when trying to import a module

Code

import os, sys

sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)),"src"))

sys.path.insert(0, '../alpha')
import alpha
sys.path.remove('../alpha')

import data_provider as dp

print(sys.path)
print(dp.__file__)

The full codebase could be found here

weshouman
  • 632
  • 9
  • 17
  • 1
    when you imported `alpha` all the modules that were present in the `alpha.__init__.py` file were imported and cached in `sys.modules`. Now `data_provided` will be imported from `sys.module` if it was earlier present in `alpha` otherwise it will look according to paths present in `sys.path`. – Vishal Singh Jun 16 '21 at 11:32
  • do you know what is the cleanest way to invalidate the caches of all `alpha`'s modules? I've tried to reload the imported modules, but unfortunately, reloading doesn't result in having all the members of `beta`'s `data_provider` defined, only the common one `get_tests()` could be found but not `get_products()` – weshouman Jun 16 '21 at 12:39
  • you've only presented the problem but what's your end goal? – Vishal Singh Jun 16 '21 at 12:47
  • I would like to be able to call other packages using absolute import in a clean way, taking into consideration that, due to similar design patterns, some module names collide – weshouman Jun 16 '21 at 13:16

1 Answers1

1

The first place checked during import search is sys.modules. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths. So if foo.bar.baz was previously imported, sys.modules will contain entries for foo, foo.bar, and foo.bar.baz. Each key will have as its value the corresponding module object.

During import, the module name is looked up in sys.modules and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is None, then a ModuleNotFoundError is raised. If the module name is missing, Python will continue searching for the module.

Read more about it here https://docs.python.org/3/reference/import.html#the-module-cache.

Vishal Singh
  • 6,014
  • 2
  • 17
  • 33