How can I transparently redirect a Python import?

Question

I'm looking for a way to emulate symlinks for Python imports. I'd like to be able to unzip the following folder structure in-place without duplicating files:

root
├─ python_lib
│  └─ my_utils
│     ├─ __init__.py
│     └─ etc.py
├─ app1
├─ app2
└─ app3
   ├─ lib
   │  ├─ __init__.py
   │  └─ my_utils.py
   └─ run.py

app3/run.py contains this:

from lib.my_utils import etc

etc.pancakes()

I'd like the code to use the etc located in python_lib/my_utils/. Is there anything I can put in app3/lib/my_utils.py so that Python >= 3.1 will transparently import the python_lib/my_utils/ folder (using relative paths and ..), and subpackages will also work?

score 8 · Answer 1 · answered Feb 01 '11 at 21:06

8

You will have to execute something before app3/run.py reaches the import statement.

import python_lib
import sys
sys.modules['lib'] = python_lib
# ...
from lib import etc
print etc.__file__
print dir(etc)

answered Feb 01 '11 at 21:06

Apalala

9,017
3
30
48

score 3 · Answer 2 · answered Jan 31 '11 at 20:51

3

You should add this path into sys.path. For example:

lib_path = os.path.abspath( os.path.split( os.getcwd()+"/"+sys.argv[0] )[0]+"/../_lib/my_utils/" )
sys.path.append(lib_path)

answered Jan 31 '11 at 20:51

Elalfer

5,312
20
25

This wouldn't work if put in `my_utils.py`, since the import location is already resolved by then. If I run this before my main code, it still won't work, because each subpackage in `my_utils` would appear top-level to Python, and then relative imports within `my_utils` wouldn't work. – zildjohn01 Jan 31 '11 at 21:05
I usually set paths from my `run` script. Otherwise it doesn't make much sense. You could also try to play around with `PYTHONPATH` environment variable. More info here http://docs.python.org/tutorial/modules.html#the-module-search-path – Elalfer Jan 31 '11 at 21:19

score 1 · Answer 3 · edited Nov 13 '14 at 15:31

1

Add __init__.py to the python_lib folder app3/run.py contains

import python_lib.my_utils
import os

sys.modules['lib.my_utils'] = python_lib.my_utils

edited Nov 13 '14 at 15:31

n611x007

8,952
8
59
102

answered Jul 10 '14 at 22:25

user3827333

69
6

score 0 · Answer 4 · answered Feb 09 '13 at 09:35

How about this? (Yes, put it at app3/lib/my_utils.py.)

import os
_f = os.path.realpath(__file__)
_f = os.path.dirname(_f)
_f = os.path.dirname(_f)
_f = os.path.dirname(_f)
_f = os.path.join(_f, 'python_lib')
def f():
    path = sys.path
    path.insert(0, _f)
    sys.modules['lib.my_utils'] = __import__('my_utils')
    path.pop(0)
f()

Michael Allan Jackson · Answer 5 · 2012-03-06T01:26:27.503

I made the following changes to your directory structure to simplify the example:

create root/python_lib/__init__.py
rename root/python_lib to root/lib
replace root/app3/lib/ with lib.py

Place the following code in root/app3/lib.py

import os
import sys

pth = os.path.sep.join(sys.argv[0].split(os.path.sep)[0:-2])
sys.path.insert(0, pth)
del sys.modules[__name__]
import lib

Essentially, we have a dummy module that replaces its own reference in sys.modules with a reference to a module or package in another location.

Dunes · Answer 6 · 2014-11-13T17:40:08.660

Very late to this, but this is incredibly easy. All you need to do is modify the __path__ attribute of your package so it points to the folder where your modules are. This will require lib to be a package, not just a module (ie. a folder with an __init__.py). By default __path__ is a one element list that contains the path of folder that is the package -- ['/path/to/lib']. Meaning, that by default, python searches for sub modules in the module folder (seems like a very logical thing to do).

You can alter the contents of path to point to where ever you want, but it must contain absolute paths. If you only append path/to/lib_python then python will search lib first, find my_utils.py and then stop. So in that case you will need to delete my_utils.py or put the new path at the front of __path__ so that location is searched first. This also means that the lib folder can contain its own supplementary modules to the shared libraries and still work.

In practice:

lib/__init__.py

from os.path import join
__path__.insert(0, join(__path__[0], "..", "..", "python_lib"))

How can I transparently redirect a Python import?

6 Answers6

Linked