3

In my current system, all directories such as .local/lib/python2.7/site-packages are under global admin control and are standardized across users.

However, I would like to add a collection of my own directories to be brought into the PYTHONPATH on startup, preferably in an easy-to-manage way like using a .pth file within a secondary site directory.

Is there a way I can achieve this through normal Python channels, but without needing to make any kind of change to the standard locations where Python looks for path elements?

I'm specifically wondering if there is a way to just place a file like sitecustomize.py or usercustomize.py anywhere in a folder that is in the PYTHONPATH and expect that they will be automatically executed? (Rather than getting this behavior by placing such files within an admin directory like site-packages, which is what I want to avoid.)

My goal is to place a text file, paths.pth in some directory, like /home/my_configs/python_stuff/ and then just need to place that directory only in the PYTHONPATH via, e.g. a .bashrc file. Then, at least, I only manually maintain one path to be added, and it's easier to version control or parameterize what gets loaded by changing what is in that .pth file.

If the admin limitations weren't in place, I could trivially do this in site-packages but I need a solution that works totally outside of that.

In /.bashrc I have this:

export PYTHONPATH=/home/ems/python_paths/:$PYTHONPATH

In the directory /home/ems/python_paths/ I have just two files, sitecustomize.py and paths.pth (but I've also tried adding an __init__.py there too, which (as expected) didn't help). These two files look like this:

sitecustomize.py

import site
site.addsitedir("/home/ems/python_paths/")

paths.pth

/home/ems/ws/Research/Projects/python-util/src/util/
/home/ems/ws/Research/Projects/python-efficacy/src/efficacy/

The contents of paths.pth are just exactly what used to be directly exported in .bashrc and, for instance, permits me to do from util.DataManager import DataManager as a top-level import.

But after setting up the various files above, commenting out exports to PYTHONPATH in .bashrc, and getting a fresh terminal, I see this:

ems@computer ~ $ python
Enthought Python Distribution -- www.enthought.com
Version: 7.3-2 (64-bit)

Python 2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 11 2012, 17:52:16)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "credits", "demo" or "enthought" for more information.
Hello
>>> from util.DataManager import DataManager
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named util.DataManager
Community
  • 1
  • 1
ely
  • 74,674
  • 34
  • 147
  • 228
  • The answer to the question you linked to already says how to do this. You need to put the `sitecustomize.py` in the directory on the Python path, and have it use `addsitedit` to add `/some/other/dir` as a site dir, and then `.pth` files in `/some/other/dir` will work. You could also use the [user-specific packages directory](http://www.python.org/dev/peps/pep-0370/). – BrenBarn Nov 11 '13 at 19:56
  • No, this does not work. If you read the `site` docs, the file `sitecustomize.py` or `usercustomize.py` seems to be required to appear in `site-packages` *not in* the "other" folders that are on the path. It won't auto-execute them unless it finds them in one of the special places, and I'm asking how to get around that. – ely Nov 11 '13 at 20:05
  • Yes, it does. As that answer and the docs say, Python just does `import sitecustomize`, which will work as long as your `sitecustomize` is anywhere on the Python path. – BrenBarn Nov 11 '13 at 20:10
  • This does not work for me. I have `/home/ely/configs/python_path/` exported to `PYTHONPATH` in `.bashrc`, I have an `__init__.py` file in there, as well as `sitecustomize.py` (also tried `usercustomize.py`), which contain the relevant code to add my `.pth` file. If I source the `.bashrc` after doing this, the `sitecustomize.py` file *does not* get executed on startup. – ely Nov 11 '13 at 20:19
  • Please edit your question to show the contents of your `sitecustomize.py` and the contents of `sys.path`. Also, why do you have `__init__.py`? You don't want that if you want to import `sitecustomize.py` directly (which is what `site` will do), and that could be confusing it. – BrenBarn Nov 11 '13 at 20:35
  • I tried the `__init__.py` thing just because it was not working without it, but adding it did not help. Will edit with details. – ely Nov 11 '13 at 20:39
  • Is the `utils` directory a package? That is, does `utils` contain an `__init__.py`? If it does, you should have its parent directory (i.e., `/home/ems/ws/Research/Projects/python-util/src/`) in the .PTH file. – BrenBarn Nov 11 '13 at 23:28
  • Again, these *exact* paths worked fine when manually exported through .bashrc. Those paths and the respective `__init__.py` files are not the problem. These were all among the first things I tried, but thanks for trying anyway. – ely Nov 12 '13 at 13:57
  • It's hard to debug these kinds of problems over the internet because there are so many fiddly little details that can make it not work. I have essentially the same setup you describe and it works perfectly. – BrenBarn Nov 12 '13 at 19:53
  • Very true. I will keep working on it and update the post if I discover what setting is different for me that is preventing it. – ely Nov 12 '13 at 20:41
  • @EMS Is this still relevant? I might have some answers. – Korem Sep 08 '14 at 18:52

1 Answers1

1

There was PEP 370 specifically addressing the creation of per-user site-packages directories, to deal with the situation where the user has no admin access to the system-wide site-packages.

For example on Unix (including Mac OS), and assuming one is using Python 3.6, one can create the following directory and place .pth files inside there

~/.local/lib/python3.6/site-packages

divenex
  • 15,176
  • 9
  • 55
  • 55