9

With the folder structure

lib/
    abcd/
        __init.py__
        lib.py
app.py

the code

from lib.abcd import lib

works. But with this file structure: 

bin/
    app.py
lib/
    abcd/
        __init.py__
        lib.py

the code 

from ..lib.abcd import lib

gives an import error.

How to do the import properly when the library is in a sibling of the current folder? (or subfolder of a sibling folder)

I know that there might some hack that involves adding lib/ to the PATH, but is there an elegant Pythonic solution?

If not, is there a real internal reason to prevent users to do this simple import in a simple way?

Basj
  • 41,386
  • 99
  • 383
  • 673
  • 2
    Normally you just cannot. As your top level directory is `bin`. To achieve it, you need to add parent directory into `sys.path`. And then import it as `from lib.abcd import lib`. – Sraw Feb 10 '19 at 23:35
  • 1
    Thanks for your comment @Sraw. Isn't there an easier solution? – Basj Feb 10 '19 at 23:45
  • 4
    Surely there is... Moving that `lib` to `bin`. I understand your project strucutre. If you want to insist on it, I recommend you to add a bash script for launching purpose. In that script you can use `PYTHONPATH=your_parent_directory python app.py` to launch your app. – Sraw Feb 10 '19 at 23:57
  • Just to clarify, based on the code and these comments: Is there no parent folder here? Is this the _complete_ folder structure your project? – Chris Larson Feb 11 '19 at 01:00
  • 2
    IMHO you are doing it wrong. You should distinguish between the layout of the source code and the layout of an installed software. When you install the software the `lib` will be installed in the site-packages of the python interpreter and you are done, you can put `app.py` in a standard `bin` directory and it will always work independently of where it is. During development you simply flag the root directory as sources root and you are done, or you use a virtualenv where you do a [`pip install -e`](https://stackoverflow.com/questions/30306099/pip-install-editable-vs-python-setup-py-develop) – Bakuriu Apr 04 '19 at 18:32
  • @Bakuriu Sometimes yes, but sometimes you have a big project `project123/` and you have code in multiple subfolders of `project123/`. You don't necessarily want the libs to be elsewhere. There should be a way to do the import, in a simple way, like in nearly *all other languages* à la `#include "../x.h"`. – Basj Apr 04 '19 at 19:04
  • @Basj There is no need for such a thing. In python the main concept is the package. Everything inside the package can import anything else using relative imports. If you want to access to something outside that you access it via a plain `import` and it's whoever runs the code that should define the environment correctly. If you want to always use relative imports the solution is simple: just put everything into a big main package. I don't really see any value in this and Python is simply not designed for that. If you buy a bike then decide to remove the saddle and sit on it it's your choice. – Bakuriu Apr 04 '19 at 20:25

2 Answers2

2

Methods to do this

Method #1: Using the sys module: You can easily accomplish what you are trying to do using the sys module. To import the lib package, you can use one of the two codes listed below:

import sys
sys.path.append('<PATH_TO_LIB_FOLDER>')

from lib.abcd import lib

or

import sys
sys.path.insert(0, '<PATH_TO_LIB_FOLDER>')

Method #2: Using the os module: Another method is to use the os module. Here is an example code that imports the lib module using the os module by invoking the os.path.join method:

import os

path = os.path.join("<PATH>/lib/abcd", "lib")
from lib.abcd import lib

Method #3: Add the module to your PYTHONPATH: This is not the best method in most cases, but if you don't want to keep using the sys or os module to import lib, this is ideal. All you have to do is type this in your bash terminal:

export PYTHONPATH=<PATH_TO_LIB> python lib.py

Then in your python shell you can import it like this:

from lib.abcd import lib

Method #4: Combine the sys and os module (recommended): This is the most efficient method and will save you a lot of time. This code combines the os and sys module like this:

import sys, os
sys.path.append(os.path.abspath(os.path.join('..', 'lib')))

Then you can import your module with ease like this:

from lib.abcd import lib

How all the codes work:

All the codes above are very simple. All the examples except for "Method #3", add your module to the PYTHONPATH temporarily. "Method #3" on the other hand, adds the module to your PYTHONPATH permanently.

Community
  • 1
  • 1
xilpex
  • 3,097
  • 2
  • 14
  • 45
  • What new information did you add? These are both path manipulation, which I already mentioned, and is not what the OP is asking for. – Salvatore Apr 05 '19 at 18:08
  • Method #2 does not add anything to your `PYTHONPATH`. All that does is create the string `path` which is, e.g. on *nix, `"/lib/abcd/lib"`. And for method #3, you probably shouldn't overwrite `PYTHONPATH` but rather prepend to it. – Nathan Apr 09 '19 at 04:56
  • Method 2 is completely unrelated to the question (at least the *os* part). – CristiFati Apr 09 '19 at 18:29
  • @CristiFati -- Method 2 is just a way to make it work – xilpex Apr 09 '19 at 18:32
  • Maybe, but `os.path.join` is completely useless there, therefore *os* and then the method itself.. – CristiFati Apr 09 '19 at 19:43
  • These all add to the path at runtime, and your answer #3 assumes a bash environment configured in a very specific way which isn't part of the OPs question. – Salvatore Apr 11 '19 at 18:25
1

Surface level look

Normally, you can't. When importing a file, Python only searches the current directory, the directory that the entry-point script is running from, and sys.path which includes locations such as the package installation directory (it's actually a little more complex than this, but this covers most cases).

The reason you don't see this problem for importing installed modules is because they are installed in a location that is already on your path, or the location is added to the path by the installing utility (pip for example).

You can add to the Python path at runtime:

    import sys
    sys.path.insert(0, '../lib')

    import file

You can also use sys.path.append('../lib'), but then it will be searched for last in your path and may be overridden by previous path entries.

I have read through the import documentation extensively, and as far as I can tell, the answer to your question is no, there is no way to do this in a purely "Pythonic" way.

More in depth look:

Looking deeper into the import documentation explains this:

The import statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope. The search operation of the import statement is defined as a call to the __import__() function, with the appropriate arguments. The return value of __import__() is used to perform the name binding operation of the import statement.

Looking more closely at __import__:

__import__(name, globals=None, locals=None, fromlist=(), level=0)

Note: This is an advanced function that is not needed in everyday Python programming, unlike importlib.import_module().

This function is invoked by the import statement. It can be replaced (by importing the builtins module and assigning to builtins.__import__) in order to change semantics of the import statement, but doing so is strongly discouraged as it is usually simpler to use import hooks (see PEP 302) to attain the same goals and does not cause issues with code which assumes the default import implementation is in use. Direct use of __import__() is also discouraged in favor of importlib.import_module().

level specifies whether to use absolute or relative imports. 0 (the default) means only perform absolute imports. Positive values for level indicate the number of parent directories to search relative to the directory of the module calling __import__() (see PEP 328 for the details).

This makes me think that you can specify a level in some way that may make the import automatically look on the parent path. I'm guessing that when import is called from your app.py that is not in it's own directory, level is set to 0 and it searches the same level and deeper.

When you call import in app.py from a subfolder, level is still set to 0 and thus can't find the other directories above it. I am investigating a "Pythonic" way of setting this level to 1 when running your script which would appear to fix this problem.

Salvatore
  • 10,815
  • 4
  • 31
  • 69