ImportError while doing python setup.py test

Question

I'm making a simple python app. I don't know if I am doing it correctly or not so please correct me in the comments or if you have an answer for this

Error:

ImportError: No module named 'taskhandler'

and:

ImportError: No module named 'styles' while doing `python3 setup.py test

File Structure:

.
├── MANIFEST.in
├── pydotask.egg-info
│   ├── dependency_links.txt
│   ├── not-zip-safe
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   └── top_level.txt
├── README.md
├── setup.py
├── task_mod
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-35.pyc
│   │   ├── pydo.cpython-35.pyc
│   │   └── taskhandler.cpython-35.pyc
│   ├── pydo.py
│   ├── styles
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-35.pyc
│   │   │   ├── termcolor.cpython-35.pyc
│   │   │   └── text_style.cpython-35.pyc
│   │   ├── termcolor.py
│   │   └── text_style.py
│   ├── taskhandler.py
│   └── tasks.csv
└── update.txt

5 directories, 22 files

'task_mod/pydo.py':

#!/usr/bin/env python3

''' To Do App in Python '''

import sys, os
import taskhandler as task
from styles import text_style as text
from styles import termcolor

task_mod/taskhandler.py:

#!/usr/bin/env python3

import sys, os
import csv
from styles import termcolor
from styles import text_style as text

setup.py

from setuptools import setup

def readme():
    with open('README.md') as readme:
        return readme.read()

setup(
        name = 'pydotask',
        version = '0.2',
        description = 'PyDo is a CLI Application to keep you on track with your tasks and projects',
        long_description = readme(),
        classifiers = [
            'Development Status :: 3 - Alpha',
            'Programming Language :: Python :: 3.5',
            'Topic :: Office/Business :: Scheduling'
            ],
        keywords = 'utilities office schedule task reminder',
        url = '',
        author = 'Abhishta Gatya',
        author_email = 'abhishtagatya@yahoo.com',
        packages = ['task_mod'],
        scripts = ['task_mod/pydo'],
        python_requires = '>=3',
        include_package_data = True,
        zip_safe = False
        )

How do I get around this problem?

Note: If I run python3 task_mod/pydo.py it works fine! But when I try testing it it, it gives 2 ImportErrors.

Answer 1

You have to specify all modules used in setup.py, not just the top-level folder. So in your setup.py file, replace the line packages = ['task_mod'], with packages = ['task_mod', 'task_mod.styles', 'task_mod.taskhandler'],.

Alternatively, without changing your setup.py, you can import with import task_mod.styles or with from task_mod import styles. Then you can use files in styles like styles.termcolor.

Or, you can use setuptool's black magic function find_packages like so: packages = find_packages(),

Related SO post

Answer 2

First, note that you provide a package named task_mod in setup.py. It means, you should only import task_mod or import task_mod.blah, never import blah. Because you do not provide blah in your library. Trying changing the imports to the absolute ones.

Second, if you still want a relative import (this is a common practice to do withing a single library — it is easier to maintain the code then), you should import relatively: from .styles import termcolor (note the dot).

Third, the relative imports only affect the modules & packages, not the scripts that you execute direcly (because pydo.py is the package __main__, not task_mod.pydo, and it changes everything). For the scripts, you have two choices to do it properly:

Choice A (so-so): Always import absolute package/module name (import task_mod.taskhandler as tashhandler in your pydo.py; also from task_mod.styles import termcolor, etc).

Choice B (best practice): Never export any scripts as part of the library (only for building/testing/CI/CD purposes). Instead, export the entry points for the console scripts (google: setuptools entry points).

setup( .... entry_points={ 'console_scripts': [ 'pydo = task_mod.pydo:main', ], }, )

And, of course, define the main() function in that module.

The shebangs (#!...python3) are completely irrelevant here.

---

UPD:

If you have problems with importing, please keep this in mind:

You should make your package to be on the PYTHONPATH env var (or sys.path internal variable), one way or another. This is exactly where Python looks for things when you do import things.

You can print(sys.path) before any imports to see why it happens this way — the first element will be your script's dir, and it will be varying between task_mod/pydo.py & setup.py calls.

When you run a script in the main project dir (python3 setup.py), sys.path starts with the project's dir. task_mod is there, you can import it. Also, when you do something like pip install -e ., the packages will also be "installed" into the Python/virtualenv list of libraries, but in a different way.

However, when you run python3 task_mod/pydo.py, you current dir is .../task_mod/. There, you cannot find task_mod package (because it is one level upper).

On the opposite side, when your run python3 setup.py, your current dir is the project's dir, and you cannot import styles & taskhandler directly as you did innitially (without prefixing them by task_mod.).

This is exactly the reason why you should never rely on the sys.path, and never use the scripts directly, and assume its location. Because it varies.

You can try running PYTHONPATH=. python3 task_mod/pydo.py to make it work the same as for setup.py (or PYTHONPATH=./task_mod/ python3 setup.py to make it the same as for task_mod/pydo.py). But this is a dirty hack to make it work. Instead, you should layout your library properly, according to Python's conventions.