How to structure a python project with three applications that use common module

Question

My project contains three Python applications. Application 1 is a web app. Applications 2 and 3 contain scripts downloading some data.

All three apps need to use a module Common containing a "model" (classes that are saved to database) and common settings.

I have no clue how to structure this project. I could create three directories, one for each application, and copy Common three times into their directories (doesn't seem right).

Another idea that comes to mind is; create a main directory and put there all files from Common, including __init__.py. Then, crete three subdirectories (submodules), one for each application.

Another way would be installing Common using pip, but that means I would have to reinstall every time I change something in that module.

In previous projects I used .NET - the equivalent in that world would be a Solution with four projects, one of them being Common.

Any suggestions?

Answer 1

I have a similar project that is set up like this

project_root/
    App1/
        __init__.py
    FlaskControlPanel/
        app.py
        static/
        templates/
    models/
        __init__.py
        mymodels.py

Then, I run everything from project_root. I have a small script (either batch or shell depending on my environment) that sets PYTHONPATH=. so that imports work correctly. This is done because I usually develop using PyCharm, where the imports "just work", but when I deploy the final product the path doesn't match what it did in my IDE.

Once the PYTHONPATH is set to include everything from your project root, you can do standard imports.

For example, from my FlaskControlPanel app.py, I have this line:

from models.mymodels import Model1, Model2, Model3

From the App1 __init__.py I have the exact same import statement:

from models.mymodels import Model1, Model2, Model3

I can start the Flask application by running this from my command line (in Windows) while I am in the project_root directory:

setlocal
SET PYTHONPATH=.
python FlaskControlPanel\app.py

The setlocal is used to ensure the PYTHONPATH is only modified for this session.

Answer 2

I like this approach

projects/
  __init__.py
  project1/
    __init__.py
  project2/
    __init__.py
  lib1/
    __init__.py
    libfile.py
  lib2/
    __init__.py

So, I need to cd into the projects folder. To start a projects use

python -m project_name

This allows me to easily import from any external lib like

from lib1.libfile import [imoprt what you want]

or

from lib1 import libfile

Answer 3

Make standard Python modules from your apps. I recommend structure like this:

apps/
    common/
        setup.py
        common/
            __init__.py
            models.py
    app1/
        setup.py
        app1/
            __init__.py
            models.py
project/
requirements.txt

Basic setup.py for app common:

#!/usr/bin/env python

from setuptools import setup, find_packages

setup(
    name='common',
    version='1.0.0',
    packages=find_packages(),
    zip_safe=False,
)

Make similar setup.py for other apps.

Set editable "-e" option for your apps in requirements.txt:

-e apps/common
-e apps/app1

Install requirements with pip:

$ pip install -r requirements.txt

Editable option means that source files will be linked into Python enviroment. Any change in source files of your apps will have immediate effect without reinstalling them.

Now you can import models from your common app (or any other app) anywhere (in other apps, project files, ...).

Answer 4

I would create a structure like this:

project_root/
    app1/
        __init__.py
        script.py
    common/
        __init__.py
        models.py (all "common" models)

app1/script.py

import os, sys

# add parent directory to pythonpath
basepath = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')
if basepath not in sys.path:
    sys.path.append(basepath)

from common.models VeryCommonModel
print VeryCommonModel

If you don't want to set the python path at runtime, set the python path before running the script:

$ export PYTHONPATH=$PYTHONPATH:/path/to/project_root

And then you can do:

python app1/script.py