16

I've always created scripts directories in every project I've built because they're useful for putting infrequently used executable scripts. In Python, I'll always put an __init__.py in my scripts directory so I can run scripts as packages (i.e. python -m scripts.some_scripts) and load modules from sister directories. Based off this as well as googling, I'm starting to get the feeling that this is an anti-pattern.

That said, given a structure like:

project_dir/
    some_modules_dir/
        foo.py
        bar.py
        ...
    scripts/
        some_script.py
        other_script.py
        ...

What's the right way to run scripts and what's the right way to have them import from their sister directory some_modules_dir? Which dirs should contain __init__.py and which shouldn't? I want to follow PEP8 as much as possible, and want to simplify running scripts as much as possible. If having a scripts directory at all is inherently inadvisable, what do you guys do instead?

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Eli
  • 36,793
  • 40
  • 144
  • 207
  • Is it possible that someone imports a script module and calls parts of it manually? – Kos Nov 03 '14 at 23:26
  • @Kos possible, but why? I just want to have some regular python scripts that do one-off tasks. Having people import parts elsewhere seems like overkill. – Eli Nov 04 '14 at 02:37

4 Answers4

6

The link in the question only says about running scripts that reside in a package directory which is a potential problem because... well... packages are not scripts and scripts are not packages. They serve different purposes and are invoked in different ways, so if you mix them together, there'll be a mess at some point.

Since Python itself has had a Scripts directory for ages and no one complains, it's in no way an anti-pattern.

See How do I separate my executable files from my library files? on how we dealt with executable scripts at my last occupation. It never caused any problems that I'm aware of.

Community
  • 1
  • 1
ivan_pozdeev
  • 33,874
  • 19
  • 107
  • 152
3

Personally, having a __init__.py in one's scripts dir feels a bit off, but I can also see why it's useful here (and in IDEs).

That said, if they're already being run as a Python module, then maybe they're not true scripts, whatever that even means (related: do you have a shebang on these files?). Hard to say without context, but perhaps they are closer to tools modules, forming part of your overall Python codebase.

In which case, with the addition of an __init__.py at project level (project_dir in your example), you can then use normal importing in your would-be scripts:

from some_modules_dir import foo

meaning that your original python -m scripts.some_script makes absolute (sorry) sense...

declension
  • 4,110
  • 22
  • 25
3

Setuptools can create scripts automatically if you give it a list of entry points (i.e. main() functions). If you're already using Setuptools, it's very easy to turn on. Then you can merge your scripts directory with your other packages.

Kevin
  • 28,963
  • 9
  • 62
  • 81
  • How is this even relevant? – ivan_pozdeev Nov 06 '14 at 21:06
  • 2
    @ivan: OP wants executable scripts. I don't understand your question. – Kevin Nov 06 '14 at 21:09
  • the question has nothing to do with the stock `Scripts` directory or wrapping scripts into exe's. – ivan_pozdeev Nov 06 '14 at 21:18
  • 1
    I don't agree. OP has a series of executable scripts and wants to make them available in a standard, well-supported fashion. That's Setuptools. – Kevin Nov 06 '14 at 21:21
  • I'm about 99% sure this is not canonical, and it definitely feels like overkill. There's got to be an easier way than hooking setup tools into every little script I make. If this is canonical and widely used by people in place of scripts directories, I'd like to see some proof. – Eli Nov 07 '14 at 18:53
  • 3
    @Eli: Setuptools itself is canonical. Once you have Setuptools, this particular feature is canonical. Sure, for simple things it's overkill, but in that case you don't want something canonical, you want something easy. – Kevin Nov 07 '14 at 18:56
1

In general, if your Python project contains multiple packages that are native to the project, then there should probably be a single top-level package that all other packages are subpackages of (after all they are part of the same project so there should be some unifying justification for their existence). Since a directory counts as a package only when it has an __init__.py, the other way to say this is that if any two sibling directories in your project both have an __init__.py, then their parent should have one as well. So assuming that you have good reasons to have a "scripts" package and a "modules" package directly within your project root, which you very well might have, your project root directory probably should be a package as well.

If your top-level package is not the project root, it is usually considered fine to have some "loose" Python scripts adjacent to the top-level package. Like this:

project_root/
    top_level_package/
        __init__.py
        module.py
        subpackage/
            __init__.py
            anothermodule.py
    adjacent_script.py
    adjacent_script_2.py

The "loose" scripts can import directly from the packages and modules because they are in the same directory as the top-level package. The assumption with this construction is that the top-level package contains all "interesting" code of your project (your "selling point", or the "meat" of your project if you want), while the "loose" adjacent scripts function only as entry points to specific functionality that you may want to access from the top-level package. For example, you may have an adjacent script for starting the test suite, for installing the software, or to start the application if your project is an application.

Concerning your particular pattern, I'm under the impression that you distinguish "scripts" from "modules" because the scripts are meant as an interface between you and the modules. If it is only for you as a developer and you are using the "scripts" infrequently, as you said, it is probably fine to stick with your recipe as long as the root directory has an __init__.py. If, however, you (or someone else) also use the "scripts" as an end user, or if you use them frequently, it is neater to provide a script adjacent to the top-level package that bundles the functionality from your "scripts" directory in a single command (or maybe a few if they offer very dissimilar functionality) with a subcommand for each "script". You might want to use the argparse standard module for that adjacent script. In either case, it would probably be more appropriate to call the "scripts" tools.

If you have only a few tools and they don't contain any code other than glue between the command line and your modules, you may also consider to just move them to the top directory.

Some sources:

Julian
  • 4,176
  • 19
  • 40