make distutils in Python automatically find packages

Question

When describing a python package in setup.py in distutils in Python, is there a way to make it so automatically get every directory that has a __init__.py in it and include that as a subpackage?

ie if the structure is:

mypackage/__init__.py
mypackage/a/__init__.py
mypackage/b/__init__.py

I want to avoid doing:

packages = ['mypackage', 'mypackage.a', 'mypackage.b']

and instead just do:

packages = ['mypackage']

and have it automatically find things like a and b since they have an init file. thanks.

Just make sure that you don't let any junk accumulate in your package structure before building if you do this. — asmeurer, Oct 19 '12 at 05:00

score 113 · Answer 1 · answered Jan 17 '13 at 10:22

113

I would recommend using the find_packages() function available with setuptools such as:

from setuptools import setup, find_packages

and then do

packages=find_packages()

answered Jan 17 '13 at 10:22

dm76

4,130
8
35
46

6

Note that `find_packages()` doesn't find > 3.3-style no-__init__.py packages. The ticket about this issue is at https://bitbucket.org/pypa/setuptools/issue/97. So for now, it's necessary to add `__init__.py` files in packages for `find_packages()` auto collection, or you must name them explicitly in `packages`. – Iodnas Mar 21 '15 at 19:10
1

Update: Even the explicit list causes problems if `__init__.py` files are not used on Python 3.3 (`package init file 'mymod/mysubmod/__init__.py' not found (or not a regular file)`). – Iodnas Mar 21 '15 at 19:28
1

Now there are also `find_namespace_packages()` (see https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages) which works with PEP 420 namespace packages (https://www.python.org/dev/peps/pep-0420/) – ead Feb 26 '19 at 21:50

score 22 · Accepted Answer · answered Oct 19 '12 at 02:00

22

The easiest way (that I know of) is to use pkgutil.walk_packages to yield the packages:

from distutils.core import setup
from pkgutil import walk_packages

import mypackage

def find_packages(path=__path__, prefix=""):
    yield prefix
    prefix = prefix + "."
    for _, name, ispkg in walk_packages(path, prefix):
        if ispkg:
            yield name

setup(
    # ... snip ...
    packages = list(find_packages(mypackage.__path__, mypackage.__name__)),
    # ... snip ...
)

answered Oct 19 '12 at 02:00

Sean Vieira

155,703
32
311
293

1

There may (indeed, there most likely *are*) better ways to do this ... I just don't know them (yet) :-) – Sean Vieira Oct 19 '12 at 02:02
7

find_packages() from setuptools does exactly the same essentially, see my answer – dm76 Jan 17 '13 at 10:23
1

@dm76 yes but the OP is asking for distutils, not setuptools. Am I missing something? – Paolo Feb 08 '13 at 16:33
`__path__` return me `NameError`, I use `'.'` instead. – Zulu Sep 09 '13 at 13:06
Downvoter - can you explain so I can make the answer better? – Sean Vieira Aug 12 '16 at 23:30
I just think @dm76 answer is better, although is not disutils – guilhermecgs Sep 23 '16 at 21:41

score 19 · Answer 3 · answered May 10 '15 at 18:08

19

Same as dm76 answer, just that I also have tests in my repo, so I use:

from setuptools import find_packages

packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),

answered May 10 '15 at 18:08

Shubham Chaudhary

47,722
9
78
80

score 2 · Answer 4 · answered May 10 '13 at 23:51

import re, os
def find_packages(path='.'):
    ret = []
    for root, dirs, files in os.walk(path):
        if '__init__.py' in files:
            ret.append(re.sub('^[^A-z0-9_]+', '', root.replace('/', '.')))
    return ret

setup(
...
packages = find_packages()
...
)

make distutils in Python automatically find packages

4 Answers4

Linked