Python3 out of source build

Question

What I want

Put all build output generated (build, dist, my_package.egg_info, etc) by python setup.py into a specific directory.

Project structure

awesome_pkg/
    __init__.py
    mod.py
my_fancy_app/
   __init__.py
   __main__.py
target/
requirements.txt
setup.py

__main__.py:

def main():
    print("Hello virtual environment")

if __name__ == '__main__':
    main()

setup.py:

import setuptools

setuptools.setup(
        name = "my_fancy_app",
        version="0.0.1",
        author="Me Myself",
        author_email="me@my.self",
        description="Some description",
        long_description="Some long description",
        packages=setuptools.find_packages(),
        entry_points={
            'console_scripts': [ 'hlpth = my_fancy_app.__main__:main' ]
        },
        classifiers=[
            "Programming Language :: Python :: 3.7",
            "License :: OSI Approved :: MIT License",
            "Operating System :: OS Independent",
        ],
        python_requires='>=3.7'
)

__init__.py is an empty script

What I tried

I tried to simply run setup.py with a specific working directory:

cd target
python3.7 ../setup.py bdist_wheel

The problem

The problem is it does not really package anything:

$ python3.7 ../setup.py bdist_wheel 
running bdist_wheel                                                                                                                                                                                                                    
running build                                                                                                                                                                                                                          
installing to build/bdist.linux-x86_64/wheel                                                                                                                                                                                           
running install                                                                                                                                                                                                                        
running install_egg_info                                                                                                                                                                                                               
running egg_info                                                                                                                                                                                                                       
creating my_fancy_app.egg-info                                                                                                                                                                                                         
writing my_fancy_app.egg-info/PKG-INFO                                                                                                                                                                                                 
writing dependency_links to my_fancy_app.egg-info/dependency_links.txt                                                                                                                                                                 
writing entry points to my_fancy_app.egg-info/entry_points.txt                                                                                                                                                                         
writing top-level names to my_fancy_app.egg-info/top_level.txt                                                                                                                                                                         
writing manifest file 'my_fancy_app.egg-info/SOURCES.txt'                                                                                                                                                                              
reading manifest file 'my_fancy_app.egg-info/SOURCES.txt'                                                                                                                                                                              
writing manifest file 'my_fancy_app.egg-info/SOURCES.txt'                                                                                                                                                                              
Copying my_fancy_app.egg-info to build/bdist.linux-x86_64/wheel/my_fancy_app-0.0.1-py3.7.egg-info                                                                                                                                      
running install_scripts                                                                                                                                                                                                                
creating build/bdist.linux-x86_64/wheel/my_fancy_app-0.0.1.dist-info/WHEEL                                                                                                                                                             
creating 'dist/my_fancy_app-0.0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it                                                                                                                                  
adding 'my_fancy_app-0.0.1.dist-info/METADATA'                                                                                                                                                                                         
adding 'my_fancy_app-0.0.1.dist-info/WHEEL'                                                                                                                                                                                            
adding 'my_fancy_app-0.0.1.dist-info/entry_points.txt'                                                                                                                                                                                 
adding 'my_fancy_app-0.0.1.dist-info/top_level.txt'                                                                                                                                                                                    
adding 'my_fancy_app-0.0.1.dist-info/RECORD'                                                                                                                                                                                           
removing build/bdist.linux-x86_64/wheel

But running it from the source root directory works perfectly fine python3.7 setup.py bdist_wheel and both of the packages are bundled into the wheel archive.

Is there a way to do such out of source build?

match · Answer 1 · 2019-11-28T13:10:32.077

2

You can specify the output directory for each build type as follows:

python setup.py sdist -d target/ bdist -d target/ bdist_wheel -d target/

To get rid of any temporary build etc dirs/files created during the build, run the following afterwards:

python setup.py clean --all

Note, it is generally safest to run setup.py from the directory it is in. Internally it may have relative paths referencing things like src, libs etc, and this will break if you are in a different location while running.

edited Nov 28 '19 at 13:10

answered Nov 28 '19 at 11:17

match

10,388
3
23
41

Yes it works, but not exactly the way I expected. Precisely I ran `python3.7 setup.py bdist_wheel -d target/`, but `build/` and `my_fancy_app.egg_info/` were still created at source root (`dist/` was not emitted anymore). The `target/` contains only the `wheel` archive. – St.Antario Nov 28 '19 at 11:25
`bdist_wheel` calls the other types implicitly. If you need them as well, then add them to the list, with their own `-d`. I've updated the answer to reflect this. – match Nov 28 '19 at 11:35
The `build/` and `my_fancy_app.egg_info/` still get created if building this way. I ran `python3.7 setup.py sdist -d target/ bdist -d target/ bdist_wheel -d target/`, the `target/` dir now contains 2 `.tar.gz` archives along with the `whl` one. – St.Antario Nov 28 '19 at 11:49
Added some more info about `clean` – match Nov 28 '19 at 13:10

score 2 · Accepted Answer · answered Nov 28 '19 at 16:08

2

You will have to play around until you find the right combination of setuptools commands and options for your exact use case. For example I have had some success with the following combination for a bdist_wheel:

python3 setup.py egg_info --egg-base target build --build-base target/build bdist_wheel --dist-dir target/dist

answered Nov 28 '19 at 16:08

sinoroc

18,409
2
39
70

Works exactly the way I wanted. Thanks. But the question is that out-of-source build is a pretty common task. Why does it require relatively complicated solution like this? – St.Antario Nov 29 '19 at 03:26
The authors and maintainers of _setuptools_ probably didn't consider this use case. Personally, I have never needed this either. Maybe some other Python packaging tools offer an easier way to do this. – sinoroc Nov 29 '19 at 07:12

Python3 out of source build

2 Answers2