2

I would like to create a source distribution package that automatically contains only my Subversion versioned files and then installs those same files.

If I use a setuptools.setup() call in setup.py that does not contain a setting for packages or include_package_data and run python setup.py sdist build, I get a package that contains only my versioned files, which is exactly what I need. However, when I attempt to install this package, only empty directories are created. None of the files are installed.

If I include packages = setuptools.find_packages() and run python setup.py sdist build, I get a package that contains all the files in my source tree instead of only the versioned files. When I install this package, all the files in the package are installed.

The only other parameters I use in setup() are for metadata (name, version, etc).

How do I create a package that contains only the versioned files in my directory tree and then installs those same files?

Background

I have verified that the package created without settings for packages and include_package_data contains the correct files (only versioned files are included).

I think that the basic problem is that the logic that selects files for inclusion in the package is disconnected from the logic that determines which files to install from the package. Thus, the two sets of files (include and install) can be completely different and the install set is not necessarily even a subset of the include set. That is, it's quite possible to create packages containing files that will not get installed and to create packages which will attempt to install files that are not even in the package.

The include set logic runs at build time and the install set logic runs at install time. Factors that determine which files are in the sets include MANIFEST, MANIFEST.in, packages, py_modules, data_files, include_package_data, the files in the source folder tree, the files in the package and versioning metadata and, apparently, attempting to manipulate the install set may or may not also modify the include set.

So, if I don't specify any of MANIFEST, MANIFEST.in, packages, py_modules, data_files and include_package_data, setuptools creates an include set that is based exclusively on finding versioned files in the source folder tree. But this leaves the install set empty. Then, if I include a packages parameter, setuptools breaks versioned file selection by adding unversioned files to the include set and uses all of these in the install set.

In other words, I'm trying to get the install set to exactly match the include set, without causing setuptools to add unversioned files to the include set.

Roger Dahl
  • 15,132
  • 8
  • 62
  • 82
  • "However, when I attempt to install this package, only empty directories are created. None of the files are installed." What's inside the created tarball file? I have never encountered such an issue. However you can control which files are included and which not explicitly using MANIFEST.in https://packaging.python.org/en/latest/distributing.html?highlight=manifest#id18 – Mikko Ohtamaa Nov 01 '14 at 19:22
  • Also depending on the setuptools version it might not understand SVN, Git, or other version control systems. You might need to install additional packages if you want to make setuptools to know about versioned files. Also Subversion version compatibility may affect. Please see note here http://opensourcehacker.com/2012/08/14/high-quality-automated-package-releases-for-python-with-zest-releaser/#Handling_ignored_and_otherwise_special_files_%28i18n%29 – Mikko Ohtamaa Nov 01 '14 at 19:23
  • 1
    I think to narrow down this problem one can start checking the created package and find a clue if it's broken on creation or if it becomes broken on installation. Reading *.egg-info files should help here. – Mikko Ohtamaa Nov 01 '14 at 19:24
  • Could you show the setup.py file - I am not too familiar with setup.py so I would need more information. – User Nov 02 '14 at 06:12

1 Answers1

0

I found a way to make setuptools support this. I simply supply the packages list only when the package is installed and leave it as None when the package is built. I also use include_package_data = True to get data files to be installed (without it, the data files get included in the package but not installed).

if 'install' in sys.argv:
  packages = setuptools.find_packages()
else:
  packages = None

setuptools.setup(
  packages = packages,
  include_package_data = True,
  ...
)
Roger Dahl
  • 15,132
  • 8
  • 62
  • 82