5

I am using py2exe to convert a script which uses numpy and am getting a very large resulting folder, and it seems a lot of the large files are from parts of the numpy package that I'm not using, such as numpy.linalg.

To reduce the size of folder that is created, I have been led to believe I should have numpy compiled without Intel MKL/BLAS/ATLAS/LAPACK.

How would I make this change?

EDIT
In C:\Python27\Lib\site-packages\numpy\linalg I found the following files: _umath_linalg.pyd (34MB) and lapack_lite.pyd (18MB) which are being copied into the distribution folder when using py2exe. If possible I would like to remove dependence on these while still being able to use numpy arrays. The other large file being included is in C:\Python27\Lib\site-packages\numpy\core and is called _dotblas.pyd (12MB). Is it possible to remove this too?

ali_m
  • 71,714
  • 23
  • 223
  • 298
Siwel
  • 705
  • 10
  • 25

1 Answers1

9

According to the official documentation:

Disabling ATLAS and other accelerated libraries

Usage of ATLAS and other accelerated libraries in Numpy can be disabled via:

  BLAS=None LAPACK=None ATLAS=None python setup.py build

However, this information seems to be out of date, since I found that even with these options numpy v1.9.2 was still automatically finding libopenblas.so:

numpy_source_dir/$ BLAS=None LAPACK=None ATLAS=None python setup.py config
...
openblas_info:
  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['/opt/OpenBLAS/lib']
    language = f77
...

One workaround is to copy site.cfg.example to site.cfg, then edit it to make the paths to the relevant BLAS/LAPACK libraries invalid:

[openblas]
libraries =
library_dirs =
include_dirs =

When you subsequently call BLAS=None LAPACK=None ATLAS=None python setup.py config you should get an output containing this:

...
openblas_info:
/home/alistair/src/python/numpy/numpy/distutils/system_info.py:594: UserWarning: Specified path  is invalid.
  warnings.warn('Specified path %s is invalid.' % d)
  libraries  not found in []
  NOT AVAILABLE
...

I expect that the same approach will work for ATLAS and MKL, although I don't have these libraries installed in order to do a proper test.

You should, of course, be aware that not having accelerated BLAS/LAPACK libraries will have a big detrimental effect on performance for linear algebra ops.

Update

As mentioned in the comments below, you didn't actually "compile" your current version of numpy, but rather installed it from a binary distribution. The approach I gave above would require you to build numpy from source, which is not an easy thing to do in Windows (although there are official instructions here).

A much easier option would be to install one of the unoptimized numpy binaries available from Christoph Gohlke's website here.

Community
  • 1
  • 1
ali_m
  • 71,714
  • 23
  • 223
  • 298
  • I'm sorry, I have seen the documentation but I don't even understand what it means when it says it can be disabled via `BLAS=None LAPACK=None ATLAS=None python setup.py build` - I thought that meant typing it in the command line but that didn't work. Also, where do I find `site.cfg.example`? I don't use any of the linear algebra module at all, just arrays, which I presume won't be affected (at least much?) – Siwel Aug 26 '15 at 21:05
  • `setup.py` and `site.cfg.example` are files inside the numpy source directory. What platform are you using? How are you building numpy at the moment? – ali_m Aug 26 '15 at 21:09
  • I use Windows 10 and currently have `numpy` installed, which I did quite a long time ago. I take it I need to uninstall before reinstalling with the new `site.cfg` file? Or is there any way I could install it under a different name with the lack of LAPACK library? – Siwel Aug 26 '15 at 21:14
  • Yes, you would. More importantly: *how* did you install numpy originally? Given that you're using Windows, I would bet that you probably used a `.exe` installer. However, in order to rebuild numpy without BLAS/LAPACK (as shown in my answer), you would need to be able to compile numpy from source. This is *not* an easy thing to do in Windows (although there are official instructions [here](http://www.scipy.org/scipylib/building/windows.html)). – ali_m Aug 26 '15 at 21:32
  • 1
    Your best bet is probably to reinstall numpy from one of the `unoptimized` binaries provided by Christoph Gohlke [here](http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy). – ali_m Aug 26 '15 at 21:33
  • I think I used one of the optimized binaries. If there is no easy way to install it under a different name I may actually settle for the larger file size until I have time to remove the `numpy` dependence, as I may want to use the linear algebra operations in the future. Thanks for your explanations. – Siwel Aug 26 '15 at 21:46
  • Are the `unoptimized` binaries gone? I can't seem to find them on the site you linked @ali_m. – Filip S. Oct 28 '18 at 18:38
  • To answer myself: yes. I found some records from 2015 of the unoptimized versions on the [Wayback Machine](https://web.archive.org/web/20150909140143/http://www.lfd.uci.edu:80/~gohlke/pythonlibs/), but they are gone from the current site. – Filip S. Oct 28 '18 at 19:09
  • Using `NPY_BLAS_ORDER= NPY_LAPACK_ORDER=` instead of `BLAS=None LAPACK=None ATLAS=None` worked for me to disable BLAS and LAPACK. https://docs.scipy.org/doc/numpy/user/building.html#disabling-atlas-and-other-accelerated-libraries – Vortico Oct 02 '19 at 00:51