7

Documented Conda "best practices" is still to give conda-forge channel priority over defaults channel in environment.yml files. Can I continue to give priority to conda-forge whilst still downloading any mkl optimized packages from the `defaults channel?

I have never encountered any problems giving defaults priority over conda-forge in my environment files so perhaps this suggested "best practice" is no longer addressing a real issue.

Still, it would be good to know if there was a way to specify mkl in an environment file where conda-forge has higher priority than defaults.

davidrpugh
  • 4,363
  • 5
  • 32
  • 46
  • 1
    there is actually mkl optimized blas in conda-forge and it is possible to install a virtual package for that: https://conda-forge.org/docs/maintainer/knowledge_base.html#switching-blas-implementation – cel Dec 23 '19 at 08:57
  • Can your provide and example `environment.yml` file showing how this approach works? – davidrpugh Dec 23 '19 at 11:03

1 Answers1

4

conda-forge provides their own mkl optimized blas libaries that can be installed via a virtual package (see also: https://conda-forge.org/docs/maintainer/knowledge_base.html#switching-blas-implementation).

environment.yaml

name: test_foo
channels:
  - conda-forge
  - defaults
dependencies:
  - "libblas=*=*mkl"
  - numpy
  - scipy

conda -n test_foo list

ca-certificates           2019.11.28           hecc5488_0    conda-forge
certifi                   2019.11.28               py38_0    conda-forge
intel-openmp              2019.4                      233
libblas                   3.8.0                    14_mkl    conda-forge
libcblas                  3.8.0                    14_mkl    conda-forge
libcxx                    9.0.0                h89e68fa_1    conda-forge
libffi                    3.2.1             h6de7cb9_1006    conda-forge
libgfortran               4.0.0                         2    conda-forge
liblapack                 3.8.0                    14_mkl    conda-forge
llvm-openmp               9.0.0                h40edb58_0    conda-forge
mkl                       2019.4                      233
ncurses                   6.1               h0a44026_1002    conda-forge
numpy                     1.17.3           py38hde6bac1_0    conda-forge
openssl                   1.1.1d               h0b31af3_0    conda-forge
pip                       19.3.1                   py38_0    conda-forge
python                    3.8.0                hd366da7_5    conda-forge
readline                  8.0                  hcfe32e1_0    conda-forge
scipy                     1.4.0            py38h82752d6_0    conda-forge
setuptools                42.0.2                   py38_0    conda-forge
sqlite                    3.30.1               h93121df_0    conda-forge
tk                        8.6.10               hbbe82c9_0    conda-forge
wheel                     0.33.6                   py38_0    conda-forge
xz                        5.2.4             h1de35cc_1001    conda-forge
zlib                      1.2.11            h0b31af3_1006    conda-forge

Note that only the mkl implementation of blas is installed, but not openblas.

cel
  • 30,017
  • 18
  • 97
  • 117
  • I notice that when giving `defaults` priority over `conda-forge` a number of additional `mkl` related packages are also installed: `mkl-service`, `mkl_fft`, `mkl_random`, etc. Looks like these packages are available on `conda-forge` and need to be listed as separate dependencies in order to install. – davidrpugh Dec 23 '19 at 11:58
  • I built an environment using your provided file, activated the environment, started python interpreter and imported `numpy` and `scipy` and then check `numpy.show_config()` and `scipy.show_config()` which shows information about which linear algebra libs are used. These config files look significantly different between your solution and a solution that includes `defaults` as higher priority than `conda-forge`. Any ideas why? – davidrpugh Dec 24 '19 at 07:48
  • @davidrpugh, I am unfortunately not an expert in the technicalities of mkl optimized blas libraries at conda-forge. You might be able to get an answer by asking in the conda-forge gitter channel: https://gitter.im/conda-forge/conda-forge.github.io – cel Dec 24 '19 at 09:35
  • I am going to accept your answer and then start another post to try and answer the question raised in my previous comment. – davidrpugh Dec 29 '19 at 06:00