What is a reliable way to include libraries for virtualenv python packages in nix?

Question

I'll start by saying virtualenv is basically a requirement here since Nix is not yet being used by the rest of the development team. This excellent guide on Python in Nix doesn't quite drill down to this particular issue.

In some cases I can update LD_LIBRARY_PATH, but it gets to be rather tedious and potentially error prone due to the dynamic nature of Python (a particular branch could trigger the use of a library not previously included in LD_LIBRARY_PATH):

  shellHook = ''
    export LD_LIBRARY_PATH=${mysql57}/lib:${gcc6}/lib:$LD_LIBRARY_PATH
  '';

Worse, the ${ggc6}/lib doesn't work for me here, since the library I need (libatomic.so) is under the *-gcc-6.4.0-lib/lib directory, not the *-gcc-6.4.0/lib directory, and I'm not sure how to reference the former.

$ echo $LD_LIBRARY_PATH                                                                                                                                                                                                                                                                                                                                                                                                                                              
/nix/store/x3x3si0pc3w0vam9jj308b0qhcv7zlg2-mysql-5.7.19/lib:/nix/store/mc8p626zjk9zlgji1i8f85nax4c62nrj-gcc-wrapper-6.4.0/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64

Some output from find for libatomic:

/nix/store/rww78vdn2rkayrnqsjl8ib5iq2vfm3sw-gcc-6.4.0/lib/libatomic.a                                                                                                                                                                                                                                                                                                                                                                                                                                    
/nix/store/klqzvvcy1xyjjflmf7agryayc4s72jg2-gcc-6.4.0-lib/lib/libatomic.so.1                                                                                                                                                                                                                                                                                                                                                                                                                             
/nix/store/klqzvvcy1xyjjflmf7agryayc4s72jg2-gcc-6.4.0-lib/lib/libatomic.so                                                                                                                                                                                                                                                                                                                                                                                                                               
/nix/store/klqzvvcy1xyjjflmf7agryayc4s72jg2-gcc-6.4.0-lib/lib/libatomic.la                                                                                                                                                                                                                                                                                                                                                                                                                               
/nix/store/klqzvvcy1xyjjflmf7agryayc4s72jg2-gcc-6.4.0-lib/lib/libatomic.so.1.2.0

score 0 · Answer 1 · answered Jul 03 '18 at 17:29

I don't use the nixpkgs Python infrastructure much, so I'm not sure if there is a way to eliminate setting the LD_LIBRARY_PATH. Recommendations for setting it:

You can use lib.makeLibraryPath to make the process much less tedious. If you know that all the libraries that might be necessary are (mostly) in buildDepends anyway, you can use lib.makeLibraryPath (buildDepends ++ [ anything else ]).
The issue with GCC libraries has to do with the fact that Nix needs wrapped versions of GCC, so pkgs.gcc6 isn't the "raw" GCC derivation. You can use gcc6.cc.lib, or, if you are using makeLibraryPath, just gcc6.cc will be enough (as makeLibraryPath will automatically figure out that the lib output is the correct one to look under).

What is a reliable way to include libraries for virtualenv python packages in nix?

1 Answers1