9

I'm trying to use nix-shell as a #! wrapper for runghc, as suggested in the manpage. But it cannot find the libraries. Given the following example, cut-down from the manpage

#! /usr/bin/env nix-shell
#! nix-shell -i runghc -p haskellPackages.HTTP

import Network.HTTP

main = return ()

I get the following result:

[~:0]$ ./nixshelltest 

nixshelltest:4:1: error:
    Failed to load interface for ‘Network.HTTP’
    Use -v to see a list of the files searched for.
[~:1]$

to my mind, that's exactly what nix-shell -p is to avoid.

Am I doing something wrong, missing the point, or is there a bug? This is both on a nixOS 17.03 host, and also a host running nix 17.09 on top of Ubuntu.

Thanks,

user3416536
  • 1,429
  • 9
  • 20

1 Answers1

9

The environment that you're using to run the script is missing a step. It's got a GHC and a HTTP package, but the compiler doesn't know about the package.

The way GHC and library packages work in nix might be a little "inside out" from what you're expecting. You actually need to install a compiler that "depends on" all of the libraries you want, rather than simply installing the compiler and the library separately. The reason is that GHC is designed to have library packages added by modifying the file tree where GHC is installed. On a mutable file system with only a single system GHC install you would just modify GHC whenever a library was installed, but nix can't. Nix has to be able to install a frozen GHC that never changes, and potentially many of them.

So what happens instead is that you install a tiny wrapper which depends on the both the underlying "raw" GHC install and all of the libraries that you want to use. The wrapper then acts like an install of GHC that had those libraries registered, without actually needing to duplicate an entire GHC install. When you just install a library package on its own it just sits there inert, without any GHC being able to find it just by it existing.

In fact, the script you've shown here doesn't actually specify that it should have a compiler installed at all; it just asks for the HTTP library. When I tried your script I got command not found: runghc. The runghc is only working on your system because it happened to already be in your path when you ran this (perhaps because you have GHC installed in your profile?), and that GHC wasn't installed with the HTTP package and so can't see it. The nix-shell adding just the library to the environment doesn't help.

What you need to do instead is use this line:

#! nix-shell -i runghc -p "ghc.withPackages (ps: [ ps.HTTP ])"

You're not installing either ghc or HTTP directly; instead the ghc.withPackages function computes a nix package that installs a GHC wrapper that knows about the HTTP Haskell package. Under the hood this depends on a "raw" GHC with no additional libraries, and also on the HTTP library and will cause those to be installed too.

If you use lots of different Haskell environments (possibly via nix-shell scripts like this that each need a different set of libraries), then you will end up with a unique withPackages wrapper installed on your system for each combination of libraries you ever use. But that's okay because the wrappers themselves are tiny, and nix is able to share and reuse the underlying GHCs and library packages between all of those environments.

Ben
  • 68,572
  • 20
  • 126
  • 174
  • 2
    Wow, that's a great explanation, Ben. And indeed, it works perfectly. Once I'm qualified, I'll suggest an enhancement to the manpage for nix-shell, but in the meantime, many thanks for taking the time to explain what and why. You've both solved my immediate problem and empowered me to tackle the next one with a much deeper understanding. – user3416536 Jul 11 '17 at 05:32
  • 1
    @user3416536 Possibly, that would be https://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure that would need some improvement? I personally feel like the section "How to install a compiler with libraries" isn't highlighted enough. – Zimm i48 Jul 11 '17 at 12:35
  • @user3416536 Unfortunately the nix-shell man page isn't really enough to use it with everything. This particular point isn't really a consequence of how nix-shell works (or even how nix works), but how GHC works, and how the specific nix code used to support GHC environments works. There are other languages that work this way in nix (python is one), but also others that don't. It's not really appropriate to describe this specifically in the nix-shell man page, but that means it's not clear why your script didn't work unless you know there's a specific Haskell section of the nixpkgs manual. :( – Ben Jul 11 '17 at 23:45
  • @user3416536 In general with nix it seems that "end user programs" are pretty fire and forget, but for fairly technical things with lots of independently installable parts (like programming language libraries) you often have to know quite a bit about how the nix code describing them works in order to work with them on a nix system. – Ben Jul 11 '17 at 23:50
  • Your point about this being perhaps too deep for the nix-shell manpage is fair, but being as it does have an example for haskell "scripts", I think that the example should work! I argue that the example should be corrected so it does; or else removed (if we think that correcting it would make it too involved for the manpage). – user3416536 Jul 12 '17 at 06:04
  • @user3416536 Oh, definitely! I didn't actually realise there was a concrete Haskell example in the nix-shell man page; that would be a documentation bug if it doesn't work. – Ben Jul 12 '17 at 08:01
  • Seems like this doesn't work anymore, switching from single to double quotes above fixed it for me. – C. Hammill Aug 14 '18 at 23:19
  • @C.Hammill Thanks for noticing that, how strange! I've edited my answer so the example works. I can't remember now whether I would have tested the single-quotes version before posting this... probably though, since I don't really use the second shebang line nix-shell feature. – Ben Aug 14 '18 at 23:56