26

I have noticed in the past that in nix, a ./myfile.txt path type seems to

  • sometimes evaluate to /home/myuser/mydir/myfile.txt, and
  • sometimes to /nix/store/55j24v9qwdarikv7kd3lc0pvxdr9r2y8-myfile.txt.

I would like to understand exactly when which case happens.

This is especially important for files that contains any form of secret information, as all files in /nix/store are world-readable by all users on the system.

(When using nixops, there's a special "keys" feature for this purpose, see section Managing keys in the manual, but I think it's still important when and how this path-to-store-path copying happens in nix itself.)

toraritte
  • 6,300
  • 3
  • 46
  • 67
nh2
  • 24,526
  • 11
  • 79
  • 128
  • Relevant gist: [Nix String and Path Concatenation](https://gist.github.com/CMCDragonkai/de84aece83f8521d087416fa21e34df4) – toraritte Aug 13 '20 at 16:41
  • Reason for editing the title: This thread is more specifically about when will files referenced by relative paths get copied to the Nix store. For example, as I just learned, a bare relative path will also get copied to the Nix store if assigned to an environment variable in a Nix expression. For example, `pkgs.mkShell { ...; GIT_CONFIG_GLOBAL = ./assets/git.conf; ... }`. (With that said, I've been using this page for reference for a long time now, thanks again!) – toraritte Apr 06 '23 at 19:29
  • From NixOS Discourse: [When is a path in a Nix expression copied into the Nix store?](https://discourse.nixos.org/t/when-is-a-path-in-a-nix-expression-copied-into-the-nix-store/27052) – toraritte Apr 07 '23 at 01:24

1 Answers1

38

User clever on the #nixos IRC channel explained:

When it happens

The expansion into /nix/store/... happens when you use a path inside ${} string interpolation, for example mystring = "cat ${./myfile.txt}.

It does not happen when you use the toString function, e.g. toString ./myfile.txt will not give you a path pointing into /nix/store.

For example:

toString ./notes.txt == "/home/clever/apps/nixos-installer/installer-gui/notes.txt"
"${./notes.txt}"     == "/nix/store/55j24v9qwdarikv7kd3lc0pvxdr9r2y8-notes.txt"

Why it happens

Nix store paths (programs, data, configuration in /nix/store) are supposed to be hermetic and immutable once built.

"Hermetic and immutable" only works well if any files they refer to are also in the Nix store. If, for example, some nginx configuration file included another configuration file at /home/myuser/, hermeticity and immutability would be broken. This is why "copying path literals to the nix store" is a sensible default.

How it happens

The 55j24v9qwdarikv7kd3lc0pvxdr9r2y8 hash part is taken from the contents of the file referenced by the ./path, so that it changes when the file changes and things that depend on it can rebuild accordingly.

The copying of files into /nix/store happens evaluation time (e.g. when nix-instantiate runs to turn nix expressions into .drv files).

To make this possible, every string in nix has a "context" that tracks what the string depends on (in practice a list of .drv paths behind it).

For example, the string "/nix/store/rkvwvi007k7w8lp4cc0n10yhlz5xjfmk-hello-2.10" from the GNU hello package has some invisible state, that says it depends on the hello derivation. And if that string winds up as the input to stdenv.mkDerivation, the newly made derivation will "magically" depend on the hello package being built.

This works even if you mess with the string via builtins.substring. See this code of nix for how the context of the longer string is extracted in line 1653, and used as the context for the substring in line 1657.

You can get rid of a string's dependency context using builtins.unsafeDiscardStringContext.

Where it happens in the nix code

${} interpolation uses coerceToString, which has a bool copyToStore argument that defaults to true:

/* String coercion.  Converts strings, paths and derivations to a
   string.  If `coerceMore' is set, also converts nulls, integers,
   booleans and lists to a string.  If `copyToStore' is set,
   referenced paths are copied to the Nix store as a side effect. */
string coerceToString(const Pos & pos, Value & v, PathSet & context,
                      bool coerceMore = false, bool copyToStore = true);

It is implemented here, and the check for the interpolated thing being a ./path, and the copying to /nix/store, is happening just below:

if (v.type == tPath) {
    Path path(canonPath(v.path));
    return copyToStore ? copyPathToStore(context, path) : path;
}

toString is implemented with prim_toString, and it passes false for the copyToStore argument:

/* Convert the argument to a string.  Paths are *not* copied to the
   store, so `toString /foo/bar' yields `"/foo/bar"', not
   `"/nix/store/whatever..."'. */
static void prim_toString(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
    PathSet context;
    string s = state.coerceToString(pos, *args[0], context, true, false);
    mkString(v, s, context);
}
nh2
  • 24,526
  • 11
  • 79
  • 128
  • "The copying of files into /nix/store happens at the time of nix-instantiate; the evaluation of nix expressions is still purely functional (no copying around happens at evaluation time)," I don't think this is correct. Interpolating a path in the repl like `nix-repl> "${./foo.txt}"` will copy the file into the store. They seem to be copied at evaluation time https://nixos.org/manual/nix/stable/language/values.html#type-path – danidiaz Jul 29 '23 at 17:43
  • 1
    @danidiaz You are right, reading this part 6 years later, that part doesn't make sense. Also, me writing `instantiation ("building")`, doesn't make sense. I will fix it. – nh2 Jul 31 '23 at 10:03