4

I have a problem on OSX that #!/usr/bin/env python3 is causing the environment variables, such as library paths (LD_LIBRARY_PATH for example), to be lost. On Linux it works correctly.

Reading the man page on env it doesn't appear it should be doing this. It should only be modifying the environment if I request it, and I am not.

What is a portable shebang line that preserves the environment?

Note: I'm detecting the problem since a call to subprocess.open on one of my programs fails since it can't find one of the libraries. Yet if I start python interactively the same call works fine.

edA-qa mort-ora-y
  • 30,295
  • 39
  • 137
  • 267
  • Never mind, only *certain* env vars in the current shell are indeed not propagated it seems. I tested with a generic one and that worked. – Martijn Pieters Jan 28 '17 at 18:55
  • This should only happen with `-` or `-i` passed, by a plain reading of the MacOS man page for `env`. – Charles Duffy Jan 28 '17 at 18:58
  • `env env`, for instance, shows all environment variables present in the inner instance. – Charles Duffy Jan 28 '17 at 18:59
  • @CharlesDuffy: I tried with `pprint.pprint(dict(os.environ))` and can't seem to get *specific* environment variables to propagate. Not with `LD_LIBRARY_PATH =bar ./test.py` and not with `export LD_LIBRARY_PATH=bar`, then `./test.py`. – Martijn Pieters Jan 28 '17 at 18:59
  • @MartijnPieters, hmm. Only as a shebang, I assume? `env python -c 'import os, pprint; pprint.pprint(dict(os.environ))'` works for me, likewise with `python3`. – Charles Duffy Jan 28 '17 at 19:00
  • @CharlesDuffy: Yes, with the first line being `#!/usr/bin/env python3`, and of course `import os` and `import pprint`. – Martijn Pieters Jan 28 '17 at 19:00
  • Hmm. You're right, for `LD_LIBRARY_PATH` we've got something interesting going on. – Charles Duffy Jan 28 '17 at 19:01
  • 1
    @CharlesDuffy: `LD_LIBRARY_PATH=foo env env | grep LD_LIBRARY_PATH` reproduces the issue too, while `FOO=foo env env | grep FOO` works as expected. Looks like some specific filtering. – Martijn Pieters Jan 28 '17 at 19:03
  • I can reproduce it for DYLD_LIBRARY_PATH too. Interesting. – Martijn Pieters Jan 28 '17 at 19:05
  • `LD_LIBRARY_PATH` is explicitly documented to be discarded for setuid and setgid binaries, but that's not the case here. Yes, this is presumably an undocumented security-related behavior. – Charles Duffy Jan 28 '17 at 19:06
  • 1
    Cross-site duplicate of [El Capitan, make check, DYLD\_LIBRARY\_PATH](//apple.stackexchange.com/q/215030) – Martijn Pieters Jan 28 '17 at 19:06
  • 1
    Also related: [Unable to set DYLD\_FALLBACK\_LIBRARY\_PATH in shell on OSX 10.11.1](//apple.stackexchange.com/q/212945). `env`, being a `/usr/bin` binary, is 'protected' in that some environment variables are stripped by the *System Integrity Protection*. – Martijn Pieters Jan 28 '17 at 19:08

1 Answers1

3

This is the OS X System Integrity Protection at work; certain 'dangerous' environment variables such as DYLD_LIBRARY_PATH, and, as you discovered, LD_LIBRARY_PATH, are not passed to any binary with a /usr/bin or /bin path.

As such, /usr/bin/env never sees any of several environment variables and can't pass those on to the python3 child process it spawns.

See several other posts on the subject:

If you are using subprocess, I'd check for the shebang line, extract the binary named after env, prefix your command with that binary, and bypass env altogether.

Community
  • 1
  • 1
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • For my call to subprocess I could do that, but I have other scripts that call these ones (liek the build scripts) which have no control overhow they are called. – edA-qa mort-ora-y Jan 28 '17 at 19:42
  • In any case this really sucks. This "feature" of OSX makes it very difficult to setup a development environment. – edA-qa mort-ora-y Jan 28 '17 at 19:43
  • Yup, it makes things considerably harder; avoid using `/usr/bin/env` on scripts that need access to these variables, is the best I can do at the moment. – Martijn Pieters Jan 28 '17 at 19:45