4

Lately I have been doing some interactive work in Python.

My setup is an IPython notebook running on a server that uses a grid engine to manage jobs.

Today I was trying to get an IPython cluster going following an example posted here that uses subprocess.Popen to start a the cluster.

I couldn't get the example to work so I tried opening up the IPython/Jupyter terminal emulator and typing in the ipcluster start command and the cluster started right up!

After playing around with things for a while I realized that if I typed env in the terminal emulator I got a different list of environment variables than when I looked at the os.environ variable in Python. The source of the problem seemed to be that the PATH variables were different.

Now I know that I can change the PATH variable in os.environ, but I'm wondering why it is different in the first place? I know very little about environment variables, so this may be a stupid question, but I would have assumed that a terminal emulator and notebook running on the exact same node in the exact same IPython notebook server would have had the exact same environment variables.

Any insight on why the environment variables in the terminal and notebook might be different will be greatly appreciated.

Update: In case it matters, the server I am working on uses the Univa Grid Engine. Also I have noticed that it seems to make a difference whether I use qrsh or qsub to start the notebook server.

Previously I had been using qsub, but by starting the notebook server with qrsh I eliminate many of the differences between env and os.environ. There are still differences, but much fewer. Still not sure what any of that means though:)

Joseph Stover
  • 397
  • 4
  • 13
  • perhaps you're getting the outcome from different nodes of the hpc cluster? What happens if you do the following? temp = %env os.environ == temp – fernandezcuesta Aug 13 '15 at 16:13
  • @valtuarte: `%env` gives the exact same output as `os.environ.copy()`. Also, the `HOSTNAME` environment variable is the same in `os.environ` and when I type `env` in the terminal emulator, so I think the processes are on the same node. – Joseph Stover Aug 13 '15 at 16:33
  • According to this, https://docs.python.org/2/library/os.html, your issue may be that `env` is being changed *after* the `os` module is loaded. – Vince Aug 13 '15 at 19:36
  • 1
    Also, are you running qsub with the -V option? If not, even if you execute job on same node the `env` will not get propagated into the job's environment, and thus will not get migrated to subprocess.Popen(). – Vince Aug 13 '15 at 19:40
  • @Vince: I had not been using the -V option, and using it makes `env` and `os.environ` match up (at least as far as I have checked), so thanks very much for the tip. Do you have a good explanation for what's going on here? My intuition is that both the terminal and IPython notebook are part of the same job (the notebook server), so I would expect the variables to be the same regardless. This is obviously not the case, so any intuition into why would be greatly appreciated. Thanks again. – Joseph Stover Aug 13 '15 at 20:15
  • I assume you are running qrsh without `-V` option? If so, the `env` where you execute qrsh will not get imported into the interactive job (terminal), and thus not propagate to IPython notebook, and thus will not propagate to subprocess.Popen(). – Vince Aug 13 '15 at 20:27

1 Answers1

1

As per manual page for qsub, qsh, qrsh, to propagate current shell environment to the job use the -V option:

 -V     Available for qsub, qsh, qrsh with command and qalter.

        Specifies that all environment variables active within the qsub utility be exported to the context of the job.

        All environment variables specified with -v, -V or the DISPLAY variable provided with -display will be exported to the defined JSV  instances  only  optionally  when  this  is
        requested explicitly during the job submission verification.  (see -jsv option above or find more information concerning JSV in jsv(1))
Vince
  • 3,325
  • 2
  • 23
  • 41