1

I'm having a problem for python to find the installed libraries when I run it in a computer cluster.

When I try, e.g., to load numpy in the script:

 #file: /home/foo/test.py
 import numpy 
 print numpy.__version__

on the server, I get this:

foo@abax:~$ python test.py 
1.4.1

but when I try to run the same in a node with remote shell, I get an error:

foo@abax:~$ rsh -l foo ab01 "python test.py"
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    import numpy 
ImportError: No module named numpy

is there a way to tell python to load the files that are installed in the the central node of the cluster?

Amro
  • 123,847
  • 25
  • 243
  • 454
Rafael S. Calsaverini
  • 13,582
  • 19
  • 75
  • 132
  • 1
    Take a look at [find_module](http://docs.python.org/library/imp.html#imp.find_module), it might help you out. – brc Sep 19 '11 at 18:44

1 Answers1

1

First things to check:

  • Print PYTHONPATH on both the frontal server and the cluster nodes, to make sure there is no inconsistency
  • Print numpy.__file__ on the frontal server, to check where it finds numpy. Then explore the filesystem of the cluster nodes a bit to see if numpy can be found in the same place (if not, run a search to see if you can find it, then update your PYTHONPATH accordingly).

It may just be that numpy is installed locally on the frontal server, but not on cluster nodes. In that case you will need to install numpy by yourself on a filesystem that cluster nodes can access (note that on a scientific cluster, it would be better to ask the cluster admins to install numpy on cluster nodes, to make it available to everyone).

If the platforms are similar enough, copying the numpy folder from the frontal server to somewhere in the shared filesystem (e.g. a subfolder of your home dir, that you would add to your PYTHONPATH) might work, but a clean install is to be preferred.

tiho
  • 6,655
  • 3
  • 31
  • 31
  • PYTHONPATH is empty... also the library files are absent in the cluster nodes. I thought that the admin had replicated the directories for libs, includes, etc... in all nodes. I'll ask him if this is possible. If not, I'll try to install numpy in my home folder (which is replicated in all nodes). :( – Rafael S. Calsaverini Sep 20 '11 at 14:21