I am running python based software that requires a virtualenv with various additional python packages installed therein. Here are the install instructions:
Clone the repository to a shared filesysem on a cluster
>git clone https://github.com/jgurtowski/nanocorr
>cd nanocorr
Create a virtual environment to install python dependencies
>virtualenv nanocorr_ve
>source nanocorr_ve/bin/activate
install the following packages using pip:
pip install git+https://github.com/cython/cython
pip install numpy
pip install h5py
pip install git+https://github.com/jgurtowski/pbcore_python
pip install git+https://github.com/jgurtowski/pbdagcon_python
pip install git+https://github.com/jgurtowski/jbio
pip install git+https://github.com/jgurtowski/jptools
#Finally install the nanocorr package itself
> python setup.py install
I achieved this with no problems. However, While running the software, I found that it was failing with the following traceback.
(nanocorr_ve)[nanocorr_test]$ correctOxford
Traceback (most recent call last):
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/bin/correctOxford", line 9, in <module>
load_entry_point('jptools==0.1', 'console_scripts', 'correctOxford')()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 558, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2682, in load_entry_point
return ep.load()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2355, in load
return self.resolve()
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pkg_resources/__init__.py", line 2361, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/jptools/correct.py", line 21, in <module>
from pbtools.pbdagcon.q_sense import output_dag_info
File "/local/workdir/malonge/nanocorr_test/nanocorr/nanocorr_ve/lib/python2.7/site- packages/pbtools/pbdagcon/q_sense.py", line 50, in <module>
from pbcore.io import FastaReader
File "/programs/python/lib/python2.7/site-packages/pbcore-1.0.0- py2.7.egg/pbcore/io/__init__.py", line 31, in <module>
from .BasH5IO import *
File "/programs/python/lib/python2.7/site-packages/pbcore-1.0.0- py2.7.egg/pbcore/io/BasH5IO.py", line 37, in <module>
import h5py, numpy as np, os.path as op
File "/programs/python/lib/python2.7/site-packages/h5py-2.5.0-py2.7-linux- x86_64.egg/h5py/__init__.py", line 13, in <module>
from . import _errors
ImportError: /programs/python/lib/python2.7/site-packages/h5py-2.5.0-py2.7-linux-x86_64.egg/h5py/_errors.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
I have found details as to the specific error itself, but I think the source of the problem is that the traceback clearly shows a transition from modules being imported from the /local virtualenv site-packages to another site-packages directory in /programs.
My question is the following: Given that I have the same python packages installed in two separate places, 1 in the system wide location and one in virtualenv, how do I ensure that other programs properly import all modules from within the same virtualenv sit-packages directory. I thought this would have been done automatically. I am not 100% sure that this is the sole reason for the error, but being able to fix this would help me to further debug.
