0

Unable to import an existing module, using Ubuntu 16.04, python 3.5, Anaconda 4.2.15, tensorflow 0.12.

The TensorFlow package contains a helper module called mnist.py. I've verified that the file exists on my machine, and manually added the directory to sys.path. When I try to import the module, python can't seem to find it. I can import tensorflow as tf just fine, but python seems to be looking for the module in the wrong directory although I've explicitly given it the correct location:

>>> import tensorflow.contrib.learn.python.learn.datasets.mnist
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/__init__.py", line 25, in <module>
    from tensorflow.contrib import cudnn_rnn
  File "/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/__init__.py", line 21, in <module>
    from tensorflow.contrib.cudnn_rnn.python.ops.cudnn_rnn_ops import CudnnGRU
  File "/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py", line 28, in <module>
    resource_loader.get_path_to_datafile("_cudnn_rnn_ops.so"))
  File "/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/util/loader.py", line 42, in load_op_library
    ret = load_library.load_op_library(path)
  File "/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library
    None, None, error_msg, error_code)
tensorflow.python.framework.errors_impl.NotFoundError: /home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/_cudnn_rnn_ops.so: cannot open shared object file: No such file or directory

Verifying the file 

 ls -l ~/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets
    total 44
    -rw-rw-r-- 2 rh rh 4660 Dec  1 12:09 base.py
    drwxrwxr-x 3 rh rh 4096 Jan 12 19:57 data
    -rw-rw-r-- 2 rh rh 2126 Dec  1 12:09 __init__.py
    -rw-rw-r-- 2 rh rh 1334 Dec  1 12:09 load_csv_test.py
    -rw-rw-r-- 2 rh rh 8308 Dec  1 12:09 mnist.py
    -rw-rw-r-- 2 rh rh 1317 Dec  1 12:09 produce_small_datasets.py
    drwxrwxr-x 2 rh rh 4096 Jan 12 19:57 __pycache__
    -rw-rw-r-- 2 rh rh 2745 Dec  1 12:09 text_datasets.py

Verifying the path 

 >>> sys.path
    ['', '/home/rh/Applications/Anaconda/anaconda3/lib/python35.zip', 
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5', 
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/plat-linux',
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/lib-dynload',
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages',
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/Sphinx-1.4.6-py3.5.egg', 
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg', 
    '/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets']

Tensorflow Info 

$ conda list | grep tensorflow  
tensorflow                0.12.0rc0                py35_0    jjhelmus

Only 1 Python Environment 

$ conda env list
# conda environments:
#
root                  *  /home/rh/Applications/Anaconda/anaconda3

$ type -p python
/home/rh/Applications/Anaconda/anaconda3/bin/python

My source code comes from a tensorflow tutorial that runs perfectly on OSX, something about the Ubuntu install/environment is wonky. I've been beating on this problem for a while without success. This related question doesn't address the issue either. Any suggestions?

Community
  • 1
  • 1
RedHand
  • 105
  • 10
  • The real issue is the last line of your error message `/home/rh/Applications/Anaconda/anaconda3/lib/python3.5/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/_cudnn_rnn_ops.so: cannot open shared object file: No such file or directory`. Is it possible you installed the GPU version without installing CUDA and cuDNN? – gunan Jan 17 '17 at 07:50
  • My GPU is ATI, so there isn't any CUDA stuff installed. I installed Anaconda from the Py3.5 64 bit installer `Anaconda3-4.2.0-Linux-x86_64.sh`, downloaded from the Anaconda [site](https://www.continuum.io/downloads#linux). – RedHand Jan 18 '17 at 02:39
  • Regarding GPU accessibility, from [https://docs.continuum.io/accelerate/](https://docs.continuum.io/accelerate/): "Accelerate provides access to numerical libraries optimized for performance on Intel CPUs and NVidia GPUs". It requires installation of the `accelerate` package, which I don't have: `$ conda list | grep accelerate` returns blank. I can't find any indication that stock Anaconda includes GPU accessibility. – RedHand Jan 18 '17 at 02:45
  • Which tensorflow pip package did you install? -- I just realized you installed through conda. The packages in conda are community maintained, so I am not sure what might be wrong there. Is it possible for you to try a pip install, maybe in a clean conda environment? the command should be `pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp35-cp35m-linux_x86_64.whl` – gunan Jan 19 '17 at 00:49

1 Answers1

0

Found a clue in the answer to this related question, and @gunan's comment.

Although the machine in question doesn't have a python-gpu interface, python kept looking for cudnn files. tensorflow 0.12.0rc0 (installed from a private repo) offers gpu support by default, and I couldn't disable it by resetting environment variables.

The solution was to remove tensorflow 0.12.0rc0 and replace it with tensorflow 0.10.0rc0 from the default Anaconda repositories.

$ conda remove tensorflow
$ conda install tensorflow
$ conda list | grep tensorflow  
tensorflow                0.10.0rc0           np111py35_0


            ----------

Warning

Several tensorflow functions were deprecated between versions 10 and 12. At present time, version 12 is the recommended tensorflow package for OSX Anaconda Python (installed from private repos with Anaconda pip). Any script that needs to run on both Mac and non-gpu Linux should use the deprecated version 10 functions. Mac users will need to suppress the resulting warnings.

Community
  • 1
  • 1
RedHand
  • 105
  • 10