cffi error when compiling faster rcnn pytorch

Question

I'm trying to compile faster_rcnn_pytorch using the instructions given here: https://github.com/longcw/faster_rcnn_pytorch

I get this error:

(p27) [$USER@compute-1-5 faster_rcnn]$ ./make.sh
Traceback (most recent call last):
  File "setup.py", line 59, in <module>
    CUDA = locate_cuda()
  File "setup.py", line 45, in locate_cuda
    raise EnvironmentError('The nvcc binary could not be '
EnvironmentError: The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME
Compiling roi pooling kernels by nvcc...
./make.sh: line 10: nvcc: command not found
/home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling
generating /tmp/tmpP8mucv/_roi_pooling.c
setting the current directory to '/tmp/tmpP8mucv'
running build_ext
building '_roi_pooling' extension
creating home
creating home/$USER
creating home/$USER/play
creating home/$USER/play/hw2
creating home/$USER/play/hw2/gitdir
creating home/$USER/play/hw2/gitdir/hw2-release
creating home/$USER/play/hw2/gitdir/hw2-release/code
creating home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn
creating home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling
creating home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/../../lib/include -I/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/data2/$USER/anaconda2/envs/p27/include/python2.7 -c _roi_pooling.c -o ./_roi_pooling.o
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/../../lib/include -I/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/../../lib/include/TH -I/data2/$USER/anaconda2/envs/p27/include/python2.7 -c /home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src/roi_pooling.c -o ./home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src/roi_pooling.o
gcc -pthread -shared -L/data2/$USER/anaconda2/envs/p27/lib -Wl,-rpath=/data2/$USER/anaconda2/envs/p27/lib,--no-as-needed ./_roi_pooling.o ./home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src/roi_pooling.o /home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src/cuda/roi_pooling.cu.o -L/data2/$USER/anaconda2/envs/p27/lib -lpython2.7 -o ./_roi_pooling.so
gcc: error: /home/$USER/play/hw2/gitdir/hw2-release/code/faster_rcnn/roi_pooling/src/cuda/roi_pooling.cu.o: No such file or directory
Traceback (most recent call last):
  File "build.py", line 34, in <module>
    ffi.build()
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/__init__.py", line 167, in build
    _build_extension(ffi, cffi_wrapper_name, target_dir, verbose)
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/torch/utils/ffi/__init__.py", line 103, in _build_extension
    ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname)
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/cffi/api.py", line 697, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/cffi/recompiler.py", line 1520, in recompile
    compiler_verbose, debug)
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/cffi/ffiplatform.py", line 22, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/data2/$USER/anaconda2/envs/p27/lib/python2.7/site-packages/cffi/ffiplatform.py", line 58, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: LinkError: command 'gcc' failed with exit status 1

As far as I can tell, there are 2 problems:

1. nvcc is not found
2. cffi error

Regarding nvcc, I'm not sure how to add it to my path. It looks like I have both CUDA 8 and CUDA 9 so maybe I need to add it to only CUDA 9?

(p27) [$USER@compute-1-5 faster_rcnn]$ which nvcc
/usr/bin/which: no nvcc in (/data2/$USER/anaconda2/envs/p27/bin:/data2/$USER/anaconda2/bin:/opt/afni:/data2/$USER/anaconda2/bin:/opt/afni:/data2/$USER/anaconda2/bin:/opt/fsl/bin:/opt/openmpi/bin:/usr/lib64/qt-3.3/bin:/opt/afni:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ganglia/bin:/opt/ganglia/sbin:/usr/java/latest/bin:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbina:/home/$USER/bin:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbina:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbina)


(p27) [$USER@compute-1-5 faster_rcnn]$ find /opt -name nvcc
/opt/cuda-8.0/bin/nvcc
/opt/freesurfer/lib/cuda/bin/nvcc

(p27) [$USER@compute-1-5 faster_rcnn]$ echo $LD_LIBRARY_PATH
/home/$USER/lib/cuDNN/v7/cuda/lib64:/home/$USER/cuda-9.0/lib64:/home/$USER/cuda-9.0/extras/CUPTI/lib64:/home/$USER/lib/cuDNN/v7/cuda/lib64:/home/$USER/cuda-9.0/lib64:/home/$USER/cuda-9.0/extras/CUPTI/lib64:/opt/fsl/lib:/opt/openmpi/lib:/home/$USER/cuda/lib64:/home/$USER/cuda/lib64

Regarding cffi error, I have installed both gcc and cffi using conda but I'm not sure why it still complains.

(p27) [$USER@compute-1-5 faster_rcnn]$ conda list | grep cffi
cffi                      1.11.5           py27h9745a5d_0    anaconda
(p27) [$USER@compute-1-5 faster_rcnn]$ conda list | grep gcc
gcc                       4.8.5                         7    anaconda
libgcc-ng                 7.2.0                hdf63c60_3    anaconda

Thank you for helping!!

Answer 1

So looks like the problem was because I had 2 CUDA installations. For some reason, when I added CUDA 8.0 to the path, it did not work but when I added CUDA 9.0 to the path, it worked.

I added this to my bashrc:

export PATH=/home/$USER/cuda-9.0/bin:"$PATH"

To check whether nvcc is correctly added to the path, I ran this command as shown here :

cat $(which nvcc).profile