2

I have just installed gdal into my Python3 root environment using conda:

conda install -c conda-forge gdal

Installation went fine with the usual updates of some dependencies. Now returning to some of my scripts, both netCDF4 and mpl_toolkits.basemap have stopped working. These are the errors I get:

from netCDF4 import Dataset

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-7-09694a7f0e01> in <module>()
----> 1 from netCDF4 import Dataset
      2 import numpy as np
      3 import matplotlib.pyplot as plt
      4 from scipy.interpolate import griddata
      5 from mpl_toolkits.basemap import Basemap

/anaconda3/lib/python3.6/site-packages/netCDF4/__init__.py in <module>()
      1 # init for netCDF4. package
      2 # Docstring comes from extension module _netCDF4.
----> 3 from ._netCDF4 import *
      4 # Need explicit imports for names beginning with underscores
      5 from ._netCDF4 import __doc__, __pdoc__

ImportError: dlopen(/anaconda3/lib/python3.6/site-packages/netCDF4/_netCDF4.cpython-36m-darwin.so, 2): Library not loaded: @rpath/libhdf5.101.dylib
  Referenced from: /anaconda3/lib/python3.6/site-packages/netCDF4/_netCDF4.cpython-36m-darwin.so
  Reason: image not found

from mpl_toolkits.basemap import Basemap

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-8-5fb601138328> in <module>()
      3 import matplotlib.pyplot as plt
      4 from scipy.interpolate import griddata
----> 5 from mpl_toolkits.basemap import Basemap
      6 import numpy.ma as ma
      7 import glob

/anaconda3/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py in <module>()
     35 import numpy as np
     36 import numpy.ma as ma
---> 37 import _geoslib
     38 import functools
     39 

ImportError: dlopen(/anaconda3/lib/python3.6/site-packages/_geoslib.so, 2): Library not loaded: @rpath/libgeos-3.5.0.dylib
  Referenced from: /anaconda3/lib/python3.6/site-packages/_geoslib.so
  Reason: image not found

I have no idea what these errors mean, since those paths do exist. What is most baffling is that the netCDF4 and mpl_toolkits modules did not change (i.e. were not updated/downgraded) with the installation of gdal, so why now are they failing?

I'm at the end of my tether with problems like this. Fortunately I made a copy of my Python root directory before attempting this installation so I can now revert back to it (I've been here many times before).

I know about working in separate conda environments, but am I seriously supposed to install a new environment every time I want to use a new Python module? This not only takes up a lot of harddrive space but most inconveniently means adding new functionality to old codes is impossible (I want to use GDAL to plot some geotiffs using Basemap, which now does not work!).

I cannot imagine proficient Python users battling with these issues - so what am I doing wrong?

BlueSheepToken
  • 5,751
  • 3
  • 17
  • 42
izzyrizzy
  • 43
  • 4
  • "but am I seriously supposed to install a new environment every time I want to use a new Python module?" If you expect this to happen often, you can temporarily clone your root environment. – Mad Physicist Apr 11 '19 at 12:18
  • do you mean by copying my 7GB folder root folder to another location (as I am doing), or is there a more nifty way? Also how does that help me run Basemap and gdal in the same environment? – izzyrizzy Apr 11 '19 at 12:22
  • Problems like this happen to me time to time when I mix packages from different sources (pip, conda, conda-forge, so on) in the same environment. The reason is somewhere in the packages' dependencies. Probably `netCDF4` was built against another version of `libhdf5` in comparison with the one currently in use. So you may try to reinstall `gdal` using different versions/sources. This may help and may not. –  Apr 11 '19 at 13:48
  • I always use conda-forge specifically because it is supposed to manage my dependencies for me and, I thought, avoid these issues. I have re-copied my saved root environment and made a new one for gdal, basemap, netcdf4 (another 2GB of harddisk space gone). It's working but I still can't believe this is the way it's supposed to be done... – izzyrizzy Apr 11 '19 at 13:53
  • @izzyrizzy. I think you can clone the environment, which, at least in theory, should be faster than a manual copy. How did you end up with a 7GB folder? – Mad Physicist Apr 11 '19 at 14:00
  • My entire python directory, i.e. "/anaconda3" is 7GB (actually it's now 11GB since creating a new environment to work around this problem!). I'll try cloning my root environment for future re-sets. Thank you – izzyrizzy Apr 11 '19 at 14:05
  • 1
    "*copying my...root folder to another location*" oh no. There's a `conda create --clone` command that will use hardlinks so you're not duplicating most things. Also, I really think most users should not be using Anaconda, but instead Miniconda. My main gripe is that Anaconda encourages a monolithic **base** env, whereas I think you should only use **base** for infrastructure (Jupyter, git, IDEs, etc.) and make lots of specialized lightweight envs for your projects. This really mitigates against breaking **base**. – merv Apr 11 '19 at 15:53
  • Thanks Merv - I have since figured out cloning my root env- but will look into Miniconda. – izzyrizzy Apr 12 '19 at 10:25

0 Answers0