2

Environment:

Python 3.8.6
Conda 4.8.5
Mac OS Catalina 10.15.7

Attempting to read in an ESRI shapefile of UK upper tier local authority boundaries to a python environment. Attempted in both command line and PyCharm with identical error.

Setup following document guidance:

conda create -n geo_env
conda activate geo_env
conda config --env --add channels conda-forge
conda config --env --set channel_priority strict
conda install python=3 geopandas

I have confirmed all dependency packages (fiona, pyproj, shapely, dumpy, pandas) are installed and imported to environment.

But when running the following:

local_auth_shape = gpd.read_file('~/Downloads/Counties_and_Unitary_Authorities__December_2017__Boundaries_GB-shp/Counties_and_Unitary_Authorities__December_2017__Boundaries_GB.shp')

I am met with the error:

fiona.errors.DriverError: ~/Downloads/Counties_and_Unitary_Authorities__December_2017__Boundaries_GB-shp/Counties_and_Unitary_Authorities__December_2017__Boundaries_GB.shp: No such file or directory

The path is correct and I have triple checked it. The geo database folder has also downloaded without error.

I have attempted using pip install also with the same error.

RAH
  • 395
  • 2
  • 9
  • I had the same error once with geopandas 2.0.1 It was due to the geodataframe being empty, I added another check opening the shapefile in fiona and checking if it is not empty then reading it with geopandas – Hicham Zouarhi Oct 19 '20 at 08:38
  • Why would the data frame be empty? I've tried it with multiple geodatabases and boundary files and keep getting the same error. – RAH Oct 19 '20 at 08:39
  • 1
    Have you tried using another path ? an absolute path starting from root – Hicham Zouarhi Oct 19 '20 at 08:41
  • 1
    Your path is incorrect. It should look like `/Users/martin/Downloads/file.shp` not with `~`. – martinfleis Oct 19 '20 at 09:14
  • That fixed it. Why was this required when other functions such as read_csv can work fine with the tilde? – RAH Oct 19 '20 at 09:39
  • 1
    The `read_csv` function is from the pandas package, and indeed supports this. I opened https://github.com/geopandas/geopandas/issues/1664 about potentially supporting this in geopandas as well – joris Oct 19 '20 at 11:23
  • being specific with the path really saves a lot of potential problems – brygid Aug 30 '21 at 20:49

1 Answers1

-1

I don't think there is an issue with your path. You need to run the pyspark locally rather than on the cluster or Standalone.

Initially, my spark was set to

sc = SparkContext("spark://panther:7077", "FirstApp")

And it was throwing the error that the file doesn't exist but when I changed it to "local", the error went away.

sc = SparkContext("local", "FirstApp") <- This should fix the problem for you
Javad
  • 2,033
  • 3
  • 13
  • 23
Ishan Anand
  • 119
  • 1
  • 7
  • 1
    While I posted this around a year and a half ago and don't fully remember the set-up (long since gave up and tried a different approach), I want to comment just so any ideas are not going ignored. Spark was not involved at any point. It was reading a shapefile into a local python environment using geopandas – RAH Apr 19 '22 at 09:24
  • Spark irrelevant to the initial question. – guiotan Aug 03 '23 at 09:41