0

I have two issues related to the error:

first: I have one merged dem layer and multiple shapefiles, I create a list of masked shapefiles boundary, I was able to plot all of them except one "the first one" which is the biggest one: 

> plot(DEM_masked_list[[1]])

Error in file(fn, "rb") : cannot open the connection
In addition: Warning message:
  In file(fn, "rb") :
  cannot open file '/private/var/folders/2w/rjzwcrbn3pg0jmsrfkz7n52h0000gn/T/RtmpkL8Ot5/raster/r_tmp_2018-01-29_014745_982_20879.gri': No such file or directory

I notice the data source of the first dem differ from all the others, that might be due to the larger size of it (509141570 no. of the cell)!!

DEM_masked_list

[[1]]
class       : RasterLayer 
dimensions  : 20015, 25438, 509141570  (nrow, ncol, ncell)
resolution  : 9.259259e-05, 9.259259e-05  (x, y)
extent      : -70.43231, -68.07694, 45.98676, 47.84  (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs 
data source : /private/var/folders/2w/rjzwcrbn3pg0jmsrfkz7n52h0000gn/T/RtmpkL8Ot5/raster/r_tmp_2018-01-29_014745_982_20879.grd 
names       : layer 
values      : 121.4266, 856.6606  (min, max)


[[2]]
class       : RasterLayer 
dimensions  : 9043, 9896, 89489528  (nrow, ncol, ncell)
resolution  : 9.259259e-05, 9.259259e-05  (x, y)
extent      : -69.76269, -68.84639, 46.23528, 47.07259  (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs 
data source : in memory
names       : layer 
values      : 187.9911, 650.0044  (min, max)

Second: I merged 25 separate dem in one layer (DEM_merged), the data source also not stored in memory, I was able to plot it and work with it for one day which is 2018-01-28 (appear in the data source), then the same error appeared.

> DEM_merge
class       : RasterLayer 
dimensions  : 75612, 75612, 5717174544  (nrow, ncol, ncell)
resolution  : 9.259259e-05, 9.259259e-05  (x, y)
extent      : -74.00056, -66.99944, 40.99944, 48.00056  (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +no_defs 
data source : /private/var/folders/2w/rjzwcrbn3pg0jmsrfkz7n52h0000gn/T/RtmpkL8Ot5/raster/r_tmp_2018-01-28_163201_982_66674.grd 
names       : layer 
values      : -81.04944, 1915.734  (min, max)

> plot(DEM_merge)
Error in file(fn, "rb") : cannot open the connection
In addition: Warning message:
  In file(fn, "rb") :
  cannot open file '/private/var/folders/2w/rjzwcrbn3pg0jmsrfkz7n52h0000gn/T/RtmpkL8Ot5/raster/r_tmp_2018-01-28_163201_982_66674.gri': No such file or directory
>

Is there any way to fix that? I feel there is some issue with Raster package and the way that it store the data, I tried to reinstall Raster package, reinstall R, even I used a different computer after I post here but still the same issue, Appreciate your help!!

neilfws
  • 32,751
  • 5
  • 50
  • 63
Nuha J. Alhowramy
  • 55
  • 1
  • 7

1 Answers1

7

The values of large Raster* objects are written to file, to avoid memory limitation problems. If you do not explicitly provide a filename, they are stored in the temporary data folder that will be removed when the R session ends.

I am guessing you created the RasterLayers and saved the list to disk, and closed R? Or perhaps you reloaded your session when opening R again?

Just guessing, but if so, the values of the large raster should indeed have disappeared. To avoid that from happening, you can either try to force all values into memory with readAll (not recommended), or write them to a permanent file with writeRaster

Robert Hijmans
  • 40,301
  • 4
  • 55
  • 63
  • That is very clear and unique explanation, Thank you RobertH!! It took me 16 hrs to run the code, please, Which one is correct? writeRaster(DEM_merge, filename="DEM_merge.grd", bandorder='BIL', overwrite=TRUE), or writeRaster(DEM_merge, 'DEM_merge.tif', overwrite=TRUE). – Nuha J. Alhowramy Jan 30 '18 at 03:11
  • I used `saveRDS` to save a raster tile to `.rds` after performing operations in R. After closing the session and reopening, I can load the `.rds` files correctly (`ncell=437760000`) and display basic information but I get the `cannot open file` error when I try to perform any operations (e.g. `crop`, `cover`). Am I running into the same problem? Inherent limits to size of raster that can be saved to `rds`? – user3386170 Feb 03 '19 at 21:29
  • 2
    As a general practice, you should not save Raster* objects to rds. Use writeRaster instead. This is because large objects are backed by temporary files on disk that disappear at the end of a session. – Robert Hijmans Feb 04 '19 at 16:51