Calculating percentile across netcdf files fails

Question

I have five netcdf files where each file contains data for a time section. I want to calculate the 98th percentile for the whole timespan for each cell individually. The accumulated file size for the netcdf files is around 250 MB.

My approach it this:

library(raster)

  fileType="\\.nc$"
  filenameList <- list.files(path=getwd(), pattern=fileType, full.names=F, recursive=FALSE)
  #rasterStack for all layers
  rasterStack <- stack()

  #stack all data
  for(i in 1:length(filenameList)){

    filename <- filenameList[i]
    stack.temp<-stack(filename)
    rasterStack<-stack(rasterStack, stack.temp)

  }

  #calculate raster containing the 98th percentiles
  result <- calc(rasterStack,  fun = function(x) {quantile(x,probs = .98,na.rm=TRUE)} )

However, I get this error:

Error in ncdf4::nc_close(x@file@con) : 
  no slot of name "con" for this object of class ".RasterFile"

The stacking section of my code works, the crash happens during the calc function. Do you have any idea where this might come from? Is it maybe an issue of where the data is stored (memory/disk)?

Answer 1

Strange, I generated some dummy data and it seems to work just fine, it does not seems to be your method. 250MB is not overly huge. I would clip a small piece of each raster and test if it works.

dat<-matrix(rnorm(16), 4, 4)
r1<-raster(dat)
r2<-r1*2
r3<-r2+1
r4<-r3+4
rStack <- stack(r1,r2,r3,r4)
result <- calc(rStack,  fun = function(x) {quantile(x,probs = .98)} )

Answer 2

Perhaps this is related to the odd way you create a RasterStack. You should simply do:

  filenames <- list.files(pattern="\\.nc$")
  rasterStack <- stack(filenames)