1

The dataset has fillvalue of 1e30. While taking mean of the NetCDF files should i use the codena.rm=TRUE or na.rm=FALSE?

u2 <- list.files("/filepath/", pattern = "*.nc", full.names = TRUE) 
r  <- mean(u2, na.rm=TRUE)

I am getting values : -7.53555e+18, 1.263985e+18 (min, max) using na.rm=TRUE and -7.781048e+16, 3.923543e+15 (min, max) while coding withr <- mean(u2).

na.rm=TRUE is not giving correct result i feel as the data are exaggerated. Before making analysis of the data what should be done?

Data: Data_Test

Dipu
  • 123
  • 2
  • 14

1 Answers1

1
na.rm=TRUE

should lead R to ignore all missing values, have you checked to make sure that all files are using the same missing value? Are you sure you are not picking up additional files with other missing value definitions with your wild card? If I were you I would carry out the following checks:

First make sure that the files you list here are the ones you intend to average in R:

ls *.nc

Then check the definitions of the missing values:

for file in `ls *.nc` ; do ncdump $file | grep -i missing ; done

and you can also check the value you are getting by cross-checking using cdo:

cdo ensmean *.nc ensmean.nc
cdo fldmax ensmean.nc fldmax.nc
cdo fldmin ensmean.nc fldmin.nc  
ncdump fldmin.nc
ncdump fldmax.nc

and see what value that gives you and if it is different from that in the R programme...

Hope that helps to trouble shoot...

ClimateUnboxed
  • 7,106
  • 3
  • 41
  • 86
  • No other files are there in the directory and unfortunately, I am unable to install CDO in windows 10. I 'll upload all data and codes in a while. Just a few minutes – Dipu Jan 12 '20 at 18:25
  • I have uploaded all the details and data files for you. Kindly check in R and CDO if my output matches with yours. I have created outputs in NetCDF and Tif. You may check any one of them or both. There are also two documents about the data and output summary(minimumand maximum values of two files). – Dipu Jan 12 '20 at 19:07
  • hi Dipu. Ouch, unfortunately it seems CDO can't help you here, I took a look at your files and they use groups which are not supported by CDO yet :-( However, in general for future reference, CDO is now easy to use from Windows 10... Just google "install linux window 10" and you will find many blogs with the simple steps to install the linux ubuntu subsystem. Then you have pure linux within windows and you simply need to do "sudo apt install cdo". Sorry it can't help you validate these particular files though, I wonder if ec_codes from ECMWF can help? – ClimateUnboxed Jan 13 '20 at 07:43
  • no worries..Let me know if you find any solution later – Dipu Jan 13 '20 at 08:51