2

I have netcdf files where _FillValue and/or missing value are set in various combinations (e.g. as float, string, or not at all, and generally as -999./'-999' ). I am manipulating these files and saving them again and want to set the fill value properly and consistently in my output files. My questions are:

1) use _FillValue, or missing_value, or both, and what is the difference?

2) Which type should the fill value be, and what's the best practice regarding value? -999 seems like it could be confused with actual data

3) how does the netdcf fill value combine with the fill_value in my masked array (which seems to default to 10^20)? it seems like two approaches to correct the same thing.

4) I also get a warning when dealing with the data: "UserWarning: WARNING: missing_value not used since it cannot be safely cast to variable data type", and a _FillValue gets automatically set to roughly 10^36. SO I presume my -999 missing value is indeed a bad idea.

I am using python 3.7 and netcdf4.

Thanks!

Therese
  • 63
  • 1
  • 2
  • 6
  • In future, try to avoid asking multiple questions in the same post - Far better to split them up into separate questions :-) – ClimateUnboxed Nov 11 '20 at 14:07

1 Answers1

0

To answer point 2, each type (float, integer etc) has a default missing value, which is available through the specific netcdf module for the language you are using.

To give an example, if you are using Fortran90 (or above) then once you have loaded the netcdf module, you can access the default missing value for a double precision floating point variable using the predefined variable

NF90_FILL_DOUBLE

Instead, in python3, which you specifically asked about, it is a little more complicated as it depends which package you are using to handle NETCDF. But if you use the netCDF4 package (which is now fairly standard) then you can view the default missing values in this way:

import netCDF4
netCDF4.default_fillvals
{'S1': '\x00', 'i1': -127, 'u1': 255, 'i2': -32767, 'u2': 65535, 'i4': -2147483647, 'u4': 4294967295, 'i8': -9223372036854775806, 'u8': 18446744073709551614, 'f4': 9.969209968386869e+36, 'f8': 9.969209968386869e+36}

as outlined in this post

ClimateUnboxed
  • 7,106
  • 3
  • 41
  • 86