Monthly sum of of a variable stored at daily timesteps using cdo (Climate Data Operators)

Question

I have a nc file of 40 years for a variable recorded at daily temporal resolution. I would like to sum these over monthly intervals and have tried the following:

cdo monsum inputfile.nc outputfile.nc

Although this runs with no error, I only get one frame in the output file. I would expect a much larger file.

I am thinking this might be some issue with time in the input file.

Copied info from ncdump-h input file:

dimensions:
       time = UNLIMITED ; // (14975 currently)
       longitude = 40 ;
       latitude = 66 ;
variables:
       double time(time) ;
               time:standard_name = "time" ;
               time:units = "days since 1979-1-1 00:00:00" ;
               time:calendar = "proleptic_gregorian" ;
               time:axis = "T" ;
       double longitude(longitude) ;
               longitude:standard_name = "projection_x_coordinate" ;
               longitude:long_name = "longitude" ;
               longitude:units = "degrees_east" ;
               longitude:axis = "X" ;
       double latitude(latitude) ;
               latitude:standard_name = "projection_y_coordinate" ;
               latitude:long_name = "latitude" ;
               latitude:units = "degrees_north" ;
               latitude:axis = "Y" ;
       int crs ;
               crs:proj4 = "+proj=longlat +datum=WGS84" ;
       float variable(time, latitude, longitude) ;
               variable :grid_mapping = "crs" ;
               variable :_FillValue = -3.4e+38f ;
               variable :missing_value = -3.4e+38f ;
               variable :proj4 = "+proj=longlat +datum=WGS84" ;
               variable :min = 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0. ;
               variable :max = 6.5701843261719, 8.62581787109377, 6.24334106445315, 8.29082641601565, 7.59359130859377, 10.0182434082031, 9.76052246093752, 9.95830688476565, 10.3648010253906, 12.8482604980469, 13.1276489257813, 13.1267333984375, 10.9829345703125, 9.82497558593752, 10.1014953613281, 11.8195434570313, 13.9579711914063, 10.0517211914063, 9.37850341796877, 11.3964782714844, 13.0114074707031, 14.4672180175781, 11.4755798339844, 10.6639953613281, 11.3891540527344, 11.9616943359375, 13.7388854980469, 5.56426391601565, 7.07705078125002, 7.64479980468752, 8.25527343750002 ; // global attributes:
               :CDI = "Climate Data Interface version 1.9.10 (https://mpimet.mpg.de/cdi)" ;
               :Conventions = "CF-1.4" ;
               :created_by = "R, packages ncdf4 and raster (version 3.4-10)" ;
               :date = "2021-10-06 11:52:06" ;
               :history = "Mon Oct 18 13:50:40 2021: cdo setreftime,1979-01-01,00:00:00 kdd_AR_1979_2019.nc kdd_AR_1979_2019_adj.nc\n",

Help is much appreciated. Sorry for not being to provide a straightforward reproducible example. I can share the data via we-transfer or such if needed. Thanks!

should work. what do you get if you type cdo showdate file.nc on the input and output files? — ClimateUnboxed, Oct 18 '21 at 19:36
Thanks for the reply. I get the following on the input: `...00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 00:00:00 cdo showtime: Processed 1 variable over 14975 timesteps [8.49s 14MB].` and the following on the output: `Warning (cdf_read_mapping_atts): Text attribute crs:grid_mapping_name missing! 00:00:00 cdo showtime: Processed 1 variable over 1 timestep [0.20s 9288KB].` I added the warning msg although I don't believe it should matter ? — Raed Hamed, Oct 18 '21 at 19:51
... although would be nice to see the result of showdate, not showtime. — ClimateUnboxed, Oct 18 '21 at 20:01
Unfortunately, the proposed solution so far didn't work. On the other hand, here the result of showdate: `Warning (cdf_read_mapping_atts): Text attribute crs:grid_mapping_name missing! -0001-11-30` — Raed Hamed, Oct 18 '21 at 20:10

Raed Hamed · Accepted Answer · 2021-10-20T08:07:58.530

3

Thanks for Adrian Tompkins who lead me straight to the answer. I had to fix the time axis of the merged file used here as input. This is possible using the settaxis operator . Here an example of the full command:

cdo settaxis,date,time[,inc] infile outfile

edited Oct 20 '21 at 08:07

answered Oct 18 '21 at 20:35

Raed Hamed

327
1
10

Cool. Better to put the full command in your answer though. Essentially the netcdf files are not following CF conventions (naughty!) Showdate and showtime are really useful for spotting time axis issues. I feel another topic for a climate unboxed video coming on! – ClimateUnboxed Oct 18 '21 at 21:54
Thanks. Your YouTube channel is indeed very much recommended. I will keep an eye on these CF conventions. – Raed Hamed Oct 20 '21 at 08:09

ClimateUnboxed · Answer 2 · 2021-10-19T10:38:10.550

CDO uses the attribute proj_params to identify the proj parameters. The corresponding attribute in your file is proj4. You need to rename this attribute to apply the proj parameters automatically:

ncrename -a crs@proj4,proj_params file.nc

I found this answer here from Uwe Schulzweida

Once you have done that I hope you can process the file with cdo correctly, let me know if it works. If it doesn't please post the results of cdo showdate file.nc as this can help understand if CDO can process the date format correctly

EDIT: This solution doesn't solve the temporal processing issue posted by the OP but I will leave it up as it would anyway be needed to do any spatial processing of these files. NetCDF files using the coordinate reference system (CRS) need to ensure they are CF compliant for CDO to be able to handle them correctly.

Monthly sum of of a variable stored at daily timesteps using cdo (Climate Data Operators)

2 Answers2