1

The ERA5 climate dataset is, in general, defined as a [lat, lon, time] matrix. Except for recent data, where it adds the exp_ver variable that indicate if the data is provisory (recent data, up to 3 months until the present, coded as "05") or old (stable version, coded as "01"), so the matrix are defined [lat, lon, time, exp_ver] only for recent data.

The exp_ver variable only has two values: "01" (old data) and "05" (recent data), and if there is data in "01" so the corresponding time has missing values in the "05" field, and viceversa. I'm looking for merge both "01" and "05" in a unique [lat, lon, time] matrix (so, remove the exp_ver variable), but I dont' know how to conduct this procedure. This is maybe an option:

A. Split the file with cdo splitlevel obtaining the "01" and "05" exp_ver separate files.

B. Remove the missing values segment, or only select the segment with data, from both files (I don't know how to to this!)

C. Remove the redundant variable "exp_ver" from both files (with cdo reduce_dim)

Any help with this? Thank you in advance!

Diego

ClimateUnboxed
  • 7,106
  • 3
  • 41
  • 86
Diego Hernandez
  • 35
  • 10
  • 1
    Excellent question. This always annoys me when you forget about it and then retrieve recent data and find this dimension. – ClimateUnboxed Mar 21 '22 at 04:06

1 Answers1

0

Can you use ncwa to average over and remove the dimension in step two and then use merge time?

cdo splitlevel # as you were suggesting

ncwa -a exp_ver v1.nc v1rd.nc
ncwa -a exp_ver v5.nc v5rd.nc 

cdo mergetime v1rd.nc v5rd.nc out.nc

Or thinking about it what if you use ncwa directly on the original file? It averages over that dimension. I don't recall how missing+value is handled though.

ncwa -a exp_ver era5.nc out.nc

Check out this link too which could be relevant Can ncwa (NCO) understand missing_value

( Preliminary answer posted from phone, will revise tomorrow)

ClimateUnboxed
  • 7,106
  • 3
  • 41
  • 86
  • Thank you! The process was killed... I didn't know the NCWA operators... Indeed your suggestion is similar to the R way I use: apply(mean, na.rm = T). Maybe with the mask condition, do you know how to write the "only not missing" as a logical condition with ncwa -B? – Diego Hernandez Mar 21 '22 at 05:41
  • I'll have a think about it, or maybe Charlie zender will see and reply in the meantime. He is the nco master! – ClimateUnboxed Mar 21 '22 at 06:04
  • 2
    If `cdo splitlevel` splits it by level, then I would assume that `cdo vertsum infile.nc outfile.nc` will sum over exp_ver and do what you want. – Robert Wilson Mar 21 '22 at 12:57
  • The `cdo vertsum` solves the issue. Thank you both! :) – Diego Hernandez Mar 21 '22 at 15:17
  • 1
    I do not fully understand the intent of the question, and am unfamiliar with this type of dataset. The ncwa commands in the OP will average over the exp_ver dimension, if present, and that dimension will not appear in the output file. If this does not occur, then something is weird. ncwa treats explicitly labeled _FillValue attributes as expected. – Charlie Zender Mar 21 '22 at 23:40