0

Hello, everyone, this is my first question here ^^

I need to regrid some NetCDF files, so I've tried to do it with cdo and cf modules, but everything failed. Maybe someone knows some other modules for this purposes or can help me to get rid of my errors. First, I've tried cf:

df = cf.read('my_file.nc')[0]

#define new grid
lat1 = cf.DimensionCoordinate(data=cf.Data(np.arange(-90, 90.01, 0.1), 'degreesN'))
lon1 = cf.DimensionCoordinate(data=cf.Data(np.arange(-180, 180.01, 0.1), 'degreesE'))

#regrid
g = df.regrids({'lat': lat1, 'lon': lon1}, method='linear')

But it gave me: "RuntimeError: Regridding methods will not work unless the ESMF library is installed". And I've got no idea how to install it to the remote server where I'm working on.

Then I've tried cdo:

#define input and output files
infile='my_file.nc'
outfile='newgrid.nc' 

#file with needed grid parameters 
gridfile='gridfile.txt'

#bilinear regrid
cdo.remapbil('gridfile.txt', input=infile, output=outfile)

And it gave me: "... STDERR:Error (cdf_load_bounds) : Allocation of 134369280000 bytes failed. [ line 2048 file stream_cdf_i.c ] System error message : Cannot allocate memory"

My file is about 2 gb, how it comes 134 gb to be needed for this operation?

I've also tried xarray:

da_input = xarray.open_dataarray('my_file.nc') # the file the data will be loaded from
regrid_axis = np.arange(-90, 90, 0.1) # new coordinates
da_output = da_input.interp(latitude=regrid_axis) # specify calculation
da_output.to_netcdf('output.nc') # save direct to file

It worked, but I suppose that it does interpolation just on the cartesian plane and I need it to be on sphere.

Thank everyone in advance

Upd: Thank you for all your answers. Turned out that the problem was with my file (resolution too high 300x300m). So I splitted it to some smaller files (for this operation cdo.sellonlatbox also needed 134 gb, but xarray.sel did it), then regridded with cdo and merged the result.

vakuoka
  • 1
  • 2
  • hi, agree with Rob's comments, it is likely to be your target grid specification which is causing the issue, can you post your gridfile.txt file in your question please? also, please read tag metadata, cdo is for the eclipse framework, you need cdo-climate ;-) – ClimateUnboxed Nov 08 '21 at 15:09
  • CDO has to, at a minimum, have at least one horizontal slice in RAM. So has been mis-specified, e.g. the resolution is far too high, then it will likely crash most machines – Robert Wilson Nov 09 '21 at 10:58

1 Answers1

1

You could try solving this with nctoolkit, which uses CDO as a backend:

import nctoolkit as nc
ds = nc.open_data("my_file.nc")
ds.to_latlon(lon = [-180, 180], lat = [-90, 90], res = 0.1)
ds.to_nc("output.nc")

If your file is 2 GB, then presumably you have made a mistake with the grid file in your CDO example. It shouldn't be trying to allocate that much RAM. nctoolkit will generate the appropriate CDO grid file and commands for you.

Robert Wilson
  • 3,192
  • 11
  • 19