Simultaneous shell and cdo operations on a directory with multiple netCDF files

Question

I have a directory, let's call it Simulation and a number of folder in the directory with some NetCDF file in each folder.

Simulation [Directory]:
       ----ACCESS [Folder]:
                  ------ Rootzone_ACCESS_rcp26_xyz.nc
                  ------ Rootzone_ACCESS_rcp45_xyz.nc
                  ------ Rootzone_ACCESS_rcp85_xyz.nc
       ----CCSM4 [Folder]:
                  ------ Rootzone_CCSM4_rcp45_xyz.nc
                  ------ Rootzone_CCSM4_rcp85_xyz.nc
       ----- [Other folders with NetCDF files]

I want to perform 2 operations on each file in all the folders. The operations are defined below.

#lets say test.nc is the each individual file
ncatted -O -a units,lon,c,c,"degrees_east" -a units,lat,a,c,"degrees_north" test.nc
# and I want to add '0.25res' at the end of each re-gridded file
cdo remapnn,r1440x720 test.nc test_0.25res.nc

Finally, I want to store all the re-gridded files in a different folder (let's call that folder Regridded_0.25) in the same 'Simulation` directory. The final result should look something like this.

Simulation [Directory]:
       ----ACCESS [Folder]:
                  ------ Rootzone_ACCESS_rcp26_xyz.nc
                  ------ Rootzone_ACCESS_rcp45_xyz.nc
                  ------ Rootzone_ACCESS_rcp85_xyz.nc
       ----CCSM4 [Folder]:
                  ------ Rootzone_CCSM4_rcp45_xyz.nc
                  ------ Rootzone_CCSM4_rcp85_xyz.nc
       ----- [Other folders with NetCDF files]

       -----Regridded_0.25 [Folder]:
                  ------ Rootzone_ACCESS_rcp26_xyz_0.25res.nc
                  ------ Rootzone_ACCESS_rcp45_xyz_0.25res.nc
                  ------ Rootzone_ACCESS_rcp85_xyz_0.25res.nc
                  ------ Rootzone_CCSM4_rcp45_xyz_0.25res.nc
                  ------ Rootzone_CCSM4_rcp85_xyz_0.25res.nc
                  ------ [Other NetCDF files]

Can someone help me create a shell script which can perform the following operations? I am quite inexperienced in large data operations in the shell, but this would significantly reduce the time. Greatly appreciated.

Sorry, it seemed like that. I did provide the code I want to perform in the question. It's just that I need guidance on operations in `shell`. — Ep1c1aN, Feb 27 '20 at 10:11
I would separate this task into two pieces: Write a shell script which gets the path to one file, and deals with this file correctly. You can test this script separately, which is also an advantage. Then write a script which uses `find` to traverse your directory tree and applies the script written in the first step to each file. — user1934428, Feb 27 '20 at 10:38
@user1934428...thanks for the idea. I managed to do the operations. — Ep1c1aN, Feb 27 '20 at 11:53

score -2 · Accepted Answer · edited Mar 24 '21 at 12:46

-2

I finally managed to do that in a single shell operation.

#!/bin/sh
for i in $(ls -R ./*/*.nc) ; do 
        #path=$i
        file=$(basename -s .nc $i)
        #echo $file
        ncatted -O -a units,lon,c,c,"degrees_east" -a units,lat,a,c,"degrees_north" ${i}
        cdo remapnn,r1440x720 ${i} ./Regridded/${file}_0.25res.nc; 
done

edited Mar 24 '21 at 12:46

ClimateUnboxed

7,106
3
41
86

answered Feb 27 '20 at 11:52

Ep1c1aN

683
9
25

Simultaneous shell and cdo operations on a directory with multiple netCDF files

1 Answers1