How to merge multiple nc files in linux using cdo

Question

I'm using Linux and I want to merge nc files into one. I have downloaded on this website. There are lots of netcdf files (more than 1000) that takes a lot of time to merge if merging manually, copy and paste one by one. I think about using vi command and it will automatically merge and create .sh file but don't know how to do.

Anyone can help merge nc file that take less time than doing manually?

Thanks alot.

Here is the text that using to download the files(1st time using this and the text begins from imns):

imns="01 02 03 04 05 06 07 08 09 10 11 12"  

for iyr in {2000..2019};do

    for imn in $imns;do
        for idy in {1..31}; do
            if [ $idy < 10 ]; then
                idy="0"$idy
            fi
            file="https://podaac-opendap.jpl.nasa.gov/opendap/allData/merged_alt/L4/cdr_grid/ssh_grids_v1812_"$iyr$imn$idy"12.nc"
        echo $file
        wget $file
        done
    done
done

If you want to concatenate in time, use `cdo mergetime * merged.nc`, where * represents the input filelist. You can also look at answers by Adrian Tomkins or Charlie Zender. For example: https://stackoverflow.com/questions/58167605/concatenate-netcdf-files-with-different-variables-using-nco/58178800#58178800. — msi_gerva, May 25 '20 at 06:23
I tried but it doesn't work, it could be due to the size of all files are too large (more than 20GB) which I want to merge all into one. So anyway could I do? — Minh Hiếu, May 25 '20 at 09:36
Did you try `ncrcat` (from the NCO Utilities)? I see that the files have the `Time` dimension set to be UNLIMITED, so `ncrcat` ought to work. — alani, May 25 '20 at 14:08

score 1 · Answer 1 · answered Jun 10 '20 at 09:55

As the previous responders have stated, mergetime from cdo is likely to do this. A note of caution is that because of the number of files you are merging, you may run into limits due to the operating system.

Try running:

ulimit -n

This will tell you the max number of files you can have open. If it is more than you are trying to merge, I recommend first merging subsets of the data, and then merge those.

How to merge multiple nc files in linux using cdo

1 Answers1