1

I've found when trying to access a large(ish) array via OPeNDAP using Matlab and ncread, the result returns all zeros at some array size limit.
I'm trying to figure out if this is a Matlab/ncread limit or OPeNDAP issue.

I'm accessing a data set served via OPeNDAP (GrADS-DODS to be exact) via Matlab's ncread function.

  • If I select the entire array, which is 192 by 94 by 58676 (lon, lat, time), ncread does not throw an error but the resulting array is all zeros.

  • If I subset to, for example, the first 25,000 times, it works fine. I suspect there is a 2GB limit but would like confirmation. It also seems odd that there is no error, but rather a return of zeros.

% matlab R2018b 64-bit (maci64)

URL = 'http://apdrc.soest.hawaii.edu:80/dods/public_data/Reanalysis_Data/NCEP/NCEP2/6_hourly/gaussian_grid/uwnd_10m';

lat = ncread(URL,'lat');

lon = ncread(URL,'lon');

% this returns valid data

uwind = ncread(URL,'uwnd_10m',[1 1 1],[Inf Inf 29746]);

% this returns all zeros

uwind = ncread(URL,'uwnd_10m',[1 1 1],[Inf Inf 29747]);

% as does this (note there are 192x94x58676 values)

uwind = ncread(URL,'uwnd_10m');
Adam
  • 2,726
  • 1
  • 9
  • 22
JimP
  • 83
  • 5
  • Is it possible that there simply is not more data on the server? – Sven-Eric Krüger Aug 14 '19 at 09:53
  • What happens if you increment the start index too, so you fetch the same number of rows as the large subset which works, but including rows from the subset which doesn't? Note that there is a 2GB limit for mat files with default settings ([source](https://uk.mathworks.com/help/matlab/import_export/mat-file-versions.html)), so your diagnosis might make sense if there's some intermediate save process happening - MathWorks support could probably answer this for you – Wolfie Aug 14 '19 at 09:55
  • @Sven, yes, I've confirmed the data are okay on the server – JimP Aug 14 '19 at 15:53
  • @Wolfie, it seems to fail regardless of the start/end point if the total exceeds that threshold. I've sent a message to MathWorks and post here if I hear back. – JimP Aug 14 '19 at 15:54

0 Answers0