0

I want to open SST data from NOAA opendap, clip the region that I am interested in, load the variables of interest, do some summary stats, and then close the file. Because I want to do it for daily data over several years, I do not want to save the .nc files on my computer. I would prefer just open it with python, clip, summarise, and export.

This is how I do it without the Shapefile clipping part but with a ''box'' region. It works fine. However, I would like to define the region with Shapefile.

```
import netCDF4

#-- Define URLs for data access         
ncfile = 'https://podaac-opendap.jpl.nasa.gov/opendap/hyrax/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/2023/066/20230307090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'
fh = netCDF4.Dataset(ncfile)
time = fh.variables['time'][:]
lons = fh.variables['lon'][:]
lats = fh.variables['lat'][:]
    
#-- latitude and longitude boundaries
latbounds = [xxx, xxx]
lonbounds = [xxx, xxx]
    
latli = np.argmin( np.abs( lats - latbounds[0] ) )
latui = np.argmin( np.abs( lats - latbounds[1] ) ) 
lonli = np.argmin( np.abs( lons - lonbounds[0] ) )
lonui = np.argmin( np.abs( lons - lonbounds[1] ) ) 
    
sst_subset = fh.variables['analysed_sst'][ 0 , latli:latui , lonli:lonui ] 
sst_subset = sst_subset - 273.15
mean_sst = np.mean(sst_subset)
lons_subset = fh.variables['lon'][lonli:lonui]
lats_subset = fh.variables['lat'][latli:latui]
fh.close()
```

I have tried the following code that I found on a similar topics here. It works if I download and save the .nc file on my computer but it does not work if I open it directly from the url.

```
import xarray as xr
import rioxarray
import geopandas as gpd
from shapely.geometry import mapping

# Load Netcdf file
ncfile = 'https://podaac-opendap.jpl.nasa.gov/opendap/hyrax/allData/ghrsst/data/GDS2/L4/GLOB/JPL/MUR/v4.1/2023/066/20230307090000-JPL-L4_GHRSST-SSTfnd-MUR-GLOB-v02.0-fv04.1.nc'
fh = xr.open_dataset(ncfile)
fh.rio.set_spatial_dims(x_dim='lon', y_dim='lat', inplace=True)
fh.rio.write_crs('EPSG:4326', inplace=True)

# Load shapefile
lme = gpd.read_file('LMEs66.shp') # source: https://www.sciencebase.gov/catalog/item/55c77722e4b08400b1fd8244
lme = lme[lme['LME_NUMBER'] == 10] # Subsetting LME

# Clip Netcdf (SST) by shapefile (LME)
clipped = fh.rio.clip(lme.geometry.apply(mapping), lme.crs)
clipped.to_netcdf('mytest_clipped.nc')
```

I get this error when opening from the url:

"oc_open: server error retrieving url: code=? message="Error { 
    code = 500;
    message = "Unable to process <BESError> object in stream.";
}

Thanks for you help.

ArnoLB
  • 77
  • 7
  • What do you mean by "not download"? I don't see how you can possibly do this without downloading data. What you could do is find out if NOAA supplies this data over opendap, which would mean you only need to download a limited part of the data. – Robert Wilson Mar 10 '23 at 08:46
  • By download I mean to not save the .nc file on my computer. Indeed I access the data from opendap and select region of interest using a box (max/min lat and long), as shown in first part of the code, but I would like to select the region based on a polygon from a Shapefile. – ArnoLB Mar 10 '23 at 08:53
  • OK. The Q is confusing in that case, and you should rewrite to state you are having problems with opendap in this case. Can you make the error reproducible? Your code references a file on your system, so it's not possible to reproduce – Robert Wilson Mar 10 '23 at 10:49
  • Thanks Robert. I edited the title and the post to clarify. also provided the source of the Shapefile. – ArnoLB Mar 10 '23 at 11:11
  • Have you tried to download the file for one day and process it locally? If it works, you could just remove that file before proceeding to next day? – Serge Ballesta Mar 10 '23 at 11:32
  • 1
    Also just noting that nans are the same size as other floats, so you aren’t saving any memory by clipping to a shapefile relative to clipping to a box. So you could process the whole box and then clip to shapefile once you’re done – Michael Delgado Mar 10 '23 at 11:58
  • Second @MichaelDelgado's suggestion. I don't know the inner workings of the packages doing the clipping, but I wouldn't be surprised if they would actually download a box anyway prior to clipping. – Robert Wilson Mar 10 '23 at 16:21

0 Answers0