Questions tagged [netcdf4]

Questions related to NetCDF (Network Common Data Form) format and corresponding libraries/implementations may have this tag.

In 2008, the netCDF4 format was added to support per-variable compression, multiple unlimited dimensions, more complex data types, and better performance, by layering an enhanced netCDF access interface on top of the HDF5 format.

At the same time, a fourth format variant, netCDF-4 classic model format, was added for users who needed the performance benefits of the new format (such as compression) without the complexity of a new programming interface or enhanced data mode.

Every netCDF-4 or netCDF-4 classic model file can be read or written by the HDF5 library version 1.8 or later, so in that respect netCDF-4 files are already HDF5 files and need no conversion.

According to netCDF FAQ installing the netCDF4 is required for any of the following situations:

  • you need to access netCDF data that makes use of netCDF-4 compression or chunking
  • you need to access data in all netCDF formats including netCDF-4 or netCDF-4 classic model formats
  • you need to write non-record variables larger than 4GiB or record variables with more than 4GiB per record
  • you are installing netCDF to support other software packages that require netCDF-4 features
  • you want to write data that takes advantage of compression, chunking, or other netCDF-4 features
  • you want to be able to read netCDF-4 classic model data with no changes to your current software except relinking with the new library
  • you want to benchmark your current applications with the new libraries to determine whether the benefits are significant enough to justify the upgrade
  • you need to use parallel I/O with netCDF-4 or netCDF-4 classic files
651 questions
1
vote
1 answer

Fastest way to slice and download hundreds of NetCDF files from THREDDS/OPeNDap server

I am working with NASA-NEX-GDDP CMIP6 data. I currently have working code that individually opens and slices each file, however it takes days to download one variable for all model outputs and scenarios. My goal is to have all temperature and…
Clim Sci
  • 13
  • 3
1
vote
1 answer

Retrieving the path of a file opened to construct an xarray Dataset

If I open a dataset in xarray via import xarray as xr f = '/path/to/file.nc' dset = xr.open_dataset(f) Is there any way to later retrieve the source file location /path/to/file.nc from the object dset, say in a scope where f is no longer…
pretzlstyle
  • 2,774
  • 5
  • 23
  • 40
1
vote
1 answer

python netCDF4 module - how to get netCDF4.default_fillvals key for arbitrary numpy dtype?

I want to explicitly record the missing_value attribute for variables in a netCDF file I am writing with the netCDF4 module. I am aware that I can accomplish this by explicitly setting the fill_value argument to netCDF4.createVariable, and that…
1
vote
2 answers

Need help to extract the data of only one year from a .nc dataset in R

I need some help with a geospatial data project: I downloaded this dataset of yearly mean temperatures: https://psl.noaa.gov/data/gridded/data.UDel_AirT_Precip.html (this one) from which I need to data of 1990. I imported the data into R with the…
1
vote
0 answers

Extract chlorophyll a data from multiple netCDF files with multiple area of interest in python

I have a variety of chlorophyll_a netCDF files. I attempted to modify python coding from some examples that I have found, but it did not work because the example coding using the files contained the time and date data. It's possible that I'm new to…
Meeana C.
  • 21
  • 3
1
vote
1 answer

empirical orthogonal function with NCtoolkit

I am working with SST data obtained from https://podaac.jpl.nasa.gov/dataset/MUR-JPL-L4-GLOB-v4.1. I am looking at visualizing non-seasonal trends, such as those posed ENSO. right now I want to conduct an EOF analysis to decompose the dataset into…
1
vote
1 answer

Regridding from one irregular lat lon grid to another irregular lat lon grid

I have two sets of satellite data. For both sets, I have the pixel geometry (latitude and longitude of each corner of the pixel). I would like to regrid one set to the other. Thus, my goal is area-weighted regridding from an irregular grid to…
Nick
  • 11
  • 1
1
vote
2 answers

nco cut daily netcdf file from 10AM to 10AM everyday with duplicated 10am entries

I am a very beginner of nco, and I want to split my .nc file (from 1996010110 to 2019123110) as daily file, from 10AM to 10PM. In that case, each split file contains YYYY-MM-DD:10:00 to YYYY-MM-(DD+1):10:00. Note that the end hour of DD day is…
Xu Shan
  • 175
  • 3
  • 11
1
vote
2 answers

Create xarray datarray with dimension names equal to coordinate names

Dears, I need to create a xarray.datarray with the names of the dimensions equal to the names of the coordinates, however, I am not succeeding. Here is the code for reproduction: import numpy as np import xarray as xr data = [[23, 22, 21], …
1
vote
0 answers

Extracting data from Netcdf using defined spatial polygon

I followed the following example to extract data from Netcdf Link. It went all good but the problem I faced is to extract the data by spatial polygon (shapefiles). How do I extract the data (o3) using polygon shapefiles instead of lat/lon…
Lalantra
  • 67
  • 1
  • 11
1
vote
1 answer

Segfault caused by conflict between netCDF and HDF5 libraries

UPDATE: I've found a partial workaround. See bottom of this post. After a number of hours of debugging a program, I've found that there is some kind of conflict between the netCDF and HDF5 libraries (the program reads/writes files of both…
1
vote
1 answer

Error reading netCDF file from vobjtovarid4 using R

I want to get dimensions for my climate model, that is, for longitude, latitude, and time variables from netCDF file using the R code tas1 <- ncvar_get(climate_output, 'tas') dim(tas1) But I am getting error Error in vobjtovarid4(nc, varid,…
Ubaidulla
  • 13
  • 2
1
vote
0 answers

Unable to compile PnetCDF: 'tst_rec_vars' test fails

I am trying to install PnetCDF from source. I used the 1.12.2 version for installation. MPICH, Zlib, Szip, HDF5 and NetCDF-4 have been installed from source successfully (HDF5 and NetCDF-4 have been built with parallel I/O support). MPICH is…
RogUE
  • 321
  • 2
  • 16
1
vote
0 answers

Renamimg time dimension while using CRU dataset in Python

I have downloaded a CRU dataset as follows; data = netCDF4.Dataset('cru_ts4.04.2011.2019.tmn.dat.nc') print(data.variables.keys()) dict_keys(['lon', 'lat', 'time', 'tmn', 'stn']) The time variable has 108 entries given by 12 months times 9 years.…
jay
  • 1,319
  • 6
  • 23
  • 42
1
vote
1 answer

Clipping netCDF file to a shapefile and cloning the metadata variables in R

I have NetCDF files (e.g https://data.ceda.ac.uk/neodc/esacci/lakes/data/lake_products/L3S/v1.0/2019 global domain), and I want to extract the data based on a shapefile boundary ( in this case a Lake here -…
BeHana
  • 13
  • 3